How can i apply a function to an image to make it find detected figure shapes ?

Hi!

I've got image with diffirent types of shapes;

I've used imcontour function to find contours of shapes in image;

Now i've got like (contour?) of circles/squares/.... shape in image;

I know that function1 graph is "circle", function2 graph is "square".... (it's written down why "circle" and "square" are in commas)

I've got no ideas how to make MatLab work in following way: I've got a function, who's graph reminds "circle", let it be function1, i've got contours of a "circle" shape (done with imcontour) in image, so now i wanna to make MatLab to find and to count all contours of "circle" shapes in image, who's contour(shape) can be described with function1

And the problem is that those chapes, are not exactly circles/squares, they are like "transitional" circle/square form, like it's not a circle/square, only it's contour reminds of circle/square shape, and i have to work exactly with those shapes, so imfindcircles and boundingbox, is not what i'am looking for (cause they just draw "pure" circles/squares, even there where it's not a circle/square exactly) (as i understood)

Since i have no permission to give any photos, it would be great for me to get ANY help...

P.S. plz be patient, i'm kind of a MatLab lamer

Answer 1

[Image Analyst]

See my attached shape recognition demos.

[Jose Rosendo]

Hi, just to add on this. I have an image with 5 coins and I want to select one of them and rotate it. The way I'm doing it is to crop the coin I select and apply a rotation and after trying to insert it back into the original image, which I'm failing to do.

for k = 1 : num
                thisBlob = ismember(lb, k);
                measurements = props(k);
                croppedImage = imcrop(img, measurements.BoundingBox);
                % Compute angle from measurements.Orientation
                angle = 25;
                    % Rotate image
                uprightImage = imrotate(croppedImage, angle);
                %erasedCent = immultiply(~croppedImage, imgOrig);
                %cropG = rgb2gray(uprightImage);
                %K = imadd(I,J,'uint16');
                newImage = imadd(img, uprightImage);
            end
            figure;
           imshow(newImage);

That is the code and the error I get is :

Error using imadd (line 72)

X and Y must have the same size and class or Y must be a scalar double.

Error in Project1 (line 312)

newImage = imadd(img, uprightImage);

Am I doing this right? or is there an easier way?

Thank you

[Image Analyst]

No, you have to paste it back where it was, so at its centroid. See my attached copy and paste demos.

[Dmitry Surov]

Thanks for your demo examples. Also thanks for explanations for every command. It really helped a lot.

You helped me to move on, but i still got stucked with my primodial question - the problem as i see it, looks smth like that : you're using like built-in-functions, when you type rectangle/square/hegaxon, MatLab knows these functions, and by using them you've created smth like a shape detection code, but only for "right" shapes.

So as i undestand your code - it won't give a right classification for the second function's graph: (i've attached a picture of two graph functions for example, just for comfort reading) -

1) For function r = 7 (it's graph in polar coordinates is a circle with radius = 7) and your code will work for it, and classify function's (r = 7 ) graph as a circle;

2) For function r = 3-3cos(thetha) (it's graph in polar coordinates is irregular shape, so it's closer pretending to be a circle, than a square or a hexagon, and your code (as i understood) with classify this shape as "smth like a circle" or smth else, choosing between built-in-commmands of MatLab (hexagon/square/rectangle/....) ;

So now i'll try to explain my problem as best as i can:

I've got a picture with pure circles (just for an example), this picture contains:

2 circles with radius value 3 and 8 circles with radius value 15;

I know that for function: r = 3 (in polar coordinates), this function's graph will be a circle with radius 3;

Now using only this mathematical function (r=3 (polar coordinates)) i wanna MatLab to find in my image all shapes, that relates to my mathematical function's (r=3 (polar coordinates)) graph ( a circle with radius value 3) (so MatLab should find only 2 circles of 10 existing cicrles in image)

I can't use any of built-in functions, because shape's that i've got, reminds smth of a r = 3-3cos(thetha) graph, and since MatLab doesn't have this shape as a built-in-functions, it won't classify it purely right (it will classify as smth else or pretending to be a circle, but that's not what i need).

Is it even possible to do like that, using just function's graph ?

When i was looking for any help with my problem in internet, i was told that i first of all should work in Image Procceing ToolBox, and i should use an "MHK Optimization ToolBox" for, i don't know how to say it right, like for searching/picking up/guessing form's parameters. So maybe i can do it somehow in optimization toolbox ?

Since i can't ask you to just give me a code, cause that would be unfair, and it won't be my solution, i can ask you only to point me am i on the right way or not.

Thanks in advance.

[Image Analyst]

Please define "smth" -- I don't know that word and it's preventing me from understanding your comment completely.

So are you dealing with a digital image, or an analytical function? If you have a function, can you convert it to an image with poly2mask()? Because I don't know how analyze a function. I need an image, or a list of (x,y) coordinates that we can turn into an image with poly2mask.

I am not using built-in functions for my polygons. I wrote the function from lower level built-in functions in base MATLAB and the Image Processing Toolbox.

Basically the way to characterize shape (at least one way) is to count the vertices. You can find the distance of the centroid to the boundary and plot it and use findpeaks(). Wherever there is a vertex there is a peak in the distances profile. If you don't have nice polygons but some arbitrary, irregular "splat" shape then you'll have to come up with your own algorithm for how you'd like to characterize the shape. Maybe you could use deep learning to train a network to classify your shapes into some finite number of known, predetermined, predefined shape classes, like circle, cycloid, asterisk, heart, dumbbell, etc.

[Dmitry Surov]

I've got 3 diffirent shapes in image

The red one can be described with r1 = 3-3cos(thetha) (polar coordinate system)

The blue one can be described with r2 = tan(sin(2*thetha)) (polar coordinate system)

The green one can be described with r3 = 5-4sin(2*thetha) (polar coordinate system)

I used standard built in function's imread, imshow, so i can work further

I also tried to make my original image be a binary image (that worked, but's all i got)

So

I give MatLab as in imput an image with 3 diffrent shapes, every shape can be described with it's own mathematical function (wrote about it in the begining)

I've got 3 functions, and also i know how their graphs in polar coordinate look like

So now, i wanna make MatLab to work in the following way:

1) MatLab already got image with 3 different shapes as an imput;

2) I've wrote down in a live script all 3 mathematical function, that can describe my shapes in image (function's graph in polar coordinate has the same shape as the shape in image)

3) I don't understand completely how, but MatLab should do the following ( he sees a red shape in image, then he should somehow know, that function's r1=3-3cos(thetha) graph shape is the same shape as he sees in image (the red shape in image))

4), then he should find out that the red shape in image, can be described with r1=3-3cos(thetha) function and then he should say that red shape in image can be described with function r1=3-3cos(thetha) and tell me about this

Is this possible to do as i am asking or not?

I've tried to do poly2mask as you said, i've got pairs of (x,y) but i misunderstand what to do next

And the last thing - all these 3 shapes in image are not red/blue/green, they are grey, i've used colours in desmos just to say that there are 3 different shapes and every shape can be desctibed with it's mathematical functions ( to do that, just fo clearness, i've made them red/green/blue, so that's can't be related to a rgb image, it's just line colours in desmos)

Thanks in advance

[Dmitry Surov]

I'll apply you a photo which i'am working with - everything i need to do - is written there

[Image Analyst]

I would threshold the gray scale image to produce a binary image. Then I'd call regionprops() to get the centroid of the shape. Then I'd call bwboundaries() to get the outline. Then I'd subtract the centroid so you can scan the outline to get it in polar coordinates. Then you'll probably have to scale it to from 0 to 3 with rescale() to make it match up with the amplitudes of your reference "r" curves. Then I'd compare the residuals of that test radial profile to the radial profiles of your reference curves. The one with the lowest residuals is the one it is - it's the best match. Here's a start

mask = grayImage > someValue; % or < depending on if blob is brighter or darker than background.
mask = bwareafilt(mask, 1); % Take largest blob only.
props = regionprops(mask, 'Centroid');
boundary = bwboundaries(mask);
boundary = boundary{1}; % (row, column) format, not (x, y).
x = boundary(:, 2) - props.Centroid(1);
y = boundary(:, 1) - props.Centroid(2);
radius = sqrt(x.^2 + y.^2);
angles = atand(y ./ x);
radius = rescale(radius, 0, 3);
plot(angles, radius, 'b-', 'LineWidth', 2);
grid on;

I'm sure your course doesn't want me to do the whole thing for you to turn in as your own so I'll let you finish it.

[Dmitry Surov]

Thanks for your advice, btw what do you think - can i use image processing toolbox when i'll have like a hundred different shapes and a hundred corresponding functions ?

Or in this case i should use neural network ?

Also if it's better to use neural network - then all the stuff i did in image processing toolbox can't be used in neural network ? Or there'll be some stuff that i can use in neural network toolbox ?

And the last one - where is that limit of shape numbers and their corresponding functions ? At which point it's better to use neural network toolbox instead of image processing ?

Thanks

[Image Analyst]

You can do it either way. With traditional methods you have to have knowledge of the characteristics of each shape and program those in. With Deep Learning, you don't need to know those but you have to have numerous examples of each shape that you can use to train the algorithm.

[Dmitry Surov]

Thanks.

By the way - what do you mean by "The one with the lowest residuals is the one it is - it's the "BEST MATCH""?

I'll provide a picture - in this picture the contour of a shape is just deleted by an eraser in paint, so my image can contain shapes, whos contour's can look the same (like it's contour can go go go permanently, and then there can be a sudden "gap" and then contour line keeps continious) - would that code relate the shapes to its corresponding function in this case or not ?

[Image Analyst]

I mean you compute the sum of the distances from your actual shape to the reference shape. The shape it is should have the lowest distance. For example the residuals from a circle to a slightly elliptical version of the circle will be less than if you compared it to a square or asterisk shape.

It might depend on how broken up the shape is and how much clutter (grid, etc.) degrade the image. Don't you have any control over the images? Where do they come from?

You could train a classification network, CNN, to find the shapes, but again, you're going to have to provide a bunch of examples of the shape with various amounts of degradation. You can synthesize additional images with augmentation where you can vary things like the shape and size and color of the images.

[Dmitry Surov]

1) I've produced my image to binary;

2) I've called regionprops() and got the centroid of the shape;

3) I've called bwboundaries to get the outline (not of the contour of the shape but like a square line inside which my shape is);

I compelety misunderstand your next step "Then I'd SUBSTRACT the centroid so you can SCAN the OUTLINE to get it in polar coordinates) - i've no idea, what you mean by this "SUBSTRACT" - can you give any examples of this ? (or maybe there are some examples in your demos that i missed) (or maybe there exist a buit in function in MatLab to do this operation).

About - how shape contour can be broken - well, there's no rule, that rules this situation, like it can be broken 2,3 or 5 times (just like 5 "gapes" in shape's contour). Image will contain no background grips at all. About clutter - well, it depends on the image, some have no clutter at all, some of them have lots of clutter, and it's really hard to get well-binarized image, cause no doubt - it will be binary - but the quality of the shapes in this situation "leaves much to be desired" - they (contour lines of the shape) can have lot's of "gaps", or even 2 shapes, situated near to each other in grayscale image, can turn into one shape in binary image.

Those shapes come from the picture of the silicon's surface, after it gets etched - so i have no control over the images - i only receive them to work with and that's all.

Thanks

[Image Analyst]

See this and adapt as needed:

clc;    % Clear the command window.
fprintf('Beginning to run %s.m ...\n', mfilename);
close all;  % Close all figures (except those of imtool.)
clear;  % Erase all existing variables. Or clearvars if you want.
workspace;  % Make sure the workspace panel is showing.
format long g;
format compact;
fontSize = 22;

%--------------------------------------------------------------------------------------------------------
%    READ IN IMAGE
folder = pwd;
baseFileName = 'eight.tif';
% Get the full filename, with path prepended.
fullFileName = fullfile(folder, baseFileName);
% Check if file exists.
if ~exist(fullFileName, 'file')
	% The file doesn't exist -- didn't find it there in that folder.
	% Check the entire search path (other folders) for the file by stripping off the folder.
	fullFileNameOnSearchPath = baseFileName; % No path this time.
	if ~exist(fullFileNameOnSearchPath, 'file')
		% Still didn't find it.  Alert user.
		errorMessage = sprintf('Error: %s does not exist in the search path folders.', fullFileName);
		uiwait(warndlg(errorMessage));
		return;
	else
		% It found it somewhere.
		folder = fileparts(which(fullFileNameOnSearchPath)); % Determine where folder is.
		fullFileName = fullfile(folder, baseFileName);
	end
end
grayImage = imread(fullFileName);


% Get the dimensions of the image.
% numberOfColorChannels should be = 1 for a gray scale image, and 3 for an RGB color image.
[rows, columns, numberOfColorChannels] = size(grayImage);
if numberOfColorChannels > 1
	% It's not really gray scale like we expected - it's color.
	% Use weighted sum of ALL channels to create a gray scale image.
	grayImage = rgb2gray(grayImage);
	% ALTERNATE METHOD: Convert it to gray scale by taking only the green channel,
	% which in a typical snapshot will be the least noisy channel.
	% grayImage = grayImage(:, :, 2); % Take green channel.
end
% Display the image.
subplot(2, 2, 1);
imshow(grayImage, []);
title('Original Image', 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;
hFig = gcf;
hFig.WindowState = 'maximized'; % May not work in earlier versions of MATLAB.
drawnow;

% This image needs to be inverted so that the bolt is white
mask = ~imbinarize(grayImage);

%--------------------------------------------------------------------------------------------------------
%    SEGMENTATION OF IMAGE
% Fill Holes
mask = imfill(mask, 'holes');
% Extract only blobs larger than 1000.
mask = bwareaopen(mask, 1000, 4); % Connectivity of 4
% Display the mask image.
subplot(2, 2, 2);
imshow(mask);
caption = sprintf('Mask');
title(caption, 'FontSize', fontSize, 'Interpreter', 'None');
impixelinfo;
drawnow;

% Get the centroids
props = regionprops(mask, 'Centroid');

% Get the boundaries
boundaries = bwboundaries(mask);
% Get the radius and angle for each boundary
% and add them to props structure.
for k = 1 : length(boundaries)
	thisBoundary = boundaries{k};
	props(k).x = thisBoundary(:, 2);
	props(k).y = thisBoundary(:, 1);
	props(k).xCentroid = props(k).Centroid(1);
	props(k).yCentroid = props(k).Centroid(2);
	subplot(2, 2, 2);
	hold on;
	plot(props(k).xCentroid, props(k).yCentroid, 'r+', 'MarkerSize', 50, 'LineWidth', 2);
	% Get the radius and angle.
	props(k).radius = sqrt((props(k).x - props(k).xCentroid) .^ 2 + (props(k).y - props(k).yCentroid) .^ 2);
	numerator = props(k).y - props(k).yCentroid;
	denominator = props(k).x - props(k).xCentroid;
	props(k).angles = atand(numerator ./ denominator);
	
	% Plot radii.
	subplot(2, 2, 3);
	plot(props(k).radius, '-', 'LineWidth', 2);
	grid on;
	xlabel('Pixel Index', 'FontSize', fontSize);
	ylabel('Radius', 'FontSize', fontSize);
	title('Radius', 'FontSize', fontSize);
	hold on;
	
	% Plot angles.
	subplot(2, 2, 4);
	plot(props(k).angles, '-', 'LineWidth', 2);
	grid on;
	xlabel('Pixel Index', 'FontSize', fontSize);
	ylabel('Angle', 'FontSize', fontSize);
	title('Angle', 'FontSize', fontSize);
	hold on;
end

fprintf('Done running %s.m ...\n', mfilename);