Detecting objects by template

Hey, I'm working on an object detection. I have an image from multiple objects:

and want to find all screws, for example of this type:

Currently im trying to extract the screws edges using the canny algorithm. Then I want to use the generalized Hausdorff measure to search for it: http://www.cs.cornell.edu/vision/hausdorff/hausmatch.html

I'm not sure whether this is the right approach. Since I am a newbe maybe you can give me some hints for the best practices? Note that this should work for multiple objects and images. I tried using the image gradient and boundarys, too. But this failed because I could not get meaningful contours. It would be great to work with contour snippets, which would not make it neccessary to extract the whole object.

Thanks in advantage

Answer 1

[Image Analyst]

Try Hu's moments: https://www.youtube.com/watch?v=Nc06tlZAv_Q

But first you'll have to extract the objects. To do that you should use better lighting, for example bottom up light, like a radiologists light box, to produce silhouettes. Or else use a broad overhead light source to eliminate shadows.

[Anika Paulsen]

Thanks, i'll give it a try. The problem is, later on I want this to work with AR Gear. I can't take really good lightning for granted. I hope I can make this work with this defaut light too. Anyway maybe you also have a tip whether there is a chance to compute this in, let's not say realtime, but fast? Again, thanks for your quick reply, this video seems to be really helpful.

[Image Analyst]

See my demo for normxcorr2(), attached.

[Anika Paulsen]

Thanks, but this does not really work for me.

I thought about comparing the edges - searching for matching parts. Maybe this could make it possible to identify objects which are half concealed if some meaningful edge can be seen. Whats your opinion on that? I thought about using this to find the edges: https://www.mathworks.com/help/fuzzy/examples/fuzzy-logic-image-processing.html but i'm stuck comparing them to the template.

I also tried this tutorial: https://www.mathworks.com/help/vision/examples/object-detection-in-a-cluttered-scene-using-point-feature-matching.html but i couldnt find the screw.

[Image Analyst]

normxcorr2() doesn't work well for scaled or rotated templates. That's why I suggested Hu's moments in the very beginning. They are scale and rotation invariant. Please try them. I'm going to be traveling the upcoming week so I won't be able to help much, and I don't have any demo for Hu's moments, though I do have one for moments in general, attached.

[Anika Paulsen]

Thank you a lot! I will try Hu's moment and let you know.

[Anika Paulsen]

Hey, I am still working on Hu's Moment. I'm preprocessing my image using nonuniform backgound illumination. Unless there aren't really big shadows its okay. After using this: http://limitless-thoughts.blogspot.de/2011/05/hus-seven-moments-invariant-matlab-code.html to calculate the moments I have troubles comparing them. The guy in the video talks about using svd and the least squares fit. Can you give me a hint how to compare my values in matlab?

regionSize = size(regions);
matchIndex = 1;
for i = 1:regionSize(1)
     croppedImage = imcrop(binaryImage, [regions(i).BoundingBox]);          
     mask = bwareaopen(croppedImage, regions(i).Area - 1);
     %Compute Hu's moment 
     hu = invmoments(mask);

       tol = 0.1;
       LIA = ismembertol(hu,template_hu,tol)

       if(mean(LIA) > 0.9)
           matches(matchIndex) = regions(i);
           matchIndex =matchIndex + 1;
       end
  end

[Image Analyst]

Try increasing your tolerance. Also work on getting a better segmentation so that you don't have big black "bays" going into your objects. You may also have to compute a feature vector with several metrics rather than just rely on Hu's moments.

[Pete McEldowney]

I was also trying to figure out SVD and LSF from that video too - I'm still looking but all sources are very mathematical ...

Answer 2

[Pete McEldowney]

Those holes on the bolt masks look like reflectance - either fill using code, or alter the ilghting - make it more diffuse? Or from behind the scene. Harris corner features might work OK as extra identification aids.