Image stitching algorithm does not work and I don't know why

I have to combine multiple images of grapevines. Now I'm using the default matlab algorithm: http://es.mathworks.com/help/vision/examples/feature-based-panoramic-image-stitching.html * NOTE: This algorithm uses SURF features, I don't know if there is other keypoints better for me to do this.

Here the two images that I have to combine:

----------------------------------------------------------------------------------------------------------------------------

This algorithm do not work at all for my images. Each time I execute it, the output is different. Most of times the output has no sense, sometimes the output is a little bit better, but not correct. *It's important to say that I resize the images to 30% of its original size (due my computer have only 4GB of memory) Here some of the results that I have obtained:

-------------------------------------------------------------------------------------

As you can see, the panorama is not very good.

I have tried to remove (paint black) all pixels except those who are overlapped from one image to other, and also I have tried to process the image removing all pixels except the thin brawn branches (applying a color threshold and then eroding) but the result is even worse.

What operations can I do with my images to improve the panorama? Maybe change the keypoints detection algorithm? (current is SURF features)

Answer 1

[Dima Lisin]

Use the showMatchedFeatures function to visualize the point matches between pairs of images. What you want is to have as few incorrect matches as possible. Since there is not much rotation between the images, try setting the 'Upright' parameter of extractFeatures to true. Then the SURF descriptors will not be normalized for rotation, making them more specific. Also, try tweaking the parameters of matchFeatures. For example, decreasing MaxRatio may help to get rid of some of the outlier matches. On the other hand, if you have too few matches to begin with, you may want to increase MaxRatio.

estimateGeometricTransform is using RANSAC, a randomized algorithm to fit the transformation in the presence of outliers. This is why the transformation will be different each time. The fact that the difference is so noticeable could be because you have too few matches, or too many of the matches are outliers. Once you get your matches to be reasonably good, if you still get very different results every time you run your code, you may want to tweak the parameters of estimateGeometricTransform.

Answer 2

[sad asd]

I have tested setting 'Upright' to True in extractFeatures and MaxRatio to 0.4 in matchFeatures. Displaying the matched points, this is the result. As you can see there are a few matches that has no sense, I have surrounded two of them with blue:

These "errors" don't let the geometric transform work fine. To solve this I have filtered the matchings: I remove all pairs of points whose Y component difference is higher than the Image's height / 10. For example: For a pair of points {p1, p2} which represents a matching: [pseudocode] if( abs(p1.Y - p2.Y) > image.height / 10) remove the pair. [/pseoudocode]

After that, the result is this:

And the panorama is:

I think this panorame is much better than the previous, enough for the things I have to do with it.

I have also tested this algorithm with the second and third image (the previous images were 1 and 2). The result is worse. I think this could be because I haven't "tweaked" the estimageGeometricTransform, but I don't know how to do this. Anyway, the current images, matching points and results are:

Images: second (which is actually the first in this case):

and the third (which is actually the second in this case):

Mathing points:

And the two results, which I obtain randomly each time I execute the algorithm, are:

and

As you can see, the results are not very good again.

[Dima Lisin]

What type of transform are you specifying in estimateGeometricTransform? If you are using 'projective', try using 'affine' instead. It has fewer degrees of freedom, and it may give you a better fit.