Rectify images from 2x2 camera matrix

I have a camera system with a 2x2 camera matrix, which I have used for an experiment. I'd like to rectify (I believe that's the term) all these images, so that a given pixel in any image corresponds to the same(-ish) point in the real world. I would also like this process to be done as automatically as possible, since I have a lot of images I need corrected.

All images are the same size, approx 1.5k by 2k pixels, and in 8-bit greyscale bitdepth. Each of the objects photographed have been marked with 9 dots in a 3x3 matrix (see image below).

So far I've tried a few different approaches:

• Manual rectification, i.e. letting the user mark the 9 dots in the image and then using each dot's coordinate in each of the four images to correct the image (by just shifting the pixels). This produces decent results but is VERY slow.
• Different stereo rectification tutorials. These work to some extent, however I haven't found one that works with the diagonal images (1 and 4 in the image below).

I think the best approach would be if you somehow could identify the 9 dots in each image (automatically) and the shift/crop each image accordingly. Since each cameras orientation is approx the same I don't think any transformation is needed. However, I'm unable to identify the 9 dots without getting a lot of false positives.

Sketch of problem:

Example image (with 3x3 dot-matrix):

Answer 1

[Image Analyst]

I've always heard it called "registration". Which basically means rotating, scaling, translating, whatever so that two or more images are where they "should" be. There is a function imregister(), plus some other related functions that handle greater misalignments.

You could try normalized cross correlation if it's just a simple translation. I attach a demo.

Or you could use SURF and other features in the Computer Vision System Toolbox: see this for examples http://www.mathworks.com/products/computer-vision/code-examples.html

[Isak Strömberg]

Thanks for a great answer!

Now I've tried the imregister function, as displayed on this page. However the results are not good enough. I believe this is because some areas of the image are in a different depth plane than the interesting area of the image, and offsets the calculations made by imregister().

I also modified and tried the example you linked using normxcorr on a sample of the image. These results are better, but unfortunatly still not good enough. Also it's quite dependant on how the sample is chosen. This however I might be able to solve with some more testing.

Maybe a simple translation (linear x- and y-shift) might not be able to produce good enough results. I've looked a bit at the Camera Calibration and Stereo Camera Calibration toolboxes, but I'm not sure how to apply them on my 4-camera setup. I've not been able to understand how to calculation the translation matrix from the camera parameters.

Right now it seems that either this or a solution where MATLAB automatically detects the 9 dots and performs the same calculations as in my manual script is the best solution.

Here my results so far:

NormXCorr2 (as you can see the 9 dots are not at all aligned):

Manual registration (i.e. marking the 9 dots and calculating the x- and y-shift using their positions. These results are what I wish to accomplish with an automatic solution):

Imregister (too bad results here as well):