How to find projective transformation with 4 points?

Hi all, I'm trying to get the Bird's-eye view from some sport images. I found a problem in a particular image. For an image I find points corresponding to known court points and apply 'fitgeotrans' command to get the mapping between the 2D image and the court model. For example:

% These are the points in the image
pa = [425 258]';
pb = [768 277]';
pc = [694 365]';
pd = [309 342]'; 

% Refence Points from Court Model
A = [11 10]'; 
B = [255 10]';
C = [255 234]';
D = [11 234]';

pin = [pa pb pc pd]; % 2xN matrix of inputs
pout = [A B C D]; % 2xN matrix of output
H = fitgeotrans(pin',pout', 'projective');
[Iwarp, ref] = imwarp(I,H);

Attached are 2 examples. One where this code works, and another where it does not. It is clear that I am missing something.

Why the method does not work in second case?

First case

Reference point for both cases:

Second case

[Matt J]

There are 5 reference points marked in your first image. Which 4 of them did you use for the second case? It looks like it would be very hard to measure them accurately in that case. Two of the five points even appear to be blocked by people standing in the way.

[Brokenp]

I used the point from P1 ti P4. The problem I am dealing with is notizie about accuracy, that will come later.

Right now i have the problem that the second projective transformation does not work

[Matt J]

Right now i have the problem that the second projective transformation does not work and I di not understand why.

If you don't understand why, then how are you so convinced that accuracy isn't causing this problem? You only have 4 points and a very strong perspective angle to rectify. The coordinate data would have to be very accurate.

Also, I still don't see how you're identifying point P2 and P4 in the second image. There are people standing in the way! Please mark where you think P2 and P4 are located in the second image.

Answer 1

[Matt J]

The points A,B,C,D in your court model are in counter-clockwise order, whereas P1,P2,P3,P4 in the images are labelled in clock-wise order. Mismatching the ordering of the points could account for the reflection seen in the second image (the red seats spectator area are initially on top, but in the transformed image are on the bottom).

Here's what I get from the following code. Note that you should also use imwarp with the 'OutputView' option to make sure the image boundaries contain the features you are interested in. (I don't know how imwarp chooses the default OutputView.)

I=imread('Court2.jpg');

% These are the points in the image
pa = [454  648]';
pb = [1223 640]';
pc = [1248 586]';
pd = [671 594]'; 
% Refence Points from Court Model
A = [11 10]'; 
B = [255 10]';
C = [255 234]';
D = [11 234]';

pin = [pa pb pc pd]; % 2xN matrix of inputs
pout = [A B C D]; % 2xN matrix of output

     pout(2,:)=500-pout(2,:); %flip to agree with pin

H = fitgeotrans(pin',pout','projective');
[Iwarp, ref] = imwarp(I,H,'OutputView',imref2d(size(I)));

Answer 2

[Matt J]

I suspect it's because you have mis-identified point correspondences in the two images.

The court in the first image has two red rectangles bordering the key (the yellow region under the basket). You've put P3 and P4 at the corners of the red rectangles as landmarks. However, the court in the second image does not have the red rectangles, or at least they are very hard to see at that camera angle, so it's hard to see how you are identifying corresponding points P3 and P4 in the second image.

Somewhat similarly, your points P1 and P2 are on the far sideline of the court. But it is nearly impossible to locate the sideline in the second image. You can't see the darker flooring beyond it like in the first image, so it's hard to tell accurately where the court ends and the seating area begins.

On top of that, point P2 in the first image is the point on the sideline that is co-linear with the free-throw line. But our view of that point is blocked by a player in the second image, so I don't see how you are locating P2 there. If you could reliably identify the sideline, I suppose you could indirectly get P2 by drawing an extension of the free-throw line to where it meets the sideline, but again, I think it's hard to tell here where the sideline is.

Finally, using only four points doesn't give you very much data redundancy. The effect of errors in the coordinate measurements could be very much magnified because of that.

[Brokenp]

First of all, thank you for the answer.

The point selection is done in this way:

1. Detect lines: baseline, free throw line, sideline etc
2. Extend them and get the intersection points

In my mind getting the point that way solve the problem of a player blocking the the view, but I may be totally wrong.

See the image attached(Note that I have omitted the 5th point)

Correct me if I'm wrong, but from your answer I understand that if I detect the points in a more precise way I should solve the problem. Right?

[Matt J]

Correct me if I'm wrong, but from your answer I understand that if I detect the points in a more precise way I should solve the problem. Right?

No, forget it. See my other answer.