Estimate camera projection matrix from world-to-image point correspondences
Load a 3-D point cloud data captured by an RGB-D sensor into the workspace.
ld = load('object3d.mat'); ptCloud = ld.ptCloud;
Remove points with
NaN coordinates from the point cloud.
[validPtCloud,validIndices] = removeInvalidPoints(ptCloud);
Read the valid world point coordinates. Each entry specifies the x, y, z coordinates of a point in the point cloud.
worldPoints = validPtCloud.Location;
Define the corresponding image point coordinates as a orthographic projection of point cloud data onto the yz-plane.
indices = 1:ptCloud.Count; [y,z] = ind2sub([size(ptCloud.Location,1),size(ptCloud.Location,2)],indices); imagePoints = [y(validIndices)' z(validIndices)'];
Generate the 2-D image projection by using the image point coordinates and their color values.
projImage = zeros(max(imagePoints(:,1)),max(imagePoints(:,2)),3); rgb = validPtCloud.Color; for j = 1:length(rgb) projImage(imagePoints(j,1),imagePoints(j,2),:) = rgb(j,:); end
Display the point cloud data and the corresponding 2-D image projection.
figure subplot(1,2,1) pcshow(ptCloud) xlabel('X'); ylabel('Y'); zlabel('Z'); title('Point Cloud Data','Color',[1 1 1]) subplot(1,2,2) imshow(uint8(projImage)) title('2-D Image Projection','Color',[1 1 1])
Estimate the camera projection matrix and reprojection error by using the known world points and the image points.
[camMatrix,reprojectionErrors] = estimateCameraMatrix(imagePoints,worldPoints);
Use the estimated camera projection matrix as input to the
findNearestNeighbors function and find the nearest neighbors of a query point.
point = [0.4 0.3 0.2]; % Specify the query point K = 50; % Specify the number of nearest neighbors to be determined [indices,dists] = findNearestNeighbors(ptCloud,point,K,camMatrix); % Get the indices and distances of nearest neighbors
select function to get the point cloud data of nearest neighbors.
ptCloudB = select(ptCloud,indices);
Display the input point cloud and its nearest neighbors.
figure, pcshow(ptCloud) hold on pcshow(ptCloudB.Location,'ob') hold off legend('Point Cloud','Nearest Neighbors','Location','southoutside','Color',[1 1 1])
imagePoints— Coordinates of image projection points
Coordinates of image projection points, specified as an M-by-2 matrix of (x, y) coordinates. M is the number of points and it must be greater than or equal to 6.
The input image points must correspond to an undistorted image plane.
worldPoints— 3-D world points
3-D world points, specified as an M-by-3 matrix of (x, y, z) coordinates. M is the number of points and it must be greater than or equal to 6.
The input world coordinates must be non-coplanar points.
camMatrix— Camera projection matrix
Camera projection matrix, returned as a 4-by-3 matrix. The matrix maps the 3-D world points, in homogenous coordinates to the 2-D image coordinates of the projections onto the image plane.
reprojectionErrors— Reprojection errors
Reprojection errors, returned as a M-by-1 vector. The reprojection error is the error between the reprojected image points and the input image points. For more information on the computation of reprojection errors, see Algorithms.
You can use the
estimateCameraMatrix function to estimate a camera projection
If the world-to-image point correspondences are known, and the camera intrinsics and extrinsics parameters are not known.
For use with the
findNearestNeighbors object function of the
pointCloud object. The use of a camera
projection matrix speeds up the nearest neighbors search in a point cloud generated by an
RGB-D sensor, such as Microsoft®
Given the world points X and the image points x, the camera projection matrix C, is obtained by solving the equation
λx = CX.
The equation is solved using the direct linear transformation (DLT) approach . This approach formulates a homogeneous linear system of equations, and the solution is obtained through generalized eigenvalue decomposition.
Because the image point coordinates are given in pixel values, the approach for computing the camera projection matrix is sensitive to numerical errors. To avoid numerical errors, the input image point coordinates are normalized, so that their centroid is at the origin. Also, the root mean squared distance of the image points from the origin is . These steps summarize the process for estimating the camera projection matrix.
Normalize the input image point coordinates with transform T.
Estimate camera projection matrix CN from the normalized input image points.
Compute the denormalized camera projection matrix C as CNT-1.
Compute the reprojected image point coordinates xE as CX.
Compute the reprojection errors as
reprojectionErrors = |x− xE|.
 Richard, H. and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge: Cambridge University Press, 2000.