(Not recommended) Estimate camera projection matrix from world-to-image point correspondences
Estimate Camera Projection Matrix
Load a 3-D point cloud data captured by an RGB-D sensor into the workspace.
ld = load('object3d.mat'); ptCloud = ld.ptCloud;
Remove points with
NaN coordinates from the point cloud.
[validPtCloud,validIndices] = removeInvalidPoints(ptCloud);
Read the valid world point coordinates. Each entry specifies the x, y, z coordinates of a point in the point cloud.
worldPoints = validPtCloud.Location;
Define the corresponding image point coordinates as a orthographic projection of point cloud data onto the yz-plane.
indices = 1:ptCloud.Count; [y,z] = ind2sub([size(ptCloud.Location,1),size(ptCloud.Location,2)],indices); imagePoints = [y(validIndices)' z(validIndices)'];
Generate the 2-D image projection by using the image point coordinates and their color values.
projImage = zeros(max(imagePoints(:,1)),max(imagePoints(:,2)),3); rgb = validPtCloud.Color; for j = 1:length(rgb) projImage(imagePoints(j,1),imagePoints(j,2),:) = rgb(j,:); end
Display the point cloud data and the corresponding 2-D image projection.
figure subplot(1,2,1) pcshow(ptCloud) xlabel('X'); ylabel('Y'); zlabel('Z'); title('Point Cloud Data','Color',[1 1 1]) subplot(1,2,2) imshow(uint8(projImage)) title('2-D Image Projection','Color',[1 1 1])
Estimate the camera projection matrix and reprojection error by using the known world points and the image points.
[camMatrix,reprojectionErrors] = estimateCameraMatrix(imagePoints,worldPoints);
Use the estimated camera projection matrix as input to the
findNearestNeighbors function and find the nearest neighbors of a query point.
point = [0.4 0.3 0.2]; % Specify the query point K = 1000; % Specify the number of nearest neighbors to be determined [indices,dists] = findNearestNeighbors(ptCloud,point,K,camMatrix); % Get the indices and distances of nearest neighbors
select function to get the point cloud data of nearest neighbors.
ptCloudB = select(ptCloud,indices);
Display the input point cloud and its nearest neighbors.
figure pcshow(ptCloud) hold on plot3(ptCloudB.Location(:,1),ptCloudB.Location(:,2),ptCloudB.Location(:,3),'*') hold off legend('Point Cloud','Nearest Neighbors','Location','southoutside','Color',[1 1 1])
imagePoints — Coordinates of image projection points
Coordinates of image projection points, specified as an M-by-2 matrix of (x, y) coordinates. M is the number of points and it must be greater than or equal to 6. The input image points must correspond to an undistorted image plane.
worldPoints — 3-D world points
3-D world points, specified as an M-by-3 matrix of (x, y, z) coordinates. M is the number of points and it must be greater than or equal to 6. The input world coordinates must be non-coplanar points.
camMatrix — Camera projection matrix
Camera projection matrix, returned as a 4-by-3 matrix. The matrix maps the 3-D world points, in homogenous coordinates to the 2-D image coordinates of the projections onto the image plane.
reprojectionErrors — Reprojection errors
Reprojection errors, returned as a M-by-1 vector. The reprojection error is the error between the reprojected image points and the input image points. For more information on the computation of reprojection errors, see Algorithms.
You can use the
estimateCameraMatrix function to estimate a camera projection
If the world-to-image point correspondences are known, and the camera intrinsics and extrinsics parameters are not known, you can use the
To compute 2-D image points from 3-D world points, refer to the equations in
For use with the
findNearestNeighborsobject function of the
pointCloudobject. The use of a camera projection matrix speeds up the nearest neighbors search in a point cloud generated by an RGB-D sensor, such as Microsoft® Kinect®.
Given the world points X and the image points x, the camera projection matrix C, is obtained by solving the equation
λx = CX.
The equation is solved using the direct linear transformation (DLT) approach . This approach formulates a homogeneous linear system of equations, and the solution is obtained through generalized eigenvalue decomposition.
Because the image point coordinates are given in pixel values, the approach for computing
the camera projection matrix is sensitive to numerical errors. To avoid numerical errors, the
input image point coordinates are normalized, so that their centroid is at the origin. Also,
the root mean squared distance of the image points from the origin is
sqrt(2). These steps summarize the process for estimating the camera
Normalize the input image point coordinates with transform T.
Estimate camera projection matrix CN from the normalized input image points.
Compute the denormalized camera projection matrix C as CNT-1.
Compute the reprojected image point coordinates xE as CX.
Compute the reprojection errors as
reprojectionErrors = |x− xE|.
Version HistoryIntroduced in R2019a
R2022b: Not recommended
Starting in R2022b, many Computer Vision Toolbox™ functions create and perform geometric transformations using the premultiply
convention. However, the
estimateCameraMatrix function uses the
postmultiply convention. Although there are no plans to remove
estimateCameraMatrix at this time, you can streamline your geometric
transformation workflows by switching to the
estimateCameraProjection function, which supports the premultiply convention.
To update your code:
Change instances of the function name
estimateCameraProjectionfunction returns the camera projection matrix in a format consistent with the premultiply convention. The returned camera projection matrix is the transpose of the
- Evaluating the Accuracy of Single Camera Calibration
- Structure From Motion From Two Views
- Structure From Motion From Multiple Views
- Depth Estimation From Stereo Video
- Code Generation for Depth Estimation From Stereo Video
- What Is Camera Calibration?
- Using the Single Camera Calibrator App
- Using the Stereo Camera Calibrator App