Stereo Vision

Stereo vision for depth estimation

Stereo vision is the process of extracting 3-D information from multiple 2-D views of a scene. Stereo vision is used in applications such as advanced driver assistance systems (ADAS) and robot navigation where stereo vision is used to estimate the actual distance or range of objects of interest from the camera.

The 3-D information can be obtained from a pair of images, also known as a stereo pair, by estimating the relative depth of points in the scene. These estimates are represented in a stereo disparity map, which is constructed by matching corresponding points in the stereo pair.

Reconstructing a scene using a pair of stereo images (top left and top right). To visualize the disparity, the right channel is combined with the left channel to create a composite (middle left). Also shown are a disparity map of the scene (middle right) and a 3-D rendering of the scene (bottom center). See example for MATLAB code and explanation.

Stereo images are rectified to simplify matching, so that a corresponding point in one image can be found in the same row in the other image. This reduces the 2-D stereo correspondence problem to a 1-D problem. There are two approaches to stereo image rectification, calibrated and un-calibrated rectification. Un-calibrated stereo image rectification is achieved by determining a set of matched interest points, estimating the fundamental matrix, and then deriving two projective transformations. Calibrated stereo rectification uses information from the stereo camera calibration process.

Rectified stereo image pair. Notice that matching points reside on the same row. See example for MATLAB code and explanation.

Stereo anaglyph showing calibrated stereo image rectification. See example for MATLAB code and explanation.

Stereo camera calibration is used to determine the intrinsic parameters and relative location of cameras in a stereo pair, this information is used for stereo rectification and 3-D reconstruction.

Calibrating a stereo pair using a checker board pattern using the Stereo Camera Calibrator App

Stereo vision is also used in applications such as 3-D movie recording and production, object tracking, machine vision, and range sensing. For more information on stereo vision, see Computer Vision Toolbox.

See also: object detection, image and video image processing, RANSAC, feature matching, feature extraction, ransac, point cloud