opticalFlowRAFT

Estimate optical flow using RAFT deep learning algorithm

Since R2024b

Description

Use the opticalFlowRAFT object to estimate the motion direction and velocity between previous and current video frames using the recurrent all-pairs field transforms (RAFT) algorithm. This algorithm uses a deep learning network trained on the Kubric [3] dataset. The RAFT optical flow estimation algorithm outperforms approaches like Farneback by delivering greater accuracy, particularly in areas with minimal texture, motion blur, and under difficult camera movements. It provides dense (per-pixel) and highly accurate estimations, but requires more time and memory. For quicker but less precise dense optical flow estimation, opt for opticalFlowFarneback, a traditional vision algorithm that does not rely on deep learning. Use of this object requires the Deep Learning Toolbox™.

Creation

Syntax

flowModel = opticalFlowRAFT

Description

flowModel = opticalFlowRAFT returns an optical flow object that estimates the motion direction and velocity between the previous and current video frames.

Object Functions

estimateFlow	Estimate optical flow between two frames
reset	Reset the internal state of the optical flow estimation object

Examples

Estimate Optical Flow in Consecutive Image Frames

Create a RAFT optical flow object.

flowModel = opticalFlowRAFT;

Create an object to read the input video file.

vidReader = VideoReader("visiontraffic.avi",CurrentTime=11);

Create a custom figure window to visualize the optical flow vectors.

h = figure;
movegui(h);
hViewPanel = uipanel(h, Position=[0 0 1 1], Title="Plot of Optical Flow Vectors");
hPlot = axes(hViewPanel);

Read consecutive image frames to estimate optical flow. Display the current current frame and overlay optical flow vectors using a quiver plot. The estimateFlow function calculates the optical flow between two consecutive frames.

Note that the function internally stores the previous frame and utilizes it implicitly for optical flow estimation. Consequently, when the function is called for the first time on a sequence of frames, it will return a zero flow. This is because, in the absence of a genuine previous frame, the initial frame is treated as both the current and previous frame, leading to no detectable motion between the two. This is consistent with the argument structure and behavior of established optical flow estimation methods, such as opticalFlowFarneback.

while hasFrame(vidReader)
    frame = readFrame(vidReader);
    flow = estimateFlow(flowModel,frame);

    imshow(frame)
    hold on
    plot(flow,DecimationFactor=[10 10],ScaleFactor=0.45,Parent=hPlot,color="g");
    hold off
    pause(10^-3)
end

Reset the opticalFlowRAFT object after the video processing has completed. This clears the internal state of the object, including the saved previous frame.

reset(flowModel);

References

[1] Teed, Zachary, and Jia Deng. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. Proceedings of the 16th European Conference on Computer Vision. 2020.

[2] Shah, Neelay, Prajnan Goswami, and Huaizu Jiang. EzFlow: A modular PyTorch library for optical flow estimation using neural networks. 2021. Web. https://github.com/neu-vi/ezflow.

[3] Greff, Klaus, Francois Belletti, Lucas Beyer, Carl Doersch, Yilun Du, Daniel Duckworth, David J. Fleet et al. Kubric: A scalable dataset generator. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 3749-3761. 2022.

Version History

Introduced in R2024b