Deploy and Run Fog Rectification for Video on NVIDIA Jetson

This example shows you how to generate and deploy a CUDA® executable for a video-based fog rectification application. It demonstrates the deployable code generation capabilities that the GPU Coder™ Support Package for NVIDIA® GPU provides for the MATLAB® VideoReader function. This example generates a CUDA application that reads the contents of a video file, performs fog rectification operation, and displays the output video on the NVIDIA hardware.

Prerequisites

Target Board Requirements

  • NVIDIA Jetson TX1/TX2 embedded platform.

  • Ethernet crossover cable to connect the target board and host PC (if the target board cannot be connected to a local network).

  • GStreamer and SDL libraries on the target.

  • Environment variables on the target for the compilers and libraries. For information on the supported versions of the compilers and libraries and their setup, see installing and setting up prerequisites for NVIDIA boards.

  • A monitor connected to the display port of the target.

Development Host Requirements

Copy Relevant files to the Current Directory

The following line of code creates a folder in your current working directory (host) and copies all the relevant files into this folder. If you cannot generate files in this folder, change your current working directory before running this command.

gpucoderdemo_setup('fog_rectification_videoreader');

Connect to the NVIDIA Hardware

The GPU Coder Support Package for NVIDIA GPUs uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson platform. You must therefore connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. Refer to the NVIDIA documentation on how to set up and configure your board.

To communicate with the NVIDIA hardware, you must create a live hardware connection object by using the jetson function. You must know the host name or IP address, username, and password of the target board to create a live hardware connection object. For example,

hwobj = jetson('jetson-tx2-name','ubuntu','ubuntu');

Verify GPU Environment

Use the coder.checkGpuInstall function to verify that the compilers and libraries necessary for running this example are set up correctly.

envCfg = coder.gpuEnvConfig('jetson');
envCfg.BasicCodegen = 1;
envCfg.Quiet = 1;
envCfg.HardwareObject = hwobj;
coder.checkGpuInstall(envCfg);

The fog_rectification Entry-Point Function

The fog_rectification.m function takes a foggy video as an input and displays the defogged video.

type fog_rectification
function fog_rectification() %#codegen

%   Copyright 2019 The MathWorks, Inc.

coder.gpu.kernelfun;
hwobj = jetson();
width = 600;
height = 404;
videoFileName = 'foggy.mp4';
vobj = VideoReader(hwobj,videoFileName,'Width',width,'Height',height);
dispObj = imageDisplay(hwobj);

while vobj.hasFrame
    input = vobj.readFrame;
    
    % run fog rectification on the frame 
    out = fog_rectification_algorithm (input,width,height);
    
    % display the output
    image(dispObj,out);
end

end

Prepare Fog Rectification Application

Include the VideoReader and display interfaces inside the fog rectification application for code generation as shown.

hwobj = jetson;
videoFileName = 'foggy.mp4';
vobj = VideoReader(hwobj,videoFileName,'Width',640,'Height',480);
dispObj = imageDisplay(hwobj);

The video file name input accepts full and relative paths. For code generation, you must provide the video file path on the target. For example, if the video file is available at the location '/home/ubuntu/video/foggy.mp4' then this path must be provided.

Generate CUDA Code for the Target Using GPU Coder

To generate a CUDA executable that can be deployed on to a NVIDIA target, create a GPU code configuration object for generating an executable.

cfg = coder.gpuConfig('exe');

Use the coder.hardware function to create a configuration object for the Jetson platform and assign it to the Hardware property of the code configuration object cfg.

cfg.Hardware = coder.hardware('NVIDIA Jetson');

Use the BuildDir property to specify the directory for performing remote build process on the target. If the specified build directory does not exist on the target then the software creates a directory with the given name. If no value is assigned to cfg.Hardware.BuildDir, the remote build process happens in the last specified build directory. If there is no stored build directory value, the build process takes place in the home directory.

cfg.Hardware.BuildDir = '~/remoteBuildDir';

Set the GenerateExampleMain property to generate an example CUDA C++ main function and compile it. This example does not require modifications to the generated main files.

cfg.GenerateExampleMain = 'GenerateCodeAndCompile';

To generate CUDA code, use the codegen function and pass the GPU code configuration along with fog_rectification entry-point function. After the code generation takes place on the host, the generated files are copied over and built on the target.

codegen('-config ',cfg,'fog_rectification','-report');

Copy Video File on the Target

Move the video file to the target.

hwobj.putFile('foggy.mp4', hwobj.workspaceDir);

Run the Fog Rectification on the Target

To run the generated executable on the target, use the MATLAB runApplication function.

pid = runApplication(hwobj,'fog_rectification');

A window opens on the target hardware display showing the fog rectification output of the recorded video.

Cleanup

Remove the files and return to the original folder.

cleanup

Summary

This example introduced an application where fog rectification application is running on the NVIDIA hardware on the recorded video and displaying the output on the native display.