Sobel Edge Detection on NVIDIA Jetson Nano Using Raspberry Pi Camera Module V2
This example shows you how to capture and process images from a Raspberry Pi Camera Module V2 connected to the NVIDIA® Jetson Nano. The MATLAB® Coder™ Support Package for NVIDIA Jetson and NVIDIA DRIVE Platforms allows you to capture images from the Camera Module V2 and bring them into the MATLAB environment for processing. In this example you learn how to develop a Sobel edge detection algorithm by using this capability.
Prerequisites
Target Board Requirements
NVIDIA Jetson Nano embedded platform.
Raspberry Pi Camera Module V2 connected to the CSI host port of the target.
Ethernet crossover cable to connect the target board and host PC (if you cannot connect the target board to a local network).
NVIDIA CUDA toolkit installed on the board.
V4L2 and SDL (v1.2) libraries on the board.
GStreamer libraries on the board.
Environment variables on the target for the compilers and libraries. For more information, see Install and Setup Prerequisites for NVIDIA Boards.
Development Host Requirements
GPU Coder for CUDA code generation. For a tutorial, see Get Started with GPU Coder (GPU Coder).
NVIDIA CUDA toolkit on the host.
Environment variables for the compilers and libraries. For more information, see Third-Party Hardware (GPU Coder) and Setting Up the Prerequisite Products (GPU Coder).
Create a Folder and Copy Relevant Files
The following line of code creates a folder in your current working folder on the host and copies all the relevant files into this folder. If you cannot generate files in this folder, before running this command, change your current working folder.
nvidiademo_setup('sobel_edge_detection');
Connect to NVIDIA Jetson Nano
The support package uses an SSH connection over TCP/IP to execute commands while building and running the generated CUDA code on the Jetson Nano platforms. Connect the target platform to the same network as the host computer or use an Ethernet crossover cable to connect the board directly to the host computer. For information on how to set up and configure your board, see NVIDIA documentation.
To communicate with the NVIDIA hardware, create a live hardware connection object by using the jetson
function. You must know the host name or IP address, user name, and password of the target board to create a live hardware connection object. For example, when connecting to the target board for the first time, create a live object for Jetson hardware by using the command:
hwobj = jetson('jetson-nano-name','ubuntu','ubuntu');
During the hardware live object creation, the support package performs hardware and software checks, IO server installation, and gathers peripheral information on target. This information is displayed in the Command Window.
Run the getCameraList
function of the hwobj
object to find the available cameras. If this function outputs an empty table, then try re-connecting the camera and execute the function again.
camlist = getCameraList(hwobj);
Verify GPU Environment on Target Board
To verify that the compilers and libraries necessary for running this example are set up correctly, use the coder.checkGpuInstall
(GPU Coder) function.
envCfg = coder.gpuEnvConfig('jetson');
envCfg.BasicCodegen = 1;
envCfg.Quiet = 1;
envCfg.HardwareObject = hwobj;
coder.checkGpuInstall(envCfg);
Create a Camera Object
Create a camera object by using the name from the getCameraList
function. For example, if the camera is named vi-output, imx219 6-0010
, use:
camObj = camera(hwobj,"vi-output, imx219 6-0010",[640 480]);
camObj
is a handle to a camera object. To display the images captured from the Camera Module V2 in MATLAB, use these commands:
for i = 1:100 img = snapshot(camObj); imagesc(img); drawnow; end
This camera object captures RGB and 3-channel grayscale images.
Create a Display Object
To create a display object, use the imageDisplay
function. This object is a system object that uses imshow
function to display the images in MATLAB.
dispObj = imageDisplay(hwobj); img = snapshot(camObj); image(dispObj,img);
Sobel Edge Detection Algorithm
The Sobel edge detection algorithm is a 2-D spatial gradient operation on a grayscale image. This operation emphasizes the high spatial frequency regions pf the image that corresponds to edges.
Calculate Gradients
Find horizontal gradient(h) and vertical gradient (v) of the input image with respective Sobel kernels. These two Sobel kernels are orthogonal to each other. Before processing live image data from the camera, test the algorithm on a sample image.
kern = [1 2 1; 0 0 0; -1 -2 -1];
img = imread('peppers.png');
imagesc(img);
h = conv2(img(:,:,2),kern,'same'); v = conv2(img(:,:,2),kern','same');
Calculate Gradient Magnitude
Find the gradient magnitude from the horizontal and vertical gradients (h and v).
e = sqrt(h.*h + v.*v);
Threshold the Edge Image
Threshold the image to find the regions of image that are edges.
edgeImg = uint8((e > 100) * 240);
imagesc(edgeImg);
Run Sobel Edge Detection Algorithm on Live Data
Create a MATLAB entry-point function, sobelEdgeDetectionAlg.m
, out of the MATLAB code developed in the previous sections of this example. View the code in MATLAB editor.
edit('sobelEdgeDetectionAlg.m');
The function sobelEdgeDetectionAlg
takes image and threshold input for edge detection and returns the results of edge detection algorithm. Call this function on the images captured from inside a loop. You can vary the threshold variable thresh
to get a proper edge image. This way you can use the camera access capability of the support package to tune the algorithm suitable for the specified camera.
for i = 1:200 img = snapshot(camObj); thresh = 100; edgeImage = sobelEdgeDetectionAlg(img, thresh); image(dispObj,edgeImage); end
To deploy this example as a standalone application on the target board, see Deploy and Run Sobel Edge Detection with I/O on NVIDIA Jetson Nano.
Cleanup
To remove the example files and return to the original folder, call the cleanup
function.
cleanup