Deploy YAMNet Networks to FPGAs with and without Cross-Layer Equalization
This example shows how to deploy a YAMNet network with and without cross-layer equalization to an FPGA. Cross -layer equalization can improve quantized network performance by reducing the variance of the network learnable parameters in the channels while maintaining the original network mapping. You can compare the accuracies of the network with and without cross-layer equalization.
Prerequisites
Xilinx® Zynq® Ultrascale+™ ZCU102 SoC development kit
Deep Learning HDL Toolbox™ Support Package for Xilinx FPGA and SoC
Audio Toolbox™
Deep Learning Toolbox™
Deep Learning HDL Toolbox™
Deep Learning Toolbox Model Quantization Library Support Package
GPU Coder™
GPU Coder Interface for Deep Learning Libraries
Load Pretrained YAMNet Network and Download Data
The YAMNet sound classification network is trained on the AudioSet data set to predict audio events from the AudioSet ontology. This example classifies sounds from air compressor. The AudioSet data set includes recordings from air compressors[1].
The data set is classified into one healthy state and seven faulty states, for a total of eight classes.
To download and load the pretrained YAMNet network and a set of air compressor sounds, run these commands.
url = 'https://ssd.mathworks.com/supportfiles/audio/YAMNetTransferLearning.zip'; AirCompressorLocation = pwd; dataFolder = fullfile(AirCompressorLocation,'YAMNetTransferLearning'); if ~exist(dataFolder,'dir') disp('Downloading pretrained network ...') unzip(url,AirCompressorLocation) end
Downloading pretrained network ...
addpath(fullfile(AirCompressorLocation,'YAMNetTransferLearning'))The final element of the Layers property is the classification output layer. The Classes property of this layer contains the names of the classes learned by the network.
load("airCompressorNet.mat");
net = airCompressorNet;
net.Layers(end).Classesans = 8×1 categorical
Bearing
Flywheel
Healthy
LIV
LOV
NRV
Piston
Riderbelt
Use the analyzeNetwork function to obtain information about the network layers. The function returns a graphical representation of the network that contains detailed parameter information for every layer in the network.
analyzeNetwork(net)
Create Calibration Data
The YAMNet network accepts inputs of size 96-by-64-by-1. Use the getyamnet_CLEInput helper function to obtain preprocessed mel spectrograms of size 96-by-64-by-1. See Helper Functions. For best quantization results, the calibration data must be representative of actual inputs that are predicted by the YAMNet network. Expedite the calibration process by reducing the calibration data set to 20 mel spectrograms.
numSpecs = 20; [imOutCal, imdsCal] = getyamnet_CLEInput(numSpecs);
Calculate Accuracy of Quantized YAMNet Without Cross-Layer Equalization
Create a YAMNet network that does not use cross-layer equalization, quantize the network, and calculate its accuracy.
Calibrate and Quantize Network
Create a quantized network by using the dlquantizer object. Set the target execution environment to FPGA.
dlQuantObj = dlquantizer(net,ExecutionEnvironment='FPGA');Use the calibrate function to exercise the network with sample inputs, collect the dynamic ranges of the weights and biases, and returns a table. Each row of the table contains range information for a learnable parameter of the quantized network.
calibrate(dlQuantObj,imdsCal);
Deploy the Quantized Network
Define the target FPGA board programming interface by using the dlhdl.Target object. Specify that the interface is for a Xilinx board with an Ethernet interface.
hTarget = dlhdl.Target('Xilinx',Interface='Ethernet');
Prepare the network for deployment by creating a dlhdl.Workflow object. Specify the network and the bitstream name. Ensure that the bitstream matches the data type and the FPGA board. In this example, the target FPGA is the Xilinx ZCU102 SOC board. The bitstream uses an int8 data type.
hW = dlhdl.Workflow(Network=dlQuantObj,Bitstream='zcu102_int8',Target=hTarget);Run the compile method of the dlhdl.Workflow object to compile the network and generate the instructions, weights, and biases for deployment. Because the total number of frames exceeds the default value of 30, set the InputFrameNumberLimit to 100 to run predictions in chunks of 100 frames to prevent timeouts.
dn = compile(hW,InputFrameNumberLimit=100)
### Compiling network for Deep Learning FPGA prototyping ...
### Targeting FPGA bitstream zcu102_int8.
### Optimizing network: Fused 'nnet.cnn.layer.BatchNormalizationLayer' into 'nnet.cnn.layer.Convolution2DLayer'
### The network includes the following layers:
1 'input_1' Image Input 96×64×1 images (SW Layer)
2 'conv2d' 2-D Convolution 32 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
3 'activation' ReLU ReLU (HW Layer)
4 'depthwise_conv2d' 2-D Grouped Convolution 32 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
5 'activation_1' ReLU ReLU (HW Layer)
6 'conv2d_1' 2-D Convolution 64 1×1×32 convolutions with stride [1 1] and padding 'same' (HW Layer)
7 'activation_2' ReLU ReLU (HW Layer)
8 'depthwise_conv2d_1' 2-D Grouped Convolution 64 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
9 'activation_3' ReLU ReLU (HW Layer)
10 'conv2d_2' 2-D Convolution 128 1×1×64 convolutions with stride [1 1] and padding 'same' (HW Layer)
11 'activation_4' ReLU ReLU (HW Layer)
12 'depthwise_conv2d_2' 2-D Grouped Convolution 128 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
13 'activation_5' ReLU ReLU (HW Layer)
14 'conv2d_3' 2-D Convolution 128 1×1×128 convolutions with stride [1 1] and padding 'same' (HW Layer)
15 'activation_6' ReLU ReLU (HW Layer)
16 'depthwise_conv2d_3' 2-D Grouped Convolution 128 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
17 'activation_7' ReLU ReLU (HW Layer)
18 'conv2d_4' 2-D Convolution 256 1×1×128 convolutions with stride [1 1] and padding 'same' (HW Layer)
19 'activation_8' ReLU ReLU (HW Layer)
20 'depthwise_conv2d_4' 2-D Grouped Convolution 256 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
21 'activation_9' ReLU ReLU (HW Layer)
22 'conv2d_5' 2-D Convolution 256 1×1×256 convolutions with stride [1 1] and padding 'same' (HW Layer)
23 'activation_10' ReLU ReLU (HW Layer)
24 'depthwise_conv2d_5' 2-D Grouped Convolution 256 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
25 'activation_11' ReLU ReLU (HW Layer)
26 'conv2d_6' 2-D Convolution 512 1×1×256 convolutions with stride [1 1] and padding 'same' (HW Layer)
27 'activation_12' ReLU ReLU (HW Layer)
28 'depthwise_conv2d_6' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
29 'activation_13' ReLU ReLU (HW Layer)
30 'conv2d_7' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
31 'activation_14' ReLU ReLU (HW Layer)
32 'depthwise_conv2d_7' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
33 'activation_15' ReLU ReLU (HW Layer)
34 'conv2d_8' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
35 'activation_16' ReLU ReLU (HW Layer)
36 'depthwise_conv2d_8' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
37 'activation_17' ReLU ReLU (HW Layer)
38 'conv2d_9' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
39 'activation_18' ReLU ReLU (HW Layer)
40 'depthwise_conv2d_9' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
41 'activation_19' ReLU ReLU (HW Layer)
42 'conv2d_10' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
43 'activation_20' ReLU ReLU (HW Layer)
44 'depthwise_conv2d_10' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
45 'activation_21' ReLU ReLU (HW Layer)
46 'conv2d_11' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
47 'activation_22' ReLU ReLU (HW Layer)
48 'depthwise_conv2d_11' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
49 'activation_23' ReLU ReLU (HW Layer)
50 'conv2d_12' 2-D Convolution 1024 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
51 'activation_24' ReLU ReLU (HW Layer)
52 'depthwise_conv2d_12' 2-D Grouped Convolution 1024 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
53 'activation_25' ReLU ReLU (HW Layer)
54 'conv2d_13' 2-D Convolution 1024 1×1×1024 convolutions with stride [1 1] and padding 'same' (HW Layer)
55 'activation_26' ReLU ReLU (HW Layer)
56 'global_average_pooling2d' 2-D Global Average Pooling 2-D global average pooling (HW Layer)
57 'dense' Fully Connected 8 fully connected layer (HW Layer)
58 'softmax' Softmax softmax (SW Layer)
59 'Sounds' Classification Output crossentropyex with 'Bearing' and 7 other classes (SW Layer)
### Notice: The layer 'input_1' with type 'nnet.cnn.layer.ImageInputLayer' is implemented in software.
### Notice: The layer 'softmax' with type 'nnet.cnn.layer.SoftmaxLayer' is implemented in software.
### Notice: The layer 'Sounds' with type 'nnet.cnn.layer.ClassificationOutputLayer' is implemented in software.
### Compiling layer group: conv2d>>activation_26 ...
### Compiling layer group: conv2d>>activation_26 ... complete.
### Compiling layer group: global_average_pooling2d ...
### Compiling layer group: global_average_pooling2d ... complete.
### Compiling layer group: dense ...
### Compiling layer group: dense ... complete.
### Allocating external memory buffers:
offset_name offset_address allocated_space
_______________________ ______________ ________________
"InputDataOffset" "0x00000000" "8.0 MB"
"OutputResultOffset" "0x00800000" "4.0 MB"
"SchedulerDataOffset" "0x00c00000" "4.0 MB"
"SystemBufferOffset" "0x01000000" "28.0 MB"
"InstructionDataOffset" "0x02c00000" "4.0 MB"
"ConvWeightDataOffset" "0x03000000" "32.0 MB"
"FCWeightDataOffset" "0x05000000" "4.0 MB"
"EndOffset" "0x05400000" "Total: 84.0 MB"
### Network compilation complete.
dn = struct with fields:
weights: [1×1 struct]
instructions: [1×1 struct]
registers: [1×1 struct]
syncInstructions: [1×1 struct]
constantData: {}
ddrInfo: [1×1 struct]
To deploy the network on the Xilinx ZCU102 SoC hardware, run the deploy function of the dlhdl.Workflow object. This function uses the programming file to program the FPGA board and downloads the network weights and biases. The deploy function programs the FPGA device and displays progress messages, and the required time to deploy the network.
deploy(hW)
### Programming FPGA Bitstream using Ethernet... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming FPGA device on Xilinx SoC hardware board at 192.168.1.101... ### Copying FPGA programming files to SD card... ### Setting FPGA bitstream and devicetree for boot... # Copying Bitstream zcu102_int8.bit to /mnt/hdlcoder_rd # Set Bitstream to hdlcoder_rd/zcu102_int8.bit # Copying Devicetree devicetree_dlhdl.dtb to /mnt/hdlcoder_rd # Set Devicetree to hdlcoder_rd/devicetree_dlhdl.dtb # Set up boot for Reference Design: 'AXI-Stream DDR Memory Access : 3-AXIM' ### Rebooting Xilinx SoC at 192.168.1.101... ### Reboot may take several seconds... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming the FPGA bitstream has been completed successfully. ### Loading weights to Conv Processor. ### Conv Weights loaded. Current time is 06-Jul-2023 16:07:40 ### Loading weights to FC Processor. ### FC Weights loaded. Current time is 06-Jul-2023 16:07:40
Test Network
Prepare the test data for prediction. Use the entire data set consisting of preprocessed mel spectrograms. Compare the predictions of the quantized network to the predictions of Deep Learning Toolbox.
[imOutPred, imdPred] = getyamnet_CLEInput(88);
Calculate the accuracy of the predictions of the quantized network with respect to the predictions from Deep Learning Toolbox by using the getNetworkAccuracy helper function. See Helper Functions.
quantizedAccuracy = getNetworkAccuracy(hW,imOutPred,net)
### Finished writing input activations.
### Running in multi-frame mode with 88 inputs.
Deep Learning Processor Profiler Performance Results
LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 7978514 0.03191 88 701741839 31.4
conv2d 27553 0.00011
depthwise_conv2d 26068 0.00010
conv2d_1 80316 0.00032
depthwise_conv2d_1 27113 0.00011
conv2d_2 69613 0.00028
depthwise_conv2d_2 32615 0.00013
conv2d_3 126269 0.00051
depthwise_conv2d_3 21334 0.00009
conv2d_4 90407 0.00036
depthwise_conv2d_4 27856 0.00011
conv2d_5 171756 0.00069
depthwise_conv2d_5 22016 0.00009
conv2d_6 312844 0.00125
depthwise_conv2d_6 29744 0.00012
conv2d_7 617984 0.00247
depthwise_conv2d_7 29674 0.00012
conv2d_8 617664 0.00247
depthwise_conv2d_8 30024 0.00012
conv2d_9 617954 0.00247
depthwise_conv2d_9 30084 0.00012
conv2d_10 617534 0.00247
depthwise_conv2d_10 29724 0.00012
conv2d_11 617444 0.00247
depthwise_conv2d_11 26122 0.00010
conv2d_12 1207736 0.00483
depthwise_conv2d_12 38636 0.00015
conv2d_13 2409106 0.00964
global_average_pooling2d 20015 0.00008
dense 3231 0.00001
* The clock frequency of the DL processor is: 250MHz
quantizedAccuracy = 0.6477
Calculate Accuracy of Quantized YAMNet with Cross Layer Equalization
Create a YAMNet network with cross-layer equalization, quantize the network, and calculate its accuracy.
Create, Calibrate and Quantize a Cross-Layer Equalized Network
Create a cross layer equalized network by using the equalizeLayers function.
netCLE = equalizeLayers(net);
Create a quantized YAMNet with cross layer equalization by using the dlquantizer object. Set the target execution environment to FPGA.
dlQuantObjCLE = dlquantizer(netCLE,ExecutionEnvironment='FPGA');Use the calibrate function to exercise the network with sample inputs, collect the dynamic ranges of the weights and biases, and returns a table. Each row of the table contains range information for a learnable parameter of the quantized network.
calibrate(dlQuantObjCLE,imdsCal);
Deploy Quantized Network
Define the target FPGA board programming interface by using the dlhdl.Target object. Specify that the interface is for a Xilinx board with an Ethernet interface.
hTargetCLE = dlhdl.Target('Xilinx',Interface='Ethernet');
Prepare the network for deployment by creating a dlhdl.Workflow object. Specify the network and the bitstream name. Ensure that the bitstream matches the data type and the FPGA board. In this example, the target FPGA is the Xilinx ZCU102 SOC board. The bitstream uses an int8 data type.
hWCLE = dlhdl.Workflow(Network=dlQuantObjCLE,Bitstream='zcu102_int8',Target=hTargetCLE);Run the compile method of the dlhdl.Workflow object to compile the network and generate the instructions, weights, and biases for deployment. Because the total number of frames exceeds the default value of 30, set the InputFrameNumberLimit to 100 to run predictions in chunks of 100 frames to prevent timeouts.
dnCLE = compile(hWCLE,InputFrameNumberLimit=100)
### Compiling network for Deep Learning FPGA prototyping ...
### Targeting FPGA bitstream zcu102_int8.
### The network includes the following layers:
1 'input_1' Image Input 96×64×1 images (SW Layer)
2 'conv2d' 2-D Convolution 32 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
3 'activation' ReLU ReLU (HW Layer)
4 'depthwise_conv2d' 2-D Grouped Convolution 32 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
5 'activation_1' ReLU ReLU (HW Layer)
6 'conv2d_1' 2-D Convolution 64 1×1×32 convolutions with stride [1 1] and padding 'same' (HW Layer)
7 'activation_2' ReLU ReLU (HW Layer)
8 'depthwise_conv2d_1' 2-D Grouped Convolution 64 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
9 'activation_3' ReLU ReLU (HW Layer)
10 'conv2d_2' 2-D Convolution 128 1×1×64 convolutions with stride [1 1] and padding 'same' (HW Layer)
11 'activation_4' ReLU ReLU (HW Layer)
12 'depthwise_conv2d_2' 2-D Grouped Convolution 128 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
13 'activation_5' ReLU ReLU (HW Layer)
14 'conv2d_3' 2-D Convolution 128 1×1×128 convolutions with stride [1 1] and padding 'same' (HW Layer)
15 'activation_6' ReLU ReLU (HW Layer)
16 'depthwise_conv2d_3' 2-D Grouped Convolution 128 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
17 'activation_7' ReLU ReLU (HW Layer)
18 'conv2d_4' 2-D Convolution 256 1×1×128 convolutions with stride [1 1] and padding 'same' (HW Layer)
19 'activation_8' ReLU ReLU (HW Layer)
20 'depthwise_conv2d_4' 2-D Grouped Convolution 256 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
21 'activation_9' ReLU ReLU (HW Layer)
22 'conv2d_5' 2-D Convolution 256 1×1×256 convolutions with stride [1 1] and padding 'same' (HW Layer)
23 'activation_10' ReLU ReLU (HW Layer)
24 'depthwise_conv2d_5' 2-D Grouped Convolution 256 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
25 'activation_11' ReLU ReLU (HW Layer)
26 'conv2d_6' 2-D Convolution 512 1×1×256 convolutions with stride [1 1] and padding 'same' (HW Layer)
27 'activation_12' ReLU ReLU (HW Layer)
28 'depthwise_conv2d_6' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
29 'activation_13' ReLU ReLU (HW Layer)
30 'conv2d_7' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
31 'activation_14' ReLU ReLU (HW Layer)
32 'depthwise_conv2d_7' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
33 'activation_15' ReLU ReLU (HW Layer)
34 'conv2d_8' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
35 'activation_16' ReLU ReLU (HW Layer)
36 'depthwise_conv2d_8' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
37 'activation_17' ReLU ReLU (HW Layer)
38 'conv2d_9' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
39 'activation_18' ReLU ReLU (HW Layer)
40 'depthwise_conv2d_9' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
41 'activation_19' ReLU ReLU (HW Layer)
42 'conv2d_10' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
43 'activation_20' ReLU ReLU (HW Layer)
44 'depthwise_conv2d_10' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
45 'activation_21' ReLU ReLU (HW Layer)
46 'conv2d_11' 2-D Convolution 512 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
47 'activation_22' ReLU ReLU (HW Layer)
48 'depthwise_conv2d_11' 2-D Grouped Convolution 512 groups of 1 3×3×1 convolutions with stride [2 2] and padding 'same' (HW Layer)
49 'activation_23' ReLU ReLU (HW Layer)
50 'conv2d_12' 2-D Convolution 1024 1×1×512 convolutions with stride [1 1] and padding 'same' (HW Layer)
51 'activation_24' ReLU ReLU (HW Layer)
52 'depthwise_conv2d_12' 2-D Grouped Convolution 1024 groups of 1 3×3×1 convolutions with stride [1 1] and padding 'same' (HW Layer)
53 'activation_25' ReLU ReLU (HW Layer)
54 'conv2d_13' 2-D Convolution 1024 1×1×1024 convolutions with stride [1 1] and padding 'same' (HW Layer)
55 'activation_26' ReLU ReLU (HW Layer)
56 'global_average_pooling2d' 2-D Global Average Pooling 2-D global average pooling (HW Layer)
57 'dense' Fully Connected 8 fully connected layer (HW Layer)
58 'softmax' Softmax softmax (SW Layer)
59 'Sounds' Classification Output crossentropyex with 'Bearing' and 7 other classes (SW Layer)
### Notice: The layer 'input_1' with type 'nnet.cnn.layer.ImageInputLayer' is implemented in software.
### Notice: The layer 'softmax' with type 'nnet.cnn.layer.SoftmaxLayer' is implemented in software.
### Notice: The layer 'Sounds' with type 'nnet.cnn.layer.ClassificationOutputLayer' is implemented in software.
### Compiling layer group: conv2d>>activation_26 ...
### Compiling layer group: conv2d>>activation_26 ... complete.
### Compiling layer group: global_average_pooling2d ...
### Compiling layer group: global_average_pooling2d ... complete.
### Compiling layer group: dense ...
### Compiling layer group: dense ... complete.
### Allocating external memory buffers:
offset_name offset_address allocated_space
_______________________ ______________ ________________
"InputDataOffset" "0x00000000" "8.0 MB"
"OutputResultOffset" "0x00800000" "4.0 MB"
"SchedulerDataOffset" "0x00c00000" "4.0 MB"
"SystemBufferOffset" "0x01000000" "28.0 MB"
"InstructionDataOffset" "0x02c00000" "4.0 MB"
"ConvWeightDataOffset" "0x03000000" "32.0 MB"
"FCWeightDataOffset" "0x05000000" "4.0 MB"
"EndOffset" "0x05400000" "Total: 84.0 MB"
### Network compilation complete.
dnCLE = struct with fields:
weights: [1×1 struct]
instructions: [1×1 struct]
registers: [1×1 struct]
syncInstructions: [1×1 struct]
constantData: {}
ddrInfo: [1×1 struct]
To deploy the network on the Xilinx ZCU102 SoC hardware, run the deploy function of the dlhdl.Workflow object. This function uses the programming file to program the FPGA board and downloads the network weights and biases. The deploy function programs the FPGA device and displays progress messages, and the required time to deploy the network.
deploy(hWCLE)
### Programming FPGA Bitstream using Ethernet... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming FPGA device on Xilinx SoC hardware board at 192.168.1.101... ### Copying FPGA programming files to SD card... ### Setting FPGA bitstream and devicetree for boot... # Copying Bitstream zcu102_int8.bit to /mnt/hdlcoder_rd # Set Bitstream to hdlcoder_rd/zcu102_int8.bit # Copying Devicetree devicetree_dlhdl.dtb to /mnt/hdlcoder_rd # Set Devicetree to hdlcoder_rd/devicetree_dlhdl.dtb # Set up boot for Reference Design: 'AXI-Stream DDR Memory Access : 3-AXIM' ### Rebooting Xilinx SoC at 192.168.1.101... ### Reboot may take several seconds... ### Attempting to connect to the hardware board at 192.168.1.101... ### Connection successful ### Programming the FPGA bitstream has been completed successfully. ### Loading weights to Conv Processor. ### Conv Weights loaded. Current time is 06-Jul-2023 16:13:55 ### Loading weights to FC Processor. ### FC Weights loaded. Current time is 06-Jul-2023 16:13:55
Test Network
Prepare the test data for prediction. Use the entire data set consisting of preprocessed mel spectrograms. Compare the predictions of the quantized network to the predictions of Deep Learning Toolbox.
[imOutPredCLE, imdPredCLE] = getyamnet_CLEInput(88);
Compare the accuracy of the quantized network predictions against the accuracy of the predictions from Deep Learning Toolbox, by using the getNetworkAccuracy helper function. See Helper Functions.
quantizedAccuracyCLE = getNetworkAccuracy(hWCLE, imOutPredCLE, netCLE)
### Finished writing input activations.
### Running in multi-frame mode with 88 inputs.
Deep Learning Processor Profiler Performance Results
LastFrameLatency(cycles) LastFrameLatency(seconds) FramesNum Total Latency Frames/s
------------- ------------- --------- --------- ---------
Network 7977295 0.03191 88 701748593 31.4
conv2d 27523 0.00011
depthwise_conv2d 26158 0.00010
conv2d_1 80284 0.00032
depthwise_conv2d_1 27109 0.00011
conv2d_2 69334 0.00028
depthwise_conv2d_2 32244 0.00013
conv2d_3 126831 0.00051
depthwise_conv2d_3 20814 0.00008
conv2d_4 90320 0.00036
depthwise_conv2d_4 27841 0.00011
conv2d_5 171883 0.00069
depthwise_conv2d_5 22036 0.00009
conv2d_6 312914 0.00125
depthwise_conv2d_6 29674 0.00012
conv2d_7 618094 0.00247
depthwise_conv2d_7 29604 0.00012
conv2d_8 617444 0.00247
depthwise_conv2d_8 30064 0.00012
conv2d_9 617944 0.00247
depthwise_conv2d_9 30054 0.00012
conv2d_10 617384 0.00247
depthwise_conv2d_10 29704 0.00012
conv2d_11 617474 0.00247
depthwise_conv2d_11 26122 0.00010
conv2d_12 1207626 0.00483
depthwise_conv2d_12 38466 0.00015
conv2d_13 2408966 0.00964
global_average_pooling2d 20127 0.00008
dense 3179 0.00001
* The clock frequency of the DL processor is: 250MHz
quantizedAccuracyCLE = 0.8636
Using the cross-layer equalization improves the prediction accuracy of the network. The accuracy of the network with cross-layer equalization (netCLE) is 86.36% and the accuracy of the network without cross-layer equalization (net) is 64.77%.
Helper Functions
The getyamnet_CLEInput helper function obtains preprocessed mel spectrograms of size 96-by-64-by-1.
function [imOut,imds] = getyamnet_CLEInput(NumOfImg) if NumOfImg > 88 error('Provide an input less than or equal to the size of this dataset of 88 images.'); end spectData = load('yamnetInput.mat'); spectData = spectData.melSpectYam; imOut = spectData(:,:,:,1:NumOfImg); imds = augmentedImageDatastore([96 64],imOut); end
The getNetworkAccuracy helper function retrieves predictions from the FPGA and compares them to the predictions from Deep Learning Toolbox™.
function accuracy = getNetworkAccuracy(workFlow, data, network) % Function gets accuracy of predictions by workflow object with respect to predictions by DL toolbox % Predictions by workflow object hwPred = workFlow.predict(data,Profile="on"); % Matrix of probability for each class [hwValue, hwIdx] = max(hwPred,[],2); % Get value and index of class with max probability hwClasses = network.Layers(end).Classes(hwIdx); % Get class names from index % predictions by DL toolbox object dlPred = predict(network, data); % Matrix of probability for each class [dlValue, dlIdx] = max(dlPred,[],2); % Get value and index of class with max probability dlClasses = network.Layers(end).Classes(dlIdx); % Get class names from index accuracy = nnz(dlClasses == hwClasses)/size(data,4); % Number of same predictions/total predictions end
References
[1] Verma, Nishchal K., et al. “Intelligent Condition Based Monitoring Using Acoustic Signals for Air Compressors.” IEEE Transactions on Reliability, vol. 65, no. 1, Mar. 2016, pp. 291–309. DOI.org (Crossref), doi:10.1109/TR.2015.2459684.
See Also
dlhdl.Target | dlhdl.Workflow | compile | deploy | predict | dlquantizer | calibrate | equalizeLayers
You can also select a web site from the following list
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Asia Pacific
- Australia (English)
- India (English)
- New Zealand (English)
- 中国
- 日本Japanese (日本語)
- 한국Korean (한국어)