Prepare Data for Quantizing Networks

Quantizing a deep neural network in MATLAB^® requires calibration by exercising the network to determine the dynamic ranges of its weights, biases, and activations. After calibrating, you can validate the performance of the quantized network and inspect the reduction in size and performance (if any) of the network. However, not all data formats supported for training and prediction are supported for quantization workflows.

Valid Data Objects

The calibrate and validate functions require data to be passed in one of the following formats.

Function	Supported Data Objects
calibrate	cell array datastore dlarray object numeric array
validate	datastore

The input data for the validate function must be a datastore to include both the input data and the expected response. For more information about the format of the data for validation, see Data and Response Formats.

Passing data as a table is not supported for the calibrate and validate functions.

If your network is a DAGNetwork or SeriesNetwork object that is not converted to a dlnetwork, only cell arrays, datastores, and numeric arrays are supported for the calibrate function.

Datastores

Many built-in datastores are supported for quantization workflows. The supported datastores for particular networks are detailed below.

To be a valid input for validation, the read function of a datastore must return data either as a cell array or as a table with the responses as the second column.

For more information on using datastores for training and prediction, see Datastores for Deep Learning.

• Choose a built-in datastore. Supported built-in datastores depend on your network type.

  Network Type	Supported Datastores
  Image classification or regression	arrayDatastore augmentedImageDatastore imageDatastore TransformedDatastore
  Object detection	arrayDatastore CombinedDatastore TransformedDatastore
  Semantic segmentation	arrayDatastore CombinedDatastore pixelLabelDatastore (Computer Vision Toolbox)
  Classification and regression for sequence and feature input data	arrayDatastore CombinedDatastore TransformedDatastore

• Transform and combine datastores. Deep learning frequently requires data to be preprocessed and augmented before data is in an appropriate form to input to a network. The transform and combine functions of datastores are useful in preparing data to be fed into a network.

  • Transform datastores – A transformed datastore applies a particular data transformation to an underlying datastore when reading data. To create a transformed datastore, use the transform function and specify the underlying datastore and the transformation. For validation, reading from a transformed datastore must return a cell array with the predictors as the first column and the responses as the third column.

  • Combine datastores – The combine function combines data from multiple datastores. Operating on the resulting CombinedDatastore, such as resetting the datastore, performs the same operation on all of the underlying datastores. The calibrate and validate functions support only CombinedDatastore objects with two underlying datastores. The first datastore must be an imageDatastore, augmentedImageDatastore, or arrayDatastore and the second datastore must be a pixelLabelDatastore (Computer Vision Toolbox) or arrayDatastore.

Cell, Deep Learning, and Numeric Arrays

Quantization functions typically handle data in cell arrays, dlarray objects, and numeric arrays in the same manner as training and prediction functions in Deep Learning Toolbox™. For more information on how data is formatted for training and prediction, see Deep Learning Data Formats.

When using a dlarray object, format the dlarray to assure the data has the appropriate shape. Formatted dlarray objects must have the same labelled dimensions as your initialized network. Unformatted dlarray objects are treated in the same way as numeric arrays.

Predictor and Response Formats

For all data objects, the predictors must be formatted based on the type of input and the responses must be formatted based on the type of task.

Predictor Formats

The format of the predictors used for calibration and validation depends on the type of input.

Data	Format of Predictors
2-D image	h-by-w-by-c numeric array, where h, w, and c are the height, width, and number of channels of the image, respectively.
Vector sequence	c-by-s matrix, where c is the number of features of the sequence and s is the sequence length.
2-D image sequence	h-by-w-by-c-by-s array, where h, w, and c correspond to the height, width, and number of channels of the image, respectively, and s is the sequence length.
Features	1-by-c column vector, where c is the number of features.

For data returned in tables, the elements must contain a 1-by-1 cell array containing a numeric array.

For calibration data passed as a numeric array, the format of the predictors used for calibration has an additional dimension for the number of batches. For example, vector sequence input data with numeric calibration data is a c-by-s-by-b array, where c is the number of features of the sequence, s is the sequence length, and b is the number of batches.

Response Formats

For validation, the datastore must return responses. As calibration exercises the network and collects the dynamic range statistics, the calibration data does not require responses. The format of the responses depends on the type of task.

Task	Format of Responses
Classification	Categorical scalar
Regression	Scalar Numeric vector 3-D numeric array representing an image
Segmentation	The default format returned by reading from a pixelLabelDatastore (Computer Vision Toolbox)
Object detection	Categorical array

Data Pre-Processing Considerations

These are additional considerations when pre-processing data for specific input data and network types.

Input data or network type	Consideration
Image input data	The calibrate and validate functions automatically resize image data to match the input of the network using imresize. For yolov2ObjectDetector (Computer Vision Toolbox), yolov3ObjectDetector (Computer Vision Toolbox), yolov4ObjectDetector (Computer Vision Toolbox), and ssdObjectDetector (Computer Vision Toolbox) network objects, the validate function requires image input data to match the data dimensions expected by the input layer of your network.
Sequence input data	The calibrate function does not provide any padding for variable sequence length inputs because padding may affect the numerics of some computations. All computations are performed with a minibatch size of 1.
Detector network	You must preprocess the data used for calibration of a yolov3ObjectDetector (Computer Vision Toolbox) object using the preprocess (Computer Vision Toolbox) function. For an example of using preprocessed data for calibration of a yolov3ObjectDetector, see Quantize YOLO v3 Object Detector.