Forward function with frozen batch normalization layers

In my application i have both batch normalization and dropout, and i would like to perform MC dropout with the forward function, and ideally i would freeze the parameters TrainedMean and TrainedVariance for the batch normalization layers, but i cannot seem to understand is it possible. I have the bn layers after conv layers, and the dropout after the recurrent layer in my net. Thank you in advance

[Imola Fodor]

actually another problem is that i get an error for using the forward function, simply putting as input the same arguments as for the predict function..

Answer 1

[Shivansh]

Hi Imola!

It seems you are facing issues with freezing the parameters of Batch normalization layers and the forward function for the network. For the first case, you can achieve this by setting the 'OffsetLearnRateFactor' and 'ScaleLearnRateFactor' as 0. This will ensure that the "Trained Mean" and "Trained variance" are not updated during the training. You can refer to the following related MATLAB answer for freezing the weights:

https://www.mathworks.com/matlabcentral/answers/548100-how-can-i-freeze-specific-weights-of-neural-network-model.

For the second query regarding the "forward" function, the "forward" function doesn't take the same inputs as the "predict" function. The "forward" function needs input to be a formatted dlarray. You can convert the current input array to a formatted dlarray using the following command:

dlA = dlarray(input,"format");

A sample example using wNet network

You can refer to the following MATLAB documentation links for more information:

1. batchNormalizationlayer: https://www.mathworks.com/help/deeplearning/ref/nnet.cnn.layer.batchnormalizationlayer.html.

2. dlarray: https://www.mathworks.com/help/deeplearning/ref/dlarray.html.

3. forward: https://www.mathworks.com/help/deeplearning/ref/dlnetwork.forward.html.

I hope it helps!