Importing Caffe network error ' Scale layer without a preceding BatchNorm layer'

I am trying to use a pretrained Caffe model of a CNN network ( TrailNet_SResNet-18 from here ) for comparison purposes and there is a problem that I cant solve . when use

importCaffeNetwork(protofile,datafile)

I get this error :

'The layer 'sub_mean' specifies a Scale layer without a preceding BatchNorm layer. Scale layers are only supported when

preceded by a BatchNorm layer'

one of the contribution of the authours was the idea of removing the Batch Normalization layer and substituting the ReLU layer with Shifted ReLU. looking closely at the structure of the network shows that it contain a lot of 'Scale' layers.

I tried to find another importable version of the same net ( Keras or ONNX) but couldnt find any. Also tried to convert it to Keras or ONXX but failed too with many errors that have been reported to the founder of the convertors.

I thought about reproducing the network on Matlab, hoewever, there is no shfted ReLU as well as the main problem above which is ' without a preceding BatchNorm layer'.

I hope that anyone can help me to find a solution for this problem or any work around it .

Answer 1

[Shashank Gupta]

If we see the original paper of Batch Normalization, the author mentioned that, “we make sure that the transformation inserted in the network can represent the identity transform”. Without the Scale layer after the BatchNorm layer will not work, Since Caffe BatchNorm layer has no learnable parameters.

You can look at the BatchNormalization paper here:

https://arxiv.org/abs/1502.03167

Also, there is no explicit function for Shifted ReLU in MATLAB, but you can always define a custom activation layer.

Refer to this link for more information

https://www.mathworks.com/help/deeplearning/ug/define-custom-deep-learning-layers.html

I hope it helps!

[caesar]

@ Shashank Gupta

thanks for your answer .

I am afraid I couldnt understand your point regarding the Btch Normalization.

regarding the SReLU I know I can use a costume layer in Matlab and again thanks for you advise .