Normalize Multiple Outputs of Regression Network

I am using a u-net with input images of size 240x240x35 and with four different output images of size 240x240, so in all the outputlayer is 240x240x4. However, the different output images show different min, max values. The first output image is in range of 0-4096 and the other three are in range of 0-256. After training the network, only the first output images resulted in small deviations between true and predicted, however, the others showed much larger variations. I think this is due to the different ranges but I cannot work around it. I could devide the first output image by 10 but I think this is not a nice solution.

Now my question is if I can make a normalization (factor 10) for the first output layer similar to this post, when using trainNetwork():

https://de.mathworks.com/help/deeplearning/ug/normalize-errors-of-multiple-outputs.html

net.performParam.normalization = 'standard';

My Network is defined as:

    lgraph = unetLayers([240 240 35] , 3,'encoderDepth',3);
    lgraph = lgraph.removeLayers('Softmax-Layer');
    lgraph = lgraph.removeLayers('Segmentation-Layer');
    lgraph = lgraph.removeLayers('Final-ConvolutionLayer');
    lgraph = lgraph.addLayers(convolution2dLayer(1,4,'NumChannels',64,'name','Final-ConvolutionLayer'));
    lgraph = lgraph.addLayers(regressionLayer('name','regressionLayer'));
    lgraph = lgraph.connectLayers('Decoder-Stage-3-ReLU-2','Final-ConvolutionLayer');
    lgraph = lgraph.connectLayers('Final-ConvolutionLayer','regressionLayer');

Thanks for your help!

Ingo

Answer 1

[Raunak Gupta]

Hi,

The documentation link that is mentioned in the question is valid for feedforwardnet and that normalization works for vector like outputs. Here since unetLayers is used to solve the regression problem this normalization is not available as such because there can be multiple ways of normalizing the regression output.

For this case I would recommend normalizing the input training output so that all four channels have same range and saving the normalization function so original output can be retrieved for testing data. This documentation also mentions the same strategy. As per my understanding “standard” also does the same thing.

Hope it helps.

[Ingo Hermann]

Thank you for the answer. I also thought it it is automatically doing the normalization but when I normalized input and output my myself, the results where much better.

If there is no other normalization option I wil leave it like that.

Thanks again!