trainNetwork with response as categorical tensor

My goal is to train a neural Network with

• inputs of type 160x160x5 double (5 channels of images with 160x160 pixels each) and
• responses of type 160x160 categorical (pixel-wise labels with 2 classes)

as there are N samples, they are given as

• input: 160x160x5xN double
• response: 160x160x1xN categorical

The network layers map a 160x160x5 input to a 160x160x2 output (2 classes) and is taken from the example by mathworks. It has the following structure:

1  Image Input                  160x160x5 images with 'zerocenter' normalization
2  Convolution                  64 3x3 convolutions with stride [1  1] and padding [1  1  1  1]
3  Batch Normalization          Batch normalization
4  ReLU                         ReLU
5  Max Pooling                  2x2 max pooling with stride [2  2] and padding [0  0  0  0]
6  Transposed Convolution       16 4x4 transposed convolutions with stride [2  2] and output cropping [1  1]
7  Batch Normalization          Batch normalization
8  ReLU                         ReLU
9  Convolution                  2 1x1 convolutions with stride [1  1] and padding [0  0  0  0]
10 Softmax                      softmax
11 Pixel Classification Layer   Class weighted cross-entropy loss with classes 'healthy' and 'cancer'

How can I train a network with them? Following code fails :

[net, info] = trainNetwork(input,response,layers,options);
% Error using trainNetwork (line 150)
% Invalid training data. The output size ([160 160 2]) of the last layer does not match the
% response size ([1 1 2]).

According to the error message, it seems as if the the layers are defined correctly, but the response tensor is internally reduced to beeing 1x1x2 big instead of 160x160x2.

Comment on edit: changed no of classes to 2, added description of layers

Answer 1

[Malte Ebner]

In the meantime I solved the problem in a very unpleasent way: I first had to save all the input and response images as .gifs on my harddisk and then loaded them into an PixelLabelImageDatastore, as there seemed to be no direct way to create such a datastore out of a matrix directly (see also my other questions). The network is then trained using

trainNetwork(imageDatastore,layers,options) 

using the same layers, so they seem to be correct.

[Matt J]

I'm facing the same issue. Is there still no known alternative to saving all the images to separate physical files?

I can see why having the training data in imageDatastore format would be a good thing (for data augmentation purposes, for example), but I am a bit surprised that there isn't an easier way to construct an imageDatastore from images given in 4D numeric array form.

Answer 2

[Kushagr Gupta]

From the partial information provided in the question I can comment that there are issues in the way the network has been created.

For semantic segmentation problems you need to ensure the following:

1. You use a pixelClassificationLayer as the last layer in the network to perform pixelwise classification (not a classificationLayer). Read the folloiwng page for more details: Pixel Wise Classification
2. The last convolution layer must have filter size of 1. Having 1x1 convolution convoles data along the channel dimension and retains the input image's size, i.e. H and W dimensions remain the same after performing a 1x1 convolution.
3. There should not be any fully connected layer in your network as that would flatten the image and result in a vector. Note: The error message shared seems to indicate that there is probably a fullyConnectedLayer being used.

Keep the above mentioned points in mind while solving a semantic segmentation problem.

Also read creating a U-Net in MATLAB webpage which creates a semantic segmentation network with the paramters a user provides. This network helps solve the kind of problem that you are working with.

Hope this helps.