Invalid training data. Responses must be nonempty when using networkTrain on CNN

Hello,

I'm having issues with the transform of an ImageDatastore when is used as input to the networkTrain method. If I use the original ImageDatastore as input to the networkTrain, everything works as a charm. By using the read method on the transformed datastore and visualizing the images with imshow() I can see that the transform function is doing what is supposed to do (i.e., adding noise to the images in the ImageDatastore). However, when passing the TransformedDatastore to the networkTrain, the following message is thrown:

Error using trainNetwork (line 170)

Invalid training data. Responses must be nonempty.

Answer 1

[Mohammad Sami]

If you have image processing toolbox, you may perhaps use denoisingImageDatastore

https://www.mathworks.com/help/releases/R2020a/images/ref/denoisingimagedatastore.html

[Carlos Ramirez]

Thank you Sami. Unfortunately, is more than just noise. I'm trying to change luminance conditions, non-linear streching of the color histograms, etc., so it will be nice to be able to take advantage of the Transformed Datastore, which by the way is doing what I am asking it to do, it's just that networkTraing doesn't like it as input. In fact, if I pass to the networkTrain, the 'transformedDataStore.UnderlyingDatastore' , the networkTrain method smiles and start doing it's job. But, when I pass the 'transformedDataStore' itself to the networkTrain it complains that responses must not be empty, which does not make sense.

[Mohammad Sami]

Can you check the labels property on the transformed data store

[Carlos Ramirez]

Sami, the transformed data store does not have a Labels property. I believe it relies on the labels stored in the UnderlyingDatastore. I borrowed a simple example from the Matlab help to reproduce the problem. Here it is:

digitDatasetPath = fullfile(matlabroot,'toolbox','nnet', ...
    'nndemos','nndatasets','DigitDataset');
imds = imageDatastore(digitDatasetPath, ...
    'IncludeSubfolders',true, ...
    'LabelSource','foldernames');

numTrainingFiles = 750;
[imdsTrain,imdsTest] = splitEachLabel(imds,numTrainingFiles,'randomize');
layers = [ ...
    imageInputLayer([28 28 1])
    convolution2dLayer(5,20)
    reluLayer
    maxPooling2dLayer(2,'Stride',2)
    fullyConnectedLayer(10)
    softmaxLayer
    classificationLayer];

options = trainingOptions('sgdm', ...
    'MaxEpochs',20,...
    'InitialLearnRate',1e-4, ...
    'Verbose',false, ...
    'Plots','training-progress');

idmsNew = transform(imds,@transformFcn);

net = trainNetwork(idmsNew,layers,options);

function [dataOut]= transformFcn(dataIn)
    Gray=dataIn;
    %Gray=Gray+20; // do whatever transformation you want to do here
    dataOut=Gray;
end

which results in:

Error using trainNetwork (line 170)

Invalid training data. Responses must be nonempty.

Error in test (line 28)

net = trainNetwork(idmsNew,layers,options);

[Mohammad Sami]

Seems that the read function must output a cell array with input + 1 number of columns.

The last column should contain the responses (Labels).

First you need set includeinfo to true, when calling the transform.

idmsNew = transform(imds,@transformFcn,'IncludeInfo',true);

You will then need to amend your transform function as follows.

function [dataOut,info] = transformFcn(data,info)
if(length(info) > 1)
    numRows = length(info);
    dataOut = cell(numRows,2);
else
    % if readsize = 1
    numRows = 1;
    data = {data};
end
for idx = 1:numRows
    
    % Randomized 90 degree rotation
    imgOut = rot90(data{idx,1},randi(4)-1);
    
    % Return the label from info struct as the 
    % second column in dataOut.
    dataOut(idx,:) = {imgOut,info.Label(idx)};
    
end
end

See the example at this link below

https://www.mathworks.com/help/deeplearning/ug/datastores-for-deep-learning.html

[Carlos Ramirez]

Perfect !. That works. Thanks.

[M J]

Hi there, my transforming function actually takes a subset of the entire training image datastore. Can I apply it the same way as you did so that my function creates custom batches during the training process (every time it gets called)? Thanks!

[Mohammad Sami]

Yes. The structure of the code can potentially be the same.

[drummer]

Great, it worked to me too.

Thanks.

I'm performing affine geometric transformation to volumetric images and MATLAB was also returning such error.

But only one thing I would like to ask to @Carlos is why you transformed all the imds?

Wasn't the purpose of your transformation to augment training data? This way, only imdsTrain would be applied the transformation...

I know there are several purposes to transform data. My question is just out of curiosity...

cheers

[JAYANTHI BALASUBRAMANIAN]

hello,

I am also facing the same problem, but the transform function I used is for Image contrast enhancement for brightness preservation based on dynamic streatching.Can you suggest any methods to correct it

Thank You