Problems with trainNetwork and LSTM networks

Hello,

I am trying to develop a NN using a LSTM layer to predict future values of Time Series, however when trying to train the network with some data, some problems arise. The structure of the network and the options is the following.

layers = [...

sequenceInputLayer(numFeatures)

lstmLayer(numHiddenUnits,'OutputMode','last')

%lstmLayer(numHiddenUnits)

fullyConnectedLayer(numResponses)

regressionLayer];

options = trainingOptions('adam','MaxEpochs',250,...

'InitialLearnRate',0.005,...

'LearnRateSchedule','piecewise',...

'LearnRateDropPeriod',125,...

'LearnRateDropFactor',0.2, ...

'Verbose',0);

Secondly, I prepared the data for the training. I created two vectors: trainingInput and trainingOutput (NaNs are eliminated previously). I also created cell arrays of these vectors, using the num2cell function.

trainInput = trainInput(indNaN,:);

trainInputCell=num2cell(trainInput');

trainOutput = trainOutput(indNaN);

trainOutputCell=num2cell(trainOutput');

This is what the workspace ouput shows of the vectors and arrays created.

After that, I tried to use the trainNetwork function (as shown in Matlab examples) to train the network, with the following lines and errors popping up:

1st-Try both vectors of type double

net=trainNetwork(trainInput',trainOutput',layers,options);

Error using trainNetwork (line 184)

Invalid training data. For a recurrent layer with output mode 'last', inputs must be cell arrays.

2nd- Try cell array input and output

net=trainNetwork(trainInputCell,trainOuputCell,layers,options);

Error using trainNetwork (line 184)

Invalid training data. For regression tasks, responses must be a vector, a matrix, or a 4-D array of numeric responses.

Responses must not contain NaNs.

3rd- Try cell array input and output double vector

net=trainNetwork(trainInputCell,trainOutput',layers,options);

Error using trainNetwork (line 184)

Invalid training data. Predictors and responses must have the same number of observations.

The 3rd type of coding giving an error is the one I cannot figure out. First, the Matlab Time Series Forecasting Using Deep Learning uses as inputData a double matrix, not a cell array. Secondly, the dimensions are the same for the last case, being that the problem that I do not understand at all.

Thank you

Answer 1

[Katja Mogalle]

Hello,

The following example shows quite well how to perform time series forecasting using an LSTM network: https://www.mathworks.com/help/deeplearning/ug/time-series-forecasting-using-deep-learning.html

There are two main things to watch out for:

First, setting up the network: to predict future values of a time series, you can define a sequence-to-sequence regression problem using an LSTM layer with OutputMode="sequence" and the final fully connected layer should have the same number of output channels as the input data to the network.

Secondly, the data should be one or more sequences where the output data is the input data shifted by one time step. For cell array input, the cell array must be an N-by-1 cell array of numeric arrays, where N is the number of observations. Each cell should contain a c-by-s matrix, where c is the number of features of the sequences and s is the sequence length. See this doc page for more info: https://www.mathworks.com/help/deeplearning/ref/trainnetwork.html?s_tid=doc_ta#mw_36a68d96-8505-4b8d-b338-44e1efa9cc5e

I'm not certain what the dimensions of your data mean. Is 101998 the sequence length? I assume numFeatures is 1 and numResponses is also 1? Then you can use trainInputCell = {trainInput'}; to mean you have one sequence with 101998 time steps and a single feature channel.

I hope this helps.