Error using Deep Learning model LSTM

Hi eveyrone

I am getting error when I run my code. I am new to MATLAB so I am not sure how to fix this issue. I got the code from this book and I copied it as exactly as it is presented but only changed the how the data was created. The book made up data but I used actual stock data. Other than this, the codes are the same:

https://www.mathworks.com/academia/books/practical-matlab-deep-learning-paluszek.html

Can someone help me fix this issue?

Error:

Error using trainNetwork (line 191)

The training sequences are of feature dimension 2122 but the input layer expects sequences of

feature dimension 1.

Error in ForecastVolatilityLSTM (line 92)

net = trainNetwork(xTrain,yTrain,layers,options);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Code:

% LSTM
layerSet = 'two lstm';

n = length(CIV.COMPOSITE_IMPLIED_VOLATILITY);
nTrain = floor(0.8*n);
sTrain = CIV.COMPOSITE_IMPLIED_VOLATILITY(1:nTrain);
sTest = CIV.COMPOSITE_IMPLIED_VOLATILITY(nTrain+1:n);
sVal = sTest;

mu = mean(sTrain);
sigma = std(sTrain);

sTrainNorm = (sTrain-mu)/sigma;

sTestNorm = (sTest-mu)/sigma;
sTest = sTestNorm(1:end-1);

xTrain = sTrainNorm(1:end-1);
yTrain = sTrainNorm(2:end);

muVal = mean(sVal);
sigmaVal = std(sVal);
sValNorm = (sVal-muVal)/sigmaVal;

xVal = sValNorm(1:end-1);
yVal = sValNorm(2:end);

numFeatures = 1;
numResponses = 1;
numHiddenUnits = 200;

switch layerSet
    case 'lstm'
        layers = [sequenceInputLayer(numFeatures)
                  lstmLayer(numHiddenUnits)
                  fullyConnectedLayer(numResponses)
                  regressionLayer];
    case 'bilstm'
        layers = [sequenceInputLayer(numFeatures) 
                  bilstmLayer(numHiddenUnits) 
                  fullyConnectedLayer(numResponses) 
                  regressionLayer];
    case 'two lstm'
        layers = [sequenceInputLayer(numFeatures) 
                  lstmLayer(numHiddenUnits) 
                  reluLayer 
                  lstmLayer(numHiddenUnits) 
                  fullyConnectedLayer(numResponses) 
                  regressionLayer];
    otherwise
        error('Only 3 sets of layers are available');
end

analyzeNetwork(layers);

options = trainingOptions('adam', ...
                          'MaxEpochs',300, ...
                          'ExecutionEnvironment','gpu', ...
                          'GradientThreshold',1, ...
                          'InitialLearnRate',0.005, ...
                          'LearnRateSchedule','piecewise', ...
                          'LearnRateDropPeriod',125, ...
                          'LearnRateDropFactor',0.2, ...
                          'Shuffle','every-epoch', ...
                          'ValidationData',{xVal, yVal}, ...
                          'ValidationFrequency',5, ...
                          'Verbose',0, ...
                          'Plots','training-progress');

net = trainNetwork(xTrain,yTrain,layers,options);

Variables:

CIV table:

Thank you

Answer 1

[Walter Roberson]

https://www.mathworks.com/help/deeplearning/ref/trainnetwork.html#mw_36a68d96-8505-4b8d-b338-44e1efa9cc5e defines for the sequences input:

Numeric or Cell Array

Vector sequences

c-by-s matrices, where c is the number of features of the sequences and s is the sequence length.

Your xtrain is 2122 x 1, so that represents c = 2122 the number of features in the sequence, and 1 is the sequence length.

But you specified

numFeatures = 1;
%...
layers = [sequenceInputLayer(numFeatures)

so your code is expecting a sequence with 1 feature.

If you really want 1 feature, then you need to call

net = trainNetwork(xTrain.',yTrain.',layers,options);

[Manny]

Thank you @Walter Roberson. That fixed the error. I am now getting a new one in the network window. What does this error mean? How do I fix it?

Answer 2

[Joss Knight]

This error is because you are analyzing your network as a dlnetwork, which does not support output layers.

It depends on how you opened the Analyzer app. If you opened it from Deep Network Designer, then to analyze a DAGNetwork you need to use the -v1 option to deepNetworkDesigner to get the legacy behaviour:

https://uk.mathworks.com/help/deeplearning/ref/deepnetworkdesigner-app.html#mw_c01f379c-a044-4c49-a2c1-13b71e3b15dd

If you opened the App by calling analyzeNetwork, you need to add the option TargetUsage="trainNetwork" to get the legacy behaviour.

https://uk.mathworks.com/help/deeplearning/ref/analyzenetwork.html?s_tid=doc_ta#mw_338ac51f-1b3c-4fbb-904e-11195da95f2f

trainNetwork and DAGNetwork are no longer recommended, so ideally you should update your code to using trainnet and dlnetwork.

[Manny]

Thank you @Joss Knight. I copied the code from the book I mentioned. It was published in 2022.

1. I would like to update the code to use the new function. How do I modify my code so it works with the new function? I am new to MATLAB so I am not fully familiar with it.
2. In order to use the legacy function (ie my current code above), I did the following modification:

analyzeNetwork(layers,TargetUsage="trainNetwork");

Is this the correct way to use the option you mentioned?

[Joss Knight]

That's the way to keep using the older trainNetwork functionality, yes.

You may find someone on MATLAB Answers willing to show you manually how to migrate. However, I would encourage you to read the documentation and try out some basic examples. This will be the best way to learn and in the process you will gain sufficient familiarity with the tools to understand how to do your migration.

I hope I'm not giving the impression that the migration is hard. It really isn't. It's something like

• Call trainnet instead of trainNetwork
• Build your network using dlnetwork instead of layerGraph
• Get rid of output layers, instead you specify a loss function in your call to trainnet
• Convert existing networks to dlnetwork objects using dag2dlnetwork

[Manny]

Thank you @Joss Knight

It will be very helpful to me if I have some context behind the change. What are the reasons for the change? What benefits does the new function provide that the old function lacks? Is there an improvement in accuracy? Is it faster? etc. The more information you can share about the changes the better so that way I can see if the revisions make sense to me or not for my particular use-case.

[Joss Knight]

• dlnetwork uses automatic differentiation and can be used as part of a more complex function for which derivatives need to be computed, including one containing multiple dlnetwork objects.
• As a consequence, dlnetwork is amenable to the authoring of a custom training algorithm, giving the user much more flexibility.
• dlnetwork and trainnet are better optimized, so are faster. A critical part of the optimization is relying on the automatic differentiation, so are optimizations not available to DAGNetwork.
• Output layers are only used for training a model and ignored during inference, which was causing confusion. In dlnetwork and trainnet the loss function used to train the model is defined separately from the model.
• dlnetwork and trainnet support networks with multiple outputs. The use of output layers prevented that being possible with DAGNetwork and trainNetwork.

[Manny]

Thank you @Joss Knight. Is there any improvement to the accuracy of the forecasts?

[Joss Knight]

No. The two network types should train in basically the same way and reach the same level of accuracy in the same number of iterations. trainnet is faster and more efficient with memory, which provides some opportunities to train for more iterations and higher batch sizes so plausibly reach higher accuracy, but that's splitting hairs really.

[Manny]

Thank you @Joss Knight. This is very helpful to know.

[Walter Roberson]

(I moved this content to its own Answer, so that you could vote for Joss's work and possibly Accept it.)

[Manny]

Hi @Joss Knight

I am not an experienced MATLAB user. I got the code above from a book I mentioned earlier and I just copied it but changed the data source. If MATLAB is recommending to not use the functions I did use then I need help in moving over the code. Can you please re-write the old code using the new functions? I hope this isn't too much to ask.

Thank you

[Joss Knight]

I understand. I'm saying it is too much to ask - for me. You're going to have to read documentation, read the code you're adapting, learn what it's doing, and make some straightforward changes as I listed. But maybe someone else will help more directly.

Alternatively, just continue to use DAGNetwork and trainNetwork with the workarounds I gave you (-v1 option to deepNetworkDesigner, TargetUsage="trainNetwork" option to analyzeNetwork). Or downgrade MATLAB to R2023b or earlier.