Standard learning rate and momentum term trainlm NAR

Hello,

I'm looking for the standard learning rate and momentum term that is used by the Levenberg-Marquardt training algorithm. I found the standard mu factor, but as far as I can see this is not the same as the learning / momentum rate. Can someone help me with this?

Thanks in advance!

Answer 1

[Nick Durkee]

There is not really a simple answer to this question. The only thing I have found is by going into each of the layers and adjusting the learning rate individually.

 net = feedforwardnet([5 5 5]);
 net.inputWeights{1}.learnParam.lr = .0001;
 for ii = 1:3
    net.layerWeights{1+ii,ii}.learnParam.lr = .0001;

 end
 for jj = 1:4
    net.biases{jj}.learnParam.lr = .0001;
 end

I'm not entirely sure if this is correct as it doesn't seem to have a large impact on my training time.

Other than that, there is no documentation that I can find on how to change the learning rate.

[Lucas Ferreira-Correia]

I don't think it has an impact at all.

I'm using the Levenberg-Marquardt (trainlm) solver and after trying learning rates 0, 0.0001, 1 I was not able to observe a change in my MSE.

My theory is that this solver simply doesn't support any changes in learning rate.

Perhaps it only works on Gradient descent with momentum and adaptive learning rate backpropagation (traingdx)?

Hope this helps!

Answer 2

[Greg Heath]

There are 3 commands to try before posting

    help trainlm

    doc trainlm

   type trainlm

Hope this helps.

Thank you for accepting my answer

Greg

[Nick Durkee]

There are 3 commands to try before giving a snarky answer

    help trainlm
    doc trainlm
   type trainlm

Hope this helps.

The question stands, there is no simple way to change the learning rate.