How to use the Levenberg-Marquardt algorithm to fitting data and get the first derivative?

I'm implementing the Levenberg-Marquardt algorithm in matlab to minimize a set of data resulting from a black-box equation.

My question is how to calculate the jacobian matrix from the set of data?

[John D'Errico]

You don't compute the Jacobian matrix FROM the set of data.

[Zakaria OUAOUJA]

That's exactly what I am trying to do, I am trying to minimize an output data resulting from a black-box software equations, and I am using the finite difference approximation for the Jacobian but it never works, and I can't find something helpful on the internet.

If you have solved the problem I would appreciate it if you could help me solve it too.

[karima ettourabi]

If you have solved the problem I would appreciate it if you could help me solve it too. please

Answer 1

[Robert U]

Hi Nelson Saraiva,

The Levenberg-Marquardt algorithm is already ready-to-use implemented into optimization toolbox: https://de.mathworks.com/help/optim/ug/lsqcurvefit.html?searchHighlight=lsqcurvefit&s_tid=doc_srchtitle

There are lots of descriptions on implementation on the internet. As an example, with implementation: http://people.duke.edu/%7Ehpgavin/ce281/lm.pdf

For most curve-fitting algorithms you have to have a model function defined with a set of free parameters. In order to find best fitting results with as less iterations as possible some algorithms (see gradient descent methods) do implement a kind of sensitivity analysis on all the free parameters. That would be the Jacobian matrix. Thus, Jacobian matrix does not depend on your data set but on the model function you chose.

Kind regards,

Robert

[Naif Alsalem]

Robert,

I came by your answer and i though it is very informative. If you dont mind, i have, say, measured & predicated spectra that i need to compare using LM method in matlab.

X= measured; Y= predicated;

How would i used LM function that is already included in matlab to fit these spectra?

Thank you very much in advance.

[Abhilash Awasthi]

@Naif Alsalem you can define your objective function as the difference between X & Y as a column vector. Then you can use the MATLAB inbuilt function 'lsqnonlin' from Optimization toolbox. You can also play with the options if you want to have a better control over your algorithm. Check this link for more details.

Answer 2

[Dhananjaya]

I want to train a neural network with some simulated data using Levenberg-Marquardt algorithm. Can anyone provide me the code.