Is a grid search for the global minima helpful for my optimization?

I have some series of data taken at different frequency domains and I am trying to fit each series of data to a nonlinear model. The main problem that I have is I do get inconsistent results for some of data series. The data I have is a nonlinear system with multiple distinct peaks. And the fitting function is essentially a fraction where I do have multiplication of two exponential functions in both nominator and denominator and one of the exponential functions are an exponential function of a series of Gaussian functions where the fitting parameters are the amplitude and width of each Gaussian and the distance between the Gaussians and the last fitting parameter is the argument of the other one of the two exponential functions. I tried 'Levenberg-Marquardt' algorithm in the curve fitting toolbox. But the result is changing each time I change the startpoints. Also, I do have 4 fitting parameters and everytime automatically it assumes two of them are fixed at startpoint and only fits the other two. So, I thought it would be helpful to see the result of globalsearch and multistart in the optimization toolbox and I tried it using different solvers and algorithms. However, I get close or similar results as the local solvers even by using multistart and globalsearch. One suggestion was to do a grid search for the global minima and so first I tried to plot the function that I want to optimize vs two of my fitting parameters (so, the other two are constant) to have a sense of what is going on. I have attached the obtained surfplot (SSE is the function to be minimized, a, d, sigma are parameters that I want to obtain by minimization). I am wondering do you think if a grid search would be helpful in my case?

I appreciate your comments!

[Alex Sha]

Hi, it will be more convenient for others to help you if all data and fitting function are given out

[Jan]

"And the fitting function is essentially a fraction where I do have multiplication of two exponential functions in both nominator and denominator and one of the exponential functions are an exponential function of a series of Gaussian functions where the fitting parameters are the amplitude and width of each Gaussian and the distance between the Gaussians and the last fitting parameter is the argument of the other one of the two exponential functions."

WOW! This is pure lyric.

I cannot open fig files currently, so I have no idea like your function looks like. But if it is chaotic (e.g. as the Manelbrot-set) a local optimization cannot be "successful". Of course try a grid search. You have to check, if this is useful and successful. There is no way to predict this reliably.

Answer 1

[Jasvin]

Hi, so as Jan has mentioned, most optimization is a lot of trial and error, no way to reliably pin down a model or optimization framework for your use-case. So for sure Grid Search must be attempted, but if your search space is large then running this Grid Search can take a significant amount of time.

However if your goal is to find the global minima then you can have a look at Simulated Annealing which is an interesting approach to find the global minima in a much faster way.

https://www.mathworks.com/help/gads/simulannealbnd.html

And if you want to try something different out then the whole domain of Genetic Algorithms can help you out as well!

https://www.mathworks.com/help/gads/ga.html

Basically unless you can directly observe clearly discernible patterns in the objective function, it is unlikely that you can directly hit the right configuration based on instinct alone and you will have to end up trying various things and progress in an iterative manner.

Hope this helps!