Nonlinear fit to 100 data sets with seven unknown

Hello all,

I have a problem with lsqcurvefit optimization.Here is my problem,

I have been working on project which non-linear equation fitting with lsqcurvefit. I need to fit nonlinear equations to several data sets. I want to fit 7 unknown with using a least-squares method (lsqcurvefit) . The code has worked but the results of seven unkonowns are not good. Standart error is very big and resnorm of optimization is 0.5212 . The MATLAB find local minimum points.

How can I solve non-liner curve fitting problem ?I want resnorm to be 10^-13.

Could anyone help me about this problem?

Thank you for alll,

Answer 1

[Star Strider]

The residual norm you want may be unrealistic. I would use the ga (genetic algorithm) function (perhaps in a loop to run it several times) in order to get the best parameter estimates. See: How to save data from Genetic Algorithm in case MATLAB crashes? for a way of storing the best results from each ga call for later analysis.

[Deniz Toprak]

First of all thank you for your answering,

Maybe I couldn't ask the problem properly.In this optimization code, I know the seven values ​​which optimization should find .I am trying to change initial values,lower bound and upper bound values to find right results however I can't find these values from optimization with lsqcurvefit. Although I increase data sets from 50 to 100 data for getting best results, I can't get what I want .

Where can be the problem? and how should I solve this problem?

Thank you for your interest.

[Star Strider]

My pleasure!

Nonlinear parameter estimation is very sensitive to the initial parameter estimates, since almost all of them use gradient-descent algorithms to estimate the best parameters. The problem is that they can end up in local (not global) minima, and produce less than optimal results.

That is the reason I suggested using the ga algorithm to search for good candidates, since it will search the global parameter space for the best parameter set. It can produce good parameter estimates that you can then use as initial parameter estimates to various nonlinear parameter estimation routines to get the final result.

[Alex Sha]

Hi, Deniz, post your data file and function you want to fit.

[Deniz Toprak]

Dear Star Strider,

Thank you very much your explanation is very clearly. I will try ga algorithm on my problem.

[Star Strider]

My pleasure!

[Deniz Toprak]

Dear Star Strider,

Here I am again, my mind got confused about genetic algorithm.As I said my matlab code includes lsqcurvefit optimization , can ı use ga after lsqcurvefit function. Breifly, I found this link ,

https://www.mathworks.com/matlabcentral/answers/447293-is-it-possible-to-use-simulated-annealing-or-genetic-algorithm-for-parameter-estimation-of-a-paramet

I looked this code and tried it but matlab gave this error ;

Invalid solver specified. Provide a solver name or handle (such as 'fmincon' or @fminunc). Type DOC OPTIMOPTIONS for a list of solvers

I don't know what should I do. Would you help me with this?

Thank for your attention ,

[Star Strider]

I have no idea what the problem is, since you have posted only the error message and not your code. Note that the ga function requires the Global Optimization Toolbox.

[Deniz Toprak]

Hi Star,

Thank you for you returning. I tried ga with optimization toolbox but I couldn't do it . Can I do it wth writing ga code to my matlab. How can I write ga code to my code ?

Can you help me with this?

[Star Strider]

The ga function documentation and associated links should provide you with all the guidance you need.

Your fitness function must be written so that it returns a scalar fitness value. How you write it depends on what you are doing and what your data are. Before you use your fitness function with ga, run it first with a random parameter vector to determine that it runs without error, and that it meets this criterion, for a representative sample of your data. Then, use it with ga and observe the results. Note that several runs with different initial populations may be necessary to get an acceptable result.

[Deniz Toprak]

Hi Strider,

First of all thank you for your help. I have tried ga with global optimization but when I wrote my fitness function it gaved me not enough input arguments error. Then I tried to write ga code to my code but it gaved same error. The problem is with fitness function and I don't understand what should I do to solve this problem.

I want to improve my lsqcurvefit optimization results so in my code first it solves lsqcurvefit then ga code will run. Can I use lsqcurvefit and ga at the same code?

[Star Strider]

My pleasure!

I have no idea what the problem is with the fitness function, since I do not see it anywhere.

You can use both in the same code. I would run ga first, then lsqcurvefit with the parameter estimates ga returns as the initial parameter estimates for lsqcurvefit.

[Deniz Toprak]

Hi Strider,

Thank you for your advice . I have some question too. Can I fix some parameters?

For example, lsqcurvefit solves the seven parameter and gives me the results. 3 of parameters are acceptable. I know that lsqcurvefit gives always same results for this three parameter. I want to add fixing code for 3 parameters . Then when I run code, first lsqcurvefit will solves the 7 parameters then fix the three parameter and make again optimzation to four parameter. It will fit four parameter.

I want to make this arrangement at the same code .

Can you help me with this? How can I perform this at my code?

[Star Strider]

My pleasure.

I am not certain what you are asking. If you know the parameters you want not to change, specify them as such in your model, and re-number the remaining parameters so that the optimisation rotine fits only the remaining parameters. (A significantly less efficient alternative is to specify the bounds of the parameters you want not to change as tight bounds in the ‘lb’ and ‘ub’ vectors, with the others as -Inf and +Inf.)

[Deniz Toprak]

Hi Star,

First of all thank you for your answer. I tried what you said but it didn't work well. I have code to show you how I want to fix some parameters. At this code first lsqcurvefit works then code will find the x parameters. The value of x2 is not good and the value of x1 and x3 fixed later x2 is fitted again with lsqcurvefit. I would like to use this type of code. Will it work well?

This code has problem I couldn't repair it. The aim is can I do this type of fixing ?

Thank you for your response

function fixing

myfun = @(x, xdata) x(1) + x(2)*xdata + x(3)./xdata;

%% Some Data
xdata = [0.036;0.14;0.42;0.49;0.66;0.68;0.76;0.79;0.8;0.85;0.92;0.93;0.96];
ydata = [0.83;1;0.66;0.53;0.58;0.6;0.64;0.62;0.62;0.55;0.41;0.39;0.33];

%% Initial conditions
x0 = [1; 2; 3];

%% Bounds
lb = [-1; -2; -3];
ub = [0.1; 0.2; 0.3];

%% Fitting options
options = optimset( 'Display', 'iter' );

%% Fit all three parameters
x = lsqcurvefit( myfun, x0, xdata, ydata, lb, ub, options )

function ydata = myfun_fix(value2, xdata)  % value2 should be scalar

    value1_fix = 0; % initialize to your value1
    value3_fix = 0; % initialize to your value3

    x = [value1_fix, value2, value3_fix];

    ydata = myfun(x, xdata);
end
 value2=x(2);
 [x2]= lsqcurvefit(@myfun_fix, value2, xdata, ydata, lb, ub, options);

end

[Star Strider]

Letting only ‘x(2)’ be nonzero is essentially:

y = x(2)*xdata;

That is linear, and so ‘x(2)’ can easily be calculated as:

xdata = [0.036;0.14;0.42;0.49;0.66;0.68;0.76;0.79;0.8;0.85;0.92;0.93;0.96];
ydata = [0.83;1;0.66;0.53;0.58;0.6;0.64;0.62;0.62;0.55;0.41;0.39;0.33];

x2 = xdata(:) \ ydata(:)

producing:

x2 =
   0.687942250557827

Nothing else is required.

[Deniz Toprak]

Hi Star Strider,

It has been a while thank you for your answers. I have been working on my code but I can't get the best results from optimization. As a said before I have 7 variable which wanted to solve and I have 2500-10 000 data set to find these variable. My code first solve non-linear equations with fsolve then it uses data set to find these parameter. In order to determine the wrong part of the code , I decided to use 4 variable with 2500 data set but I didn't get the true parameter. Can you suggest me a different method ?

[Star Strider]

My pleasure!

I do not remember what we were doing with this. It has actually been months since this began.

If you are having problems getting appropriate parameter estimates, I would use the ga (genetic algorithm) function. It is usually better at finding appropriate parameters since it does not use gradient-descent algorithms.

However your problem must always be well-posed, and the function you are fitting must be a realistic approximation of the process or function that describes the data. The ga documentation is straightforward, and there are a number of other examples using ga on MATLAB Answers (some of which I wrote), that will help you understand how to use it most effectively.

Answer 2

[John D'Errico]

That you WANT the residual norm to be low is not really that relevant. Pick anything you may choose to want. For example, suppose you WANT to win the lottery. Is that relavant to winning the lottery? Not really. I suppose if you buy enough lottery tickets, it might be. :)

Not all models can be used to fit any set of data you would choose to fit them to. If that were true, then the art of mathematical modeling would be truly easy to practice.

The residual norm that you hope to see is extemely tiny, compared to that which you got. That probably implies that if it is possible to achieve the desired residual norm, the data was actually constructed to have no error. If this is real measured data, then such a residual norm improvement would seem to be highly unlikely.

There are several common reasons why such a modelling erffort fails - either the model is simply inappropriate to the data under study, or the starting values were poor, or the model was simply coded improperly. I cannot know which of these cases is true, or if it is just that you misunderstand what resnorm as it is returned means.

My guess is it may be most likely you misunderstand the resnorm parameter. I think you may be thinking of that as a convergence tolerance, almost a tolerance that you can apply. That is not what it means. The residual norm would be simply the norm of the errors for the model, as compared to the target values you provide.

In your duplicate question, you state that you used multi-start with lsqcurvefit, but you actually got poorer results than if you gave it your own starting values. That may simply mean your starting values were better than those chosen randomly using multi-start. multi-start has no intelligence, no understanding about your model. And a high dimensional space is a big place to search.

This is not a problem with lsqcurvefit, but most likely a poorly fitting model for this data, and perhaps a misunderstanding of what the returns mean.