lsqnonlin seems stuck at the same residual value,norm step and first order optimality for a while now

Hi hi, my lsqnonlin seems stuck after 7th iteration.

Iteration  Func-count     f(x)          step          optimality     CG-iters
     0          9        0.247067                        0.0282
     1         18         0.23938             10         0.0413            0
     2         27        0.238046             20         0.0464            0
     3         36        0.236123             40         0.0529            0
     4         45        0.232245             80          0.124            0
     5         54        0.221193            160         0.0699            0
     6         63        0.208611            320         0.0877            0
     7         72        0.208611        34.3923         0.0877            0
     8         81        0.208611        34.3923         0.0877            0
     9         90        0.208611        34.3923         0.0877            0
    10         99        0.208611        34.3923         0.0877            0
    11        108        0.208611        34.3923         0.0877            0
    12        117        0.208611        34.3923         0.0877            0
    13        126        0.208611        34.3923         0.0877            0
    14        135        0.208611        34.3923         0.0877            0
    15        144        0.208611        34.3923         0.0877            0
    16        153        0.208611        34.3923         0.0877            0

what does this mean? Thanks in advance.

[Sargondjani]

does lsqnonlin stop here (ie. exit) or does it really get stuck?

i dont know what the criterium if for the optimality but the optimality could still be improving with each iteration... it depends a bit on your stopping criterim (how precise)

[J J]

Hi Sargondjani, Thanks for the comment. No, it goes on iterating indefinitely. I have left it on for 24 hours and it is stuck at this point until the criteria MaxIter or MaxFunEvals is exceeded. Is it still iterating? My TolFun or TolX are not unreasonably small; or are they? Are the outcome of this optimization reliable?

options = optimset('Diagnostics','on','Display','iter','TolFun',10^-5,'TolX',10^-5,'LargeScale','on','MaxIter',800,'MaxFunEvals',10000);

[Sargondjani]

the the tolerances are ok (quite large actually, i would say)

did you try different starting values?

it is quite weird for the optimizer to get stuck without exiting. you seem to have only 3 variables, so that should also not be the the problem...

most logical explanation seems that there is some non-differentiablility around the point where it gets stucks (so it goes in loops)

how long does one iteration/function evaluation take?

maybe you should give us more information on what you are trying to solve.. a short outline of your model.

[J J]

Hi Sargondjani,

yes, I have tried different starting values, and this seems to help. e.g. for my simulations, starting at the true values can be all right; but then lsqnonlin can get stuck with small deviations from that true value. so , you may be correct about the non-differentiability issue. But is there a way to set up a criteria for lsqnonlin to jump to a different point when this happens?

no, actually I have 10 parameters to be estimated. This behavior happens more for my other model with 12 parameters. and I only have 120 data points. yes, too many parameters.

each iteration takes about 3 sec-1min. (as function-count increases with iteration number, I believe).

[Marylen Sun]

Hello, did you find a way to set up a criteria for lsqnonlin to jump to a different point when the non-differentiability happen, please?

Answer 1

[Matt J]

For a 3-variable problem known to have non-differentiabilities, it would make sense to use fminsearch instead of lsqnonlin. There are no differentiability requirements in fminsearch, though it works better when there is at least continuity.

It may also be practical in a 3-variable problem to sample the objective on a discrete grid and do a simple discrete search for the minimum. This is especially advisable if the function evaluations can be done in a fast, vectorized manner. If nothing else, this is often a useful way to find a good initial guess for iterative solvers like fminsearch.