fminunc optimisation values not always decreasing

Hi, I have a code that is aiming on minimising my function, which is basically the difference between two matrices, by rotating and scaling one of the matrices.

My question is that during the optimisation, the value to be minimised is not always reducing. I understand that the optimisation may be trying out a new scaling factor etc, but it never goes back to the previous, better option, but continues on. Is this a setting within the optimsation that I could change? They always seem to get stuck on a value for 7 optimisations then continues on...

I am worried that with this method, it could be strongly dependent on the starting point.

Any help would be great, thanks.

For an example, the sum square difference between values in the matrices are listed below;

 0.006
0.006
0.006
0.006
0.006
0.006
41.1635
41.1635
41.1635
41.1635
41.1635
41.1635
41.1635
0.4115
0.4115
0.4115
0.4115
0.4115
0.4115
0.4115
2.61E-04
2.61E-04
2.61E-04
2.61E-04
2.61E-04
2.61E-04
2.61E-04
2.54E-04
2.54E-04
2.54E-04
2.54E-04
2.54E-04
2.54E-04
2.54E-04
1.99E-04
1.99E-04
1.99E-04
1.99E-04
1.99E-04
1.99E-04
1.99E-04
2.56E-05
2.56E-05

[Walter Roberson]

How many variables is it optimizing over?

[awinka]

6; rotation in 3 directions and scaling in 3 directions. I've gone through and checked, and it doesn't always finish on the smallest number acheived during optimisation, but often slightly higher value.

At the moment the optimisations tend to finish before completing on all optimisations;

 'fminunc stopped because the size of the current step is less than the default value of the step size tolerance'

Thanks

Answer 1

[Alan Weiss]

I suggest that you consult the documentation.

In particular, you can try different algorithms, and even better, try to provide derivative information.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

Answer 2

[John D'Errico]

You state: "I am worried that with this method, it could be strongly dependent on the starting point. "

Any optimizer will be strongly dependent on the start point. Ok, a global one will be less so, but fminunc is not that class of optimizer.

As for the value getting "stuck"for 7 objective function evals, then continues on, hmm. Lets see.

fminunc will compute a gradient estimate. You have 6 unknowns. It will use a finite difference approximation, so needing a total of 7 function evals to estimate a gradient. All 7 points will be VERY close together, since this is how you would estimate a derivative. I wonder if that explains why it seems to be "stuck"?

Finally, you claim that it is not giving the smallest value. It seems the last value reported there was 2.56e-5. Is that not the least value in that series of function evaluations? It is less than 1.99e-4 after all.

So please explain what the problem is, as I don't see any problems here so far.