fmincon not optimizing over one variable

I am trying to use fmincon to optimize a function over four variables. One of the variables is never being varied. I have tried just optimizing over the one variable, and fmincon stops after the initial input is evaluated.

I am using fmincon with just upper and lower limits for all of the variables. Within the function being optimized, all of the varibales are being passed to ode45. There aren't any rounding, floor, or ceiling commands within the function, so that should not be causing the issue.

For comparison sake, I tried fminsearch, which did seem to be optimizing over all of the variables, but I really need to use a constrained optimization on this problem.

Are there any reasons why fmincon would not optimize over just one of the variables in a function, regardless of what the initial guess or bounds are?

[Sean de Wolski]

Perhaps it's overconstrained?

Answer 1

[Steve Grikschat]

It may appear to fmincon that the function has a zero derivative with respect to the variable in question. How sensitive is your objective function to small changes in this variable?

It's likely that fmincon's finite difference gradient estimation routine is varying this variable by small amounts which do not change the ODE results significantly.

Try tuning (most likely increasing) the finite difference control options, DiffMinChange and DiffMaxChange (also FinDiffRelStep if you have a newer version of the toolbox).

http://www.mathworks.com/help/toolbox/optim/ug/f19175.html

Also, consider changing the tolerances (TolFun, TolX, TolCon) used by the solver to reflect the overall sensitivity of your function.

Answer 2

[Marcel Clementino]

Hi,

I'm facing a similar problem using fmincon with SQP algorithm. But I guess in my case, the variable which assumes the initial guess value has this behavior due to it's boundary. In my problem, three variables have bounds in the order of 1e-3 and one variable in the order of 1e3. So when I run the optimization code, there isn't enough variation in the latter one, because the stepsize is too small. If I set the DiffMinChange options with a greater stepsize, then the algorithm does't run because the other variables are smaller than the stepsize.

Is there any way of setting a step size which can account for both variables orders? Or does anyone suggests another approach to achieve the optimal solution with the bounds ranging so much?

Thanks in advance!

[Steve Grikschat]

Look into FinDiffRelStep. It accepts a vector of values which would allow you to specify different magnitudes for each of the variables. (Make sure you reset DiffMinChange a little bit if you set FinDiffRelStep).

http://www.mathworks.com/help/toolbox/optim/ug/f19175.html

Another possibility is that you can scale your variables (and their effect on the objective function and constraints) so that they are on similar scales (on the order of 1 if you can).

Answer 3

[Andrew Newell]

If your function has zero derivative with respect a variable, it won't vary that variable.

[Catherine]

The function does not have a zero derivative with respect to this variable. The output will vary from approximately -5 to -10 when I vary the initial guess for the variable from 1 to 4.

[Andrew Newell]

I could continue to play a guessing game, or you could post your code.

Answer 4

[Catherine]

Thanks for all the suggestions.

When I plotted the results of the function using a very fine step size, I doscovered that what I thought was a smooth function actually is not smooth. I'm thinking that this is most likely the reason why it is not changing. I'll try playing with some of the tolerances to see if I can get it to jump over these small irregularitites, but I'm thinking I will most likely just end up guessing different values and manually finding the min.

Answer 5

[Talfan Evans]

I recommend following Catherine's advice - plot your function for small changes and see whether it is continuous.

Check that your variables are stored as doubles - even storing as singles can cause the gradient check to fail, if the resolution isn't high enough.