Finding 2 independent input variables that provides mimimum of a function

I am trying to minimize the return value from the function called "blackbox" using the Matlab optimization function "fmincon". To make it simple, the "blackbox" function takes two scalar variables and returns one scalar output, and there is no equality or inequality relationship with these two variables. However, there are fixed maximum and minimum values for the two variables. 2Variables is 1 X 2 matrix containing two scalar values. The syntax that I used is as below:

fmincon(@blackbox,2Variables,[],[],[],[],[1000,5000],[2000,10000],[],optimset('TolX',1e-3,'LargeScale','off')

The challenge is that the function doesn't seem to optimize the '2Variables'. However, I am getting the message such as:

Optimization terminated successfully: Search direction less than 2*options.TolX and maximum constraint violation is less than options.TolCon.

Do you know how to modify either function syntax or option settings to obtain any reasonably optimized values for '2Variables'?

Answer 1

[Walter Roberson]

Remember, fmincon is a local minimizer. You should expect that blackbox functions might have arbitrary numbers of arbitrary deep local minima. A global optimizer from the Global Optimization Toolbox has a better chance of finding a "good" minima -- though of course with any black-box function you can never know if you have reached the global minimum.

Answer 2

[Leno]

Thank you for your answer. I agree that I would have a better chance of finding a "good" minima by using matlab function such as fminsearch. However, I couldn't find a way to set boundaries (min and max) for the two input variables using fminsearch function. Thus, I considered using fmincon function. Are there any good matlab function to use to simultaneously optimize the two variables with fixed min and max?

[Walter Roberson]

fminsearch is not a global minimizer. For a global minimizer, see for example http://www.mathworks.com/help/toolbox/gads/globalsearch.run.html

None of the functions provided with base MATLAB are global minimizers. Plausibly, though, there is a global minimizer in the MATLAB File Exchange.

[Leno]

Thank you for the correction. I tried to copy and paste the example in the link provided in order to use the global minimizer. I keep seeing the error message "??? Undefined function or variable 'GlobalSearch'" for the line gs = GlobalSearch.

Also, in the initial question, the message that I get is "Optimization terminated successfully:

Search direction less than 2*options.TolX and

maximum constraint violation is less than options.TolCon

No Active Constraints". I missed the part "No Active Consraints". Does it provide any hint why I do not obtain the results that I expect?

[Walter Roberson]

If GlobalSearch is coming up undefined, you probably do not have the optional Global Optimization Toolbox.

[Leno]

Does it return the message "No Active Constraints" since there is no equality or inequality relationship in the following syntax?

fmincon(@blackbox,2Variables,[],[],[],[],[1000,5000],[2000,10000],[],optimset('TolX',1e-3,'LargeScale','off')

Answer 3

[Matt Tearle]

Given that blackbox is a function of two variables, why not do some visualization to see what it looks like?

x = linspace(xmin,xmax);
y = linspace(ymin,ymax);
[X,Y] = meshgrid(x,y);
[m,n] = size(X);
Z = zeros(m,n)
for k = 1:n
   for j = 1:m;
      Z(j,k) = blackbox(X(j,k),Y(j,k));
   end
end
contour(X,Y,Z)

[Walter Roberson]

Define any fixed mesh size, and I can come up with a blackbox function that has an arbitrarily deep minimum completely hidden within one of the mesh squares. It might be difficult to discover if the grid spacing gets down to eps() of the grid coordinates, but any arbitrarily thin downward spike can be clearly added to the blackbox function at any point. If the function was not black box you might have a choice of detecting this.

[Matt Tearle]

True, but in that case a numerical optimization routine will mostly likely miss it as well. So the problem is the nature of the blackbox function. I don't see why it's not a good idea to get a feel for the landscape. It might give Leno an idea of what the distribution of local minima looks like.

[Leno]

I already visualized the function values for the example that I was optimizing. The function looks like a partially rolled up paper.