Baysian Optimisation. Any way of increasing the sample size of candidates solutions to satisfy XConstraintFunction in bayesopt()?

I'm investigating the suitability of bayesopt() for problems of increasing dimensionality and range within the variables. As both of these increase, the number of solutions in the solution space grows quickly, whilst the number of feasible solutions grows less quickly. I have included a constraint function as an input to bayesopt(). The bayeopt() algorithm checks whether this can be satisfied be inputting 10,000 possible solutions (I assume drawn from the solution space) to it and seeing if any are feasible. However as mentioned before, the solution space grows a lot more quickly than the number of feasible solutions. This means that it becomes very unlikely to draw a feasible solution in 10,000 selections. If a feasible solution is not found, bayesopt() considers there to be no feasible solutions and terminates the algorithm. However, I know that there are feasible solutions so would like the algorithm to continue so I can see how it performs. Is there a way of increasing the 10,000 limit? I can see the variable that would be need to changed in the bayesopt() code, but when I try to change it I get a pop up box saying that I dont have access to alter the file.

[Bob Hickish]

Furthermore, it is strange that one can provide an InitialX to bayesopt() but the ConstrainFunction checking process doesn't evaluate this. Is there anyway of passing InitialX to the constraint function during the checking process to convince bayesopt() that the constraint function is satisfiable? Thanks

[LANG Wu]

Hi Bob,

Have you solved the issue that passing InitialX to the constraint function?

Thanks,

Lang

Answer 1

[Don Mathis]

Unfortunately the API doesn't allow that, but you can assign into the Options object once you have one. The bayesopt function creates one right away, so you could do something like this:

x = optimizableVariable('x',[-8,8]);
fun = @(T)sin(T.x);
BO = my_bayesopt(fun, x)

function Results = my_bayesopt(ObjectiveFcn, VariableDescriptions, varargin)
Options = bayesoptim.BayesoptOptions(ObjectiveFcn, VariableDescriptions, varargin);
Options.NumRestartCandidates = 1e6;
Results = BayesianOptimization(Options);
end

[Don Mathis]

Or you could directly modify the bayesopt.m source file in-place. The risk, of course, in modifying source files instead of overriding them is that you might mess up your MATLAB installation.

In Windows 10 you can right-click on the MATLAB icon, choose Run as Administrator, and then use the MATLAB editor to edit and save the file:

edit bayesopt

[Bob Hickish]

Thanks for your reply Don, I'm making this response incase anyone finds this question in the future, and also to ask a question. Your initial suggestion seemed better to me, and I tried it, but unfortunately it does not seem to have worked. I think this is because the number of the solutions that are checked against the constraints is actually hard-coded (the variable N in the code below, taken from BayesianOptimization.m).

    function checkXConstraintFcnSatisfiability(this)
            % Check satisfiability of the XConstraint
            N = 10000;
            X = iUniformPointsWithinBounds(N, this.PrivOptions.VarSpec.LBTrans, ...
                this.PrivOptions.VarSpec.UBTrans);
            NumFeas = sum(isInputFeasible(this, X));
            if NumFeas == 0
                bayesoptim.err('NoFeasiblePoints', N);
            elseif NumFeas < .01*N
                bayesoptim.warn('FewFeasiblePoints', N, NumFeas);
            end
        end

Running MATLAB as an administrator and altering the value of N in the BayesianOptimization.m file directly did work (as you suggested after). I don't suppose you are able to tell me why the value of 10,000 was chosen? This seems a rather a strange way to check whether the ConstraintFunction can be satisfied. I ask because I wonder if I have not realised an inherent limitation in the bayesopt() algorithms which makes this choice of 10,000 sensible?

Thanks,

Bob

[Don Mathis]

Hi Bob,

Thanks for pointing this out. That 10000 should really be in the Options object so that the first method works. You could edit its value in the code as you're doing, or you could just comment out the call to checkXConstraintFcnSatisfiability since it's just there to head off later errors during optimization if no feasible points can be found. But since you're increasing NumRestartCandidates anyway, you could just omit this check.

There's nothing sacred about the number 10000 here or in NumRestartCandidates. It's a compromise between speed and coverage.

There's also somewhat of an implicit assumption that the XConstraint is satisfied in a non-miniscule fraction of the search space. If it's not, we would encourage reparameterization of the problem to try to improve that. In your case it looks like you're explicitly studying a case where Prob(XSatisfied) goes to zero. I'm not exactly sure what will happen, though I do know that the XConstraintFcn gets evaluated on all NumRestartCandidates points each iteration, but the Acquisition function only gets evaluated on the feasible points found. So if your number of feasible points remains constant as P->0, you may only see slowdown due to the time required to evaluate the XConstraint.

[Bob Hickish]

Ah ok, thank you Don for the clarification on the 10000, NumRestartCandidates and XConstraintFcn. I'm back again to ask if you can think of another solution that does not involve editing the MATLAB program files. I ask because, whilst your earlier solution was fine for working on my local machine, I am now wanting to run my script on an HPC for which I am not an administrator and so hence don't have access to altering the code within bayesopt(). Before when I have had similar problems you have suggested that I create a new folder in the working directory with the same name as the function so that it comes up on the path before the actual function. Would something similar work here? Somehow removing the call to checkXConstraintFcnSatisfiability would be best, but any suggestions appreciated!

[Don Mathis]

You can define your own modified version of the BayesianOptimization class and have your "my_bayesopt" function call that. Put a copy of BayesianOptimization.m into your local folder and rename it MyBayesianOptimization.m. Then, inside MyBayesianOptimization.m,

• Rename the class on line 1:

classdef MyBayesianOptimization

• Rename the constructor:

function this = MyBayesianOptimization(Options)

• Make checkXConstraintFcnSatisfiability do nothing:

function checkXConstraintFcnSatisfiability(this)
end

Inside your my_bayesopt.m, make it call your new class:

function Results = my_bayesopt(ObjectiveFcn, VariableDescriptions, varargin)
Options = bayesoptim.BayesoptOptions(ObjectiveFcn, VariableDescriptions, varargin);
Options.NumRestartCandidates = 1e6;
Results = MyBayesianOptimization(Options);
end

This works for me in MATLAB version R2016b. Let me know how it goes.

[Bob Hickish]

I have just got round to implementing this. It has worked perfectly, thanks.