Bayesopt: min observed objective not decreasing

I am optimizing the parameters of a neural network using bayesopt in Matlab. Interestingly, the min observed objective sometimes jumps up which is counterintuitive since I expect that number to be non-increasing. Would appreciate any insights on this. By the way, I used a coupled constraint model: whenever the constraint is not satisfied the objective function will return an error. I also as 'IsObjectiveDeterministic' to true. Thanks!

[Greg Heath]

Unclear explanation.

Greg

[Stephen Wu]

To be more clear, I expect that the min observed objective to be non-increasing with respect to the number of iterations. But it is not the case in my problem (as can be seen in the plot, the min observed objective sometimes bumps up). Thanks.

Answer 1

[Alan Weiss]

Take a look at the note in the documentation of Coupled Constraints.

Alan Weiss

MATLAB mathematical toolbox documentation