Hyperparameter optimization using bayesopt - what does 'Transform' value of 'log' do in optimizableVariable?

MATLAB documentation on https://www.mathworks.com/help/stats/optimizablevariable.html

gives the following example for setting an optimizableVariable with log transform:

Integer variable from 1 to 1000 on a log scale:

var2 = optimizableVariable('ivar',[1 1000],'Type','integer','Transform','log')

var2 =

optimizableVariable with properties:

Name: 'ivar'

Range: [1 1000]

Type: 'integer'

Transform: 'log'

Optimize: 1

Does this search the following values for the 'ivar' hyperparameter? 1, 10, 100, and 1000?

Could someone explain how this works?

Answer 1

[Don Mathis]

The Bayesian Optimization algorithm will model that variable on a log scale. The Gaussian Process model will fit a kernel scale parameter that applies to the log of the variable. All integer values 1..1000 are still available.

[Louis]

so MATLAB's above 'var2' case will try integer values between log(1)=0 and log(1000)= 6.9078, hence making the searce space for that variable to be [0, 1, 2, 3, 4, 5, 6] ?

[Don Mathis]

No, var2 will try all integer values 1:1000. Those are the values that will be passed to the objective function.

Internally, the bayesopt function will transform the integers 1:1000 into the real range log(1:1000) for the purposes of modeling the objective function with a Gaussian Process (GP) model. The GP model will see var2 in its transformed space, log(1:1000). The main consequence has to do with the kernel scale parameter of the GP. GP learns a separate Kernel Scale for each variable, which is used to measure how "similar" two points are. For example, if KernelScale=1, then the GP will consider the points 2 and 3 to be as similar to each other as the points 5 and 6, because both pairs differ by one KernelScale. Because the GP is operating in the log space, it really means that the GP considers the var2 values exp(2) and exp(3) to be a similar to each other as the points exp(5) and exp(6). Rounded to the nearest integer, it considers 3 and 7 to be as similar as 148 and 403.