What is the difference between "Bestpoint hyperparameter" and "minimum error hyperparameter" in hyperparameter optimization?

In Regression Learner App I use Optimizable GPR option to do a multivariate regression. Once the hyperparameter optimization is completed the Minimum MSE plot shows two highlighted points one is called "Bestpoint hyperparameter" (highlighted as solid red square on the plot), and a second one called "minimum error hyperparameter" (solid orange circle). I have four questions:

1 - What is the relative significance of these points (i.e., which set of hyperparameters is more significant)?

2 - When the optimizable model is saved which set of hyperparameters is used in the saved model?

3 – Does the RMSE value appearing in front of the model selection tab after the completion of the optimization represent the performance of the optimized model and if so what hyperparameter set does it correspond to?

4 - Why not use Minimum RMSE instead of MSE for the plot after all the regression models are ranked based on their final RMSE value?

Answer 1

[MathWorks Support Team]

Here are the answers to your questions:

1. Bestpoint hyperparameters and Minimum error hyperparameters may use different criteria due to different optimization objectives. For example, the optimized hyperparameters do not always provide the observed minimum MSE. When the app performs hyperparameter tuning using Bayesian optimization, it chooses the set of hyperparameter values that minimizes an upper confidence interval of the MSE objective model, rather than the set that minimizes the MSE. For more information, you can access the release-specific documentation by executing the following command in the MATLAB R2020b command window:

>> web(fullfile(docroot, 'stats/bayesianoptimization.bestpoint.html'))

As for Minimum error hyperparameters, the yellow point indicates the iteration that corresponds to the hyperparameters yielding the observed minimum MSE. For more information, see the 'Criterion','min-observed' name-value pair argument of bestPoint. Bestpoint hyperparameter and Minimum error hyperparameters may use different criteria for different purposes, and there is no absolute significance difference between them. For more details on hyperparameter optimization in the regression learner app, execute the following command in MATLAB R2020b:

>> web(fullfile(docroot, 'stats/hyperparameter-optimization-in-regression-learner-app.html'))

2. When the app finishes tuning model hyperparameters, it returns a model trained with the optimized hyperparameter values (Bestpoint hyperparameters). For more information, refer to the documentation by executing following command in MATLAB R2020b:

>> web(fullfile(docroot, 'stats/hyperparameter-optimization-in-regression-learner-app.html'))

3. Yes. The RMSE is shown in the result. To inspect the optimization results of a trained optimizable model, select the model in the 'History' list and look at the 'Current Model' pane. The model is trained with the optimized hyperparameter values (Bestpoint hyperparameters).

4. The Minimum MSE is shown in the plot because the model is optimized by minimizing the MSE value for iterations. Therefore, it is more direct to show the optimization process by MSE value.

Please follow the link below to search for the required information regarding the current release:

https://www.mathworks.com/help/