hyperparameters

Variable descriptions for optimizing a fit function

Syntax

VariableDescriptions = hyperparameters(FitFcnName,predictors,response)

VariableDescriptions = hyperparameters(FitFcnName,predictors,response,LearnerType)

Description

VariableDescriptions = hyperparameters(FitFcnName,predictors,response) returns the default variables for the given fit function. These are the variables that apply when you set the OptimizeHyperparameters name-value argument to "auto".

VariableDescriptions = hyperparameters(FitFcnName,predictors,response,LearnerType) returns the variables for an ensemble fit with specified learner type. This syntax applies when FitFcnName is "fitcecoc", "fitcensemble", or "fitrensemble".

Examples

Obtain Default Hyperparameters

Obtain the default hyperparameters for the fitcsvm classifier.

Load the ionosphere data.

load ionosphere

Obtain the hyperparameters.

VariableDescriptions = hyperparameters('fitcsvm',X,Y);

Examine all the hyperparameters.

for ii = 1:length(VariableDescriptions)
    disp(ii),disp(VariableDescriptions(ii))
end

     1

  optimizableVariable with properties:

         Name: 'BoxConstraint'
        Range: [1.0000e-03 1000]
         Type: 'real'
    Transform: 'log'
     Optimize: 1

     2

  optimizableVariable with properties:

         Name: 'KernelScale'
        Range: [1.0000e-03 1000]
         Type: 'real'
    Transform: 'log'
     Optimize: 1

     3

  optimizableVariable with properties:

         Name: 'KernelFunction'
        Range: {'gaussian'  'linear'  'polynomial'}
         Type: 'categorical'
    Transform: 'none'
     Optimize: 0

     4

  optimizableVariable with properties:

         Name: 'PolynomialOrder'
        Range: [2 4]
         Type: 'integer'
    Transform: 'none'
     Optimize: 0

     5

  optimizableVariable with properties:

         Name: 'Standardize'
        Range: {'true'  'false'}
         Type: 'categorical'
    Transform: 'none'
     Optimize: 1

Change the PolynomialOrder hyperparameter to have a wider range and to be used in an optimization.

VariableDescriptions(4).Range = [2,5];
VariableDescriptions(4).Optimize = true;
disp(VariableDescriptions(4))

  optimizableVariable with properties:

         Name: 'PolynomialOrder'
        Range: [2 5]
         Type: 'integer'
    Transform: 'none'
     Optimize: 1

Obtain Ensemble Hyperparameters

Obtain the default hyperparameters for the fitrensemble ensemble regression function.

Load the carsmall data.

load carsmall

Use Horsepower and Weight as predictor variables, and MPG as the response variable.

X = [Horsepower Weight];
Y = MPG;

Obtain the default hyperparameters for a Tree learner.

VariableDescriptions = hyperparameters('fitrensemble',X,Y,'Tree');

Examine all the hyperparameters.

for ii = 1:length(VariableDescriptions)
    disp(ii),disp(VariableDescriptions(ii))
end

     1

  optimizableVariable with properties:

         Name: 'Method'
        Range: {'Bag'  'LSBoost'}
         Type: 'categorical'
    Transform: 'none'
     Optimize: 1

     2

  optimizableVariable with properties:

         Name: 'NumLearningCycles'
        Range: [10 500]
         Type: 'integer'
    Transform: 'log'
     Optimize: 1

     3

  optimizableVariable with properties:

         Name: 'LearnRate'
        Range: [1.0000e-03 1]
         Type: 'real'
    Transform: 'log'
     Optimize: 1

     4

  optimizableVariable with properties:

         Name: 'MinLeafSize'
        Range: [1 50]
         Type: 'integer'
    Transform: 'log'
     Optimize: 1

     5

  optimizableVariable with properties:

         Name: 'MaxNumSplits'
        Range: [1 99]
         Type: 'integer'
    Transform: 'log'
     Optimize: 0

     6

  optimizableVariable with properties:

         Name: 'NumVariablesToSample'
        Range: [1 2]
         Type: 'integer'
    Transform: 'none'
     Optimize: 0

Change the MaxNumSplits hyperparameter to have a wider range and to be used in an optimization.

VariableDescriptions(5).Range = [1,200];
VariableDescriptions(5).Optimize = true;
disp(VariableDescriptions(5))

  optimizableVariable with properties:

         Name: 'MaxNumSplits'
        Range: [1 200]
         Type: 'integer'
    Transform: 'log'
     Optimize: 1

Input Arguments

FitFcnName — Name of fitting function

"fitcdiscr" | "fitcecoc" | "fitcensemble" | "fitcgam" | "fitckernel" | "fitcknn" | "fitclinear" | "fitcnb" | "fitcnet" | "fitcsvm" | "fitctree" | "fitrensemble" | "fitrgam" | "fitrgp" | "fitrkernel" | "fitrlinear" | "fitrnet" | "fitrsvm" | "fitrtree"

Name of the fitting function, specified as one of the listed classification or regression fit function names.

• Classification fit functions: fitcdiscr, fitcecoc, fitcensemble, fitcgam, fitckernel, fitcknn, fitclinear, fitcnb, fitcnet, fitcsvm, fitctree

• Regression fit functions: fitrensemble, fitrgam, fitrgp, fitrkernel, fitrlinear, fitrnet, fitrsvm, fitrtree

If FitFcnName is "fitcecoc", "fitcensemble", or "fitrensemble", then you also need to specify the learner type in the LearnerType argument.

Example: "fitctree"

predictors — Predictor data

matrix with D predictor columns | table with D predictor columns

Predictor data, specified as a matrix with D predictor columns or a table with D predictor columns, where D is the number of predictors.

Example: X

Data Types: double | logical | char | string | table | cell | categorical | datetime

response — Class labels or numeric response

grouping variable | scalar

Class labels or numeric response, specified as a grouping variable (see Grouping Variables) or a scalar.

Example: Y

Data Types: single | double | logical | char | string | cell

LearnerType — Learner type for ensemble fit

"discriminant" | "ensemble" | "kernel" | "knn" | "linear" | "svm" | "tree" | template of a listed learner

Learner type for an ensemble fit, specified as "discriminant", "ensemble", "kernel", "knn", "linear", "svm", "tree", or a template of one of these learners. Use this argument when FitFcnName is "fitcecoc", "fitcensemble", or "fitrensemble".

For "fitcensemble", you can specify only "discriminant", "knn", "tree", or an associated template.

For "fitrensemble", you can specify only "tree" or its associated template.

Example: "tree"

Output Arguments

VariableDescriptions — Variable descriptions

vector of optimizableVariable objects

Variable descriptions, returned as a vector of optimizableVariable objects. The variables have their default parameters set, such as range and variable type. All eligible variables exist in the descriptions, but the variables unused in the "auto" setting have their Optimize property set to false. You can update the variables by using dot notation, as shown in Examples.

Version History

Introduced in R2016b

R2024a: Optimize hyperparameters of ensemble binary learners when using ECOC classification

fitcecoc supports hyperparameter optimization when you use ensemble binary learners. You can specify the OptimizeHyperparameters name-value argument when the Learners value is "ensemble" or an ensemble template created using the templateEnsemble function.

• When you perform hyperparameter optimization using ensemble binary learners, the ensemble method must be "AdaBoostM1", "GentleBoost", or "LogitBoost", and the ensemble weak learners must be trees.

• The "ensemble" value corresponds to an ensemble that uses an adaptive logistic regression aggregation method (LogitBoost), 100 learning cycles, and tree weak learners.

fitcecoc uses a default value of 70 for MaxObjectiveEvaluations when performing Bayesian optimization with ensemble binary learners. For more information, see HyperparameterOptimizationOptions.

You can use the hyperparameters function to see the eligible and default hyperparameters for the ensemble binary learners. Additionally, you can use the function to adjust the hyperparameters and their ranges. Specify "fitcecoc" as the fitting function name and "ensemble" or an ensemble template as the learner type.