Main Content

ベイズ最適化および ASHA 最適化による分類器の自動選択

この例では、fitcauto を使用し、指定した学習予測子と応答データに基づいてさまざまなハイパーパラメーターの値をもつ分類モデルのタイプの選択を自動的に試す方法を示します。既定では、この関数はモデルの選択と評価にベイズ最適化を使用します。学習データ セットに多数の観測値が含まれている場合は、代わりに非同期連続半減アルゴリズム (ASHA) を使用できます。最適化が完了すると、fitcauto は、データセット全体で学習済みの、新しいデータの分類に最適であると予測したモデルを返します。検定データに対するモデルの性能をチェックします。

標本データの読み込み

この例では census1994.mat に保存されている 1994 年の国勢調査データを使用します。このデータセットは、個人の年収が $50,000 を超えるかどうかを予測するために使用できる、米国国勢調査局の人口統計情報から構成されています。

学習データ adultdata および検定データ adulttest を含む、標本データ census1994 を読み込みます。学習データセットの最初の数行をプレビューします。

load census1994
head(adultdata)
ans=8×15 table
    age       workClass          fnlwgt      education    education_num       marital_status           occupation        relationship     race      sex      capital_gain    capital_loss    hours_per_week    native_country    salary
    ___    ________________    __________    _________    _____________    _____________________    _________________    _____________    _____    ______    ____________    ____________    ______________    ______________    ______

    39     State-gov                77516    Bachelors         13          Never-married            Adm-clerical         Not-in-family    White    Male          2174             0                40          United-States     <=50K 
    50     Self-emp-not-inc         83311    Bachelors         13          Married-civ-spouse       Exec-managerial      Husband          White    Male             0             0                13          United-States     <=50K 
    38     Private             2.1565e+05    HS-grad            9          Divorced                 Handlers-cleaners    Not-in-family    White    Male             0             0                40          United-States     <=50K 
    53     Private             2.3472e+05    11th               7          Married-civ-spouse       Handlers-cleaners    Husband          Black    Male             0             0                40          United-States     <=50K 
    28     Private             3.3841e+05    Bachelors         13          Married-civ-spouse       Prof-specialty       Wife             Black    Female           0             0                40          Cuba              <=50K 
    37     Private             2.8458e+05    Masters           14          Married-civ-spouse       Exec-managerial      Wife             White    Female           0             0                40          United-States     <=50K 
    49     Private             1.6019e+05    9th                5          Married-spouse-absent    Other-service        Not-in-family    Black    Female           0             0                16          Jamaica           <=50K 
    52     Self-emp-not-inc    2.0964e+05    HS-grad            9          Married-civ-spouse       Exec-managerial      Husband          White    Male             0             0                45          United-States     >50K  

各行には、成人 1 人の人口統計情報が格納されています。最後の列 salary は個人の年収が $50,000 以下か、$50,000 を超えるかどうかを示します。

adultdataadulttest から欠損値を含む観測値を削除します。

adultdata = rmmissing(adultdata);
adulttest = rmmissing(adulttest);

ベイズ最適化による自動モデル選択の使用

fitcauto を使用して、adultdata のデータに適切な分類器を見つけます。既定では、fitcauto はベイズ最適化を使用してモデルとそのハイパーパラメーターの値を選択し、各モデルの交差検証の分類誤差 (Validation loss) を計算します。既定の設定では、fitcauto は、最適化のプロット、および最適化の結果の反復表示を提供します。これらの結果を解釈する方法の詳細については、Verbose の表示を参照してください。

観測値の重みを設定し、ベイズ最適化を並列実行するよう指定します。これには Parallel Computing Toolbox™ が必要です。並列でのタイミングに再現性がないため、並列ベイズ最適化で再現性のある結果が生成されるとは限りません。最適化の複雑度に応じて、特に大きなデータセットでは、この処理に時間がかかる場合があります。

bayesianOptions = struct("UseParallel",true);
[bayesianMdl,bayesianResults] = fitcauto(adultdata,"salary","Weights","fnlwgt", ...
    "HyperparameterOptimizationOptions",bayesianOptions);
Warning: Data set has more than 10000 observations. Because ASHA optimization often finds good solutions faster than Bayesian optimization for data sets with many observations, try specifying the 'Optimizer' field value as 'asha' in the 'HyperparameterOptimizationOptions' value structure.
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 8).
Copying objective function to workers...
Done copying objective function to workers.
Learner types to explore: ensemble, nb, svm, tree
Total iterations (MaxObjectiveEvaluations): 120
Total time (MaxTime): Inf

|=======================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Estimated min   | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | validation loss |              |                                         |
|=======================================================================================================================================================|
|    1 |       8 | Best   |    0.14546 |            9.0638 |         0.14546 |         0.14546 |         tree | MinLeafSize:                        118 |
|    2 |       7 | Accept |    0.24677 |            11.074 |         0.14546 |         0.14704 |          svm | BoxConstraint:                   3.8856 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.00432 |
|    3 |       7 | Accept |    0.14868 |            9.9597 |         0.14546 |         0.14704 |         tree | MinLeafSize:                         26 |
|    4 |       8 | Accept |    0.15457 |            156.73 |         0.14546 |         0.14704 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  203 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                        105 |
|    5 |       8 | Accept |    0.18015 |            153.77 |         0.14546 |         0.14704 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  239 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       2417 |
|    6 |       8 | Accept |    0.14921 |            233.35 |         0.14546 |         0.14704 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  258 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                        542 |
|    7 |       8 | Accept |    0.24677 |            113.54 |         0.14546 |         0.14704 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  298 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                      11733 |
|    8 |       8 | Best   |    0.14517 |            318.06 |         0.14517 |         0.14704 |          svm | BoxConstraint:                  0.10777 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.1321 |
|    9 |       8 | Accept |    0.15414 |            196.23 |         0.14517 |         0.14704 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  267 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                          1 |
|   10 |       8 | Accept |    0.24677 |            382.69 |         0.14517 |         0.14704 |          svm | BoxConstraint:                 0.043508 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     931.39 |
|   11 |       8 | Accept |    0.24677 |             393.3 |         0.14517 |         0.14704 |          svm | BoxConstraint:                 0.043508 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     931.39 |
|   12 |       8 | Accept |    0.16391 |            11.028 |         0.14517 |          0.1523 |         tree | MinLeafSize:                          5 |
|   13 |       8 | Accept |    0.17328 |            2.6893 |         0.14517 |         0.15762 |         tree | MinLeafSize:                        846 |
|   14 |       8 | Accept |    0.24677 |            1.4486 |         0.14517 |         0.17323 |         tree | MinLeafSize:                       8098 |
|   15 |       8 | Accept |    0.24169 |            425.67 |         0.14517 |         0.17323 |          svm | BoxConstraint:                  0.15557 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     72.871 |
|   16 |       8 | Accept |    0.24677 |            1.0011 |         0.14517 |         0.16798 |         tree | MinLeafSize:                      11604 |
|   17 |       8 | Accept |     0.1511 |            88.912 |         0.14517 |          0.1511 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                          0.67978 |
|   18 |       8 | Accept |    0.18099 |            162.32 |         0.14517 |         0.16694 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                            232.3 |
|   19 |       8 | Accept |    0.18403 |            2.8724 |         0.14517 |         0.16363 |         tree | MinLeafSize:                       2226 |
|   20 |       8 | Accept |    0.24677 |            9.3034 |         0.14517 |         0.16363 |          svm | BoxConstraint:                 0.019712 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                  0.0047701 |
|=======================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Estimated min   | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | validation loss |              |                                         |
|=======================================================================================================================================================|
|   21 |       8 | Accept |    0.15364 |            141.13 |         0.14517 |         0.16209 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           3.2128 |
|   22 |       8 | Accept |    0.16345 |            351.87 |         0.14517 |         0.16209 |          svm | BoxConstraint:                0.0017882 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     3.4862 |
|   23 |       8 | Accept |     0.1585 |            391.52 |         0.14517 |         0.16209 |          svm | BoxConstraint:                   19.675 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     205.34 |
|   24 |       8 | Accept |    0.18472 |            180.24 |         0.14517 |         0.16363 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           1676.9 |
|   25 |       8 | Accept |    0.15005 |            209.17 |         0.14517 |         0.15973 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  218 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         34 |
|   26 |       8 | Accept |    0.15429 |            172.14 |         0.14517 |         0.15973 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           3.6464 |
|   27 |       8 | Accept |      0.168 |            1.2833 |         0.14517 |         0.15973 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |                 |              | Width:                              NaN |
|   28 |       8 | Accept |    0.15045 |            225.03 |         0.14517 |         0.15445 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  235 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                          7 |
|   29 |       8 | Accept |      0.168 |            2.4337 |         0.14517 |         0.15445 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |                 |              | Width:                              NaN |
|   30 |       8 | Accept |      0.168 |            2.0997 |         0.14517 |         0.15445 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |                 |              | Width:                              NaN |
|   31 |       8 | Accept |    0.15761 |            198.66 |         0.14517 |         0.15445 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           5.6802 |
|   32 |       8 | Accept |    0.15055 |             214.5 |         0.14517 |          0.1515 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  202 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                          2 |
|   33 |       8 | Accept |    0.15429 |            143.42 |         0.14517 |          0.1515 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           3.6279 |
|   34 |       8 | Accept |    0.14984 |            211.73 |         0.14517 |         0.15084 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  271 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         13 |
|   35 |       8 | Accept |    0.15455 |            182.76 |         0.14517 |         0.15058 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  261 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                          1 |
|   36 |       8 | Accept |    0.24182 |            435.16 |         0.14517 |         0.15058 |          svm | BoxConstraint:                 0.088843 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     60.035 |
|   37 |       8 | Accept |    0.15528 |            150.14 |         0.14517 |         0.15062 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  244 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                          3 |
|   38 |       8 | Accept |    0.15115 |            5.0809 |         0.14517 |         0.15062 |         tree | MinLeafSize:                        262 |
|   39 |       8 | Accept |    0.14982 |            154.77 |         0.14517 |         0.15003 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  218 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         19 |
|   40 |       8 | Accept |    0.15044 |            145.66 |         0.14517 |          0.1501 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  201 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                          1 |
|=======================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Estimated min   | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | validation loss |              |                                         |
|=======================================================================================================================================================|
|   41 |       8 | Accept |    0.24677 |           0.61497 |         0.14517 |          0.1501 |         tree | MinLeafSize:                      12837 |
|   42 |       8 | Accept |    0.17342 |            8.1501 |         0.14517 |          0.1501 |         tree | MinLeafSize:                          3 |
|   43 |       8 | Accept |    0.15795 |            137.62 |         0.14517 |         0.15019 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  224 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       2353 |
|   44 |       8 | Accept |    0.17696 |            1.4365 |         0.14517 |         0.15019 |         tree | MinLeafSize:                       1170 |
|   45 |       8 | Accept |     0.1683 |            144.68 |         0.14517 |         0.15019 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           18.063 |
|   46 |       8 | Accept |     0.2011 |            125.96 |         0.14517 |         0.15019 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                            58214 |
|   47 |       8 | Accept |     0.1883 |            134.06 |         0.14517 |         0.15019 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           2612.2 |
|   48 |       8 | Accept |     0.1494 |            865.49 |         0.14517 |         0.15019 |          svm | BoxConstraint:                 0.028795 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     1.4357 |
|   49 |       8 | Accept |    0.17342 |            7.5968 |         0.14517 |         0.15019 |         tree | MinLeafSize:                          3 |
|   50 |       8 | Accept |    0.15016 |            3.9429 |         0.14517 |         0.15019 |         tree | MinLeafSize:                        171 |
|   51 |       8 | Accept |    0.16106 |            170.49 |         0.14517 |          0.1505 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  248 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                        358 |
|   52 |       8 | Accept |    0.17901 |            9.2773 |         0.14517 |          0.1505 |         tree | MinLeafSize:                          2 |
|   53 |       8 | Accept |    0.17487 |            146.78 |         0.14517 |          0.1505 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |                 |              | Width:                           65.783 |
|   54 |       8 | Accept |    0.15025 |            243.33 |         0.14517 |         0.15019 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  293 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         37 |
|   55 |       8 | Accept |    0.23138 |            406.36 |         0.14517 |         0.15019 |          svm | BoxConstraint:                 0.064193 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     38.054 |
|   56 |       8 | Accept |    0.15038 |            245.69 |         0.14517 |         0.15032 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  300 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         16 |
|   57 |       8 | Accept |    0.14853 |            224.07 |         0.14517 |         0.15057 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  259 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                        684 |
|   58 |       8 | Accept |    0.15075 |            210.06 |         0.14517 |         0.15016 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  296 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         33 |
|   59 |       8 | Accept |    0.14981 |            221.88 |         0.14517 |         0.15016 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  300 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         18 |
|   60 |       8 | Accept |    0.15111 |            192.41 |         0.14517 |         0.15004 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  250 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       1571 |
|=======================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Estimated min   | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | validation loss |              |                                         |
|=======================================================================================================================================================|
|   61 |       8 | Accept |    0.15018 |             219.2 |         0.14517 |         0.15006 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  297 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         29 |
|   62 |       8 | Accept |    0.14983 |            207.92 |         0.14517 |         0.14994 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  272 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                        553 |
|   63 |       8 | Accept |    0.15006 |            208.62 |         0.14517 |         0.15004 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  299 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                         13 |
|   64 |       8 | Accept |    0.15116 |            199.41 |         0.14517 |         0.14986 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  256 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       1574 |
|   65 |       8 | Accept |    0.14934 |            214.65 |         0.14517 |         0.14923 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  264 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       1469 |
|   66 |       8 | Accept |    0.16747 |            359.01 |         0.14517 |         0.14923 |          svm | BoxConstraint:                 0.033521 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     12.508 |
|   67 |       8 | Accept |    0.14883 |            219.75 |         0.14517 |         0.14919 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  254 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                        986 |
|   68 |       8 | Accept |    0.14959 |               207 |         0.14517 |         0.14912 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  259 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       1173 |
|   69 |       8 | Accept |    0.15016 |            207.07 |         0.14517 |         0.14918 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  290 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       1897 |
|   70 |       8 | Accept |    0.14754 |            275.92 |         0.14517 |         0.14918 |          svm | BoxConstraint:                  0.10603 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     5.0157 |
|   71 |       8 | Accept |    0.14794 |             282.2 |         0.14517 |         0.14918 |          svm | BoxConstraint:                  0.11374 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     5.4724 |
|   72 |       8 | Accept |    0.15093 |             190.3 |         0.14517 |         0.14914 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  267 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       1707 |
|   73 |       8 | Accept |    0.14908 |            300.63 |         0.14517 |         0.14914 |          svm | BoxConstraint:                 0.087041 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     6.0336 |
|   74 |       8 | Accept |    0.14875 |            246.04 |         0.14517 |         0.14913 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  299 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                        934 |
|   75 |       8 | Accept |    0.16303 |            345.34 |         0.14517 |         0.14913 |          svm | BoxConstraint:                  0.10529 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                      18.53 |
|   76 |       8 | Accept |    0.14828 |            274.59 |         0.14517 |         0.14913 |          svm | BoxConstraint:                 0.060389 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.7508 |
|   77 |       8 | Accept |    0.15908 |            326.16 |         0.14517 |         0.14913 |          svm | BoxConstraint:                     0.11 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     16.289 |
|   78 |       8 | Accept |    0.15976 |            297.48 |         0.14517 |         0.14852 |          svm | BoxConstraint:                  0.26147 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     25.792 |
|   79 |       8 | Accept |    0.14857 |             279.1 |         0.14517 |         0.14802 |          svm | BoxConstraint:                 0.048355 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.4186 |
|   80 |       8 | Accept |    0.14944 |            302.39 |         0.14517 |         0.14845 |          svm | BoxConstraint:                 0.041852 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.9836 |
|=======================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Estimated min   | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | validation loss |              |                                         |
|=======================================================================================================================================================|
|   81 |       8 | Accept |    0.19487 |            401.11 |         0.14517 |         0.14861 |          svm | BoxConstraint:                 0.078452 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     30.022 |
|   82 |       8 | Accept |     0.1461 |            315.95 |         0.14517 |         0.14876 |          svm | BoxConstraint:                  0.15302 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.7768 |
|   83 |       8 | Accept |    0.15583 |            333.62 |         0.14517 |         0.14716 |          svm | BoxConstraint:                0.0071375 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.1809 |
|   84 |       8 | Accept |    0.14761 |            297.08 |         0.14517 |          0.1478 |          svm | BoxConstraint:                 0.070313 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.4613 |
|   85 |       8 | Accept |    0.14531 |            326.83 |         0.14517 |         0.14706 |          svm | BoxConstraint:                  0.73357 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     5.3351 |
|   86 |       8 | Accept |    0.14691 |            303.06 |         0.14517 |         0.14643 |          svm | BoxConstraint:                 0.089237 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                      4.479 |
|   87 |       8 | Accept |    0.15009 |            313.15 |         0.14517 |         0.14635 |          svm | BoxConstraint:                 0.028044 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.6944 |
|   88 |       8 | Accept |     0.1578 |            212.73 |         0.14517 |         0.14635 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  283 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       2103 |
|   89 |       8 | Accept |    0.14521 |            456.62 |         0.14517 |         0.14617 |          svm | BoxConstraint:                   1.9049 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.7834 |
|   90 |       8 | Accept |    0.15028 |            224.72 |         0.14517 |         0.14617 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |                 |              | NumLearningCycles:                  271 |
|      |         |        |            |                   |                 |                 |              | MinLeafSize:                       1953 |
|   91 |       8 | Accept |    0.15213 |             316.5 |         0.14517 |         0.14614 |          svm | BoxConstraint:                 0.016203 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.3045 |
|   92 |       8 | Accept |    0.14533 |            405.52 |         0.14517 |         0.14618 |          svm | BoxConstraint:                   2.5334 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     5.0734 |
|   93 |       8 | Accept |     0.4955 |              5808 |         0.14517 |         0.14629 |          svm | BoxConstraint:                0.0055052 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.11946 |
|   94 |       8 | Accept |    0.14599 |             421.4 |         0.14517 |         0.14552 |          svm | BoxConstraint:                   0.1423 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     2.5078 |
|   95 |       8 | Accept |    0.15696 |            334.19 |         0.14517 |          0.1459 |          svm | BoxConstraint:                0.0086085 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.7011 |
|   96 |       8 | Accept |    0.16125 |            335.61 |         0.14517 |         0.14572 |          svm | BoxConstraint:                0.0043914 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.3671 |
|   97 |       8 | Accept |    0.24245 |            6787.2 |         0.14517 |         0.14594 |          svm | BoxConstraint:                   11.252 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.48339 |
|   98 |       8 | Accept |    0.16294 |            354.47 |         0.14517 |         0.14602 |          svm | BoxConstraint:                0.0062158 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     5.3367 |
|   99 |       8 | Accept |    0.16271 |            339.55 |         0.14517 |         0.14604 |          svm | BoxConstraint:                0.0046539 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.7345 |
|  100 |       8 | Accept |    0.17184 |            5978.5 |         0.14517 |         0.14589 |          svm | BoxConstraint:                   7.6879 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     1.6339 |
|=======================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Estimated min   | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | validation loss |              |                                         |
|=======================================================================================================================================================|
|  101 |       8 | Accept |    0.16142 |            355.72 |         0.14517 |         0.14624 |          svm | BoxConstraint:                0.0057866 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.8082 |
|  102 |       8 | Accept |    0.15067 |            1030.9 |         0.14517 |         0.14639 |          svm | BoxConstraint:                0.0050117 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     1.0093 |
|  103 |       8 | Best   |    0.14513 |            501.17 |         0.14513 |          0.1457 |          svm | BoxConstraint:                   3.1097 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                      5.013 |
|  104 |       8 | Accept |    0.35434 |            7976.9 |         0.14513 |         0.14584 |          svm | BoxConstraint:                  0.06472 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                  0.0099007 |
|  105 |       8 | Accept |    0.26591 |            8022.4 |         0.14513 |         0.14609 |          svm | BoxConstraint:                   3.0956 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                   0.013802 |
|  106 |       8 | Accept |    0.35499 |            5418.2 |         0.14513 |         0.14564 |          svm | BoxConstraint:                  0.37919 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.14009 |
|  107 |       8 | Accept |      0.167 |            6198.1 |         0.14513 |         0.14563 |          svm | BoxConstraint:                   659.16 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.3029 |
|  108 |       8 | Accept |    0.14539 |            747.98 |         0.14513 |         0.14574 |          svm | BoxConstraint:                   5.9153 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.9842 |
|  109 |       8 | Accept |    0.14543 |            586.07 |         0.14513 |         0.14532 |          svm | BoxConstraint:                   3.9618 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.9501 |
|  110 |       8 | Accept |    0.17056 |            5739.7 |         0.14513 |          0.1457 |          svm | BoxConstraint:                0.0024422 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.46361 |
|  111 |       8 | Accept |    0.29426 |            4906.9 |         0.14513 |         0.14557 |          svm | BoxConstraint:                0.0064138 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.31177 |
|  112 |       8 | Accept |     0.1717 |            5907.9 |         0.14513 |         0.14571 |          svm | BoxConstraint:                  0.10504 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.82207 |
|  113 |       8 | Accept |    0.17241 |            5909.3 |         0.14513 |         0.14549 |          svm | BoxConstraint:                 0.028608 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.64777 |
|  114 |       8 | Accept |    0.17806 |            6444.5 |         0.14513 |         0.14558 |          svm | BoxConstraint:                  0.34511 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.72366 |
|  115 |       8 | Accept |    0.16555 |              5481 |         0.14513 |         0.14533 |          svm | BoxConstraint:                  0.17641 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     1.0964 |
|  116 |       8 | Accept |    0.14788 |            1249.1 |         0.14513 |         0.14565 |          svm | BoxConstraint:                   11.881 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.8553 |
|  117 |       8 | Accept |    0.24664 |            5351.2 |         0.14513 |         0.14631 |          svm | BoxConstraint:                0.0048502 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.38595 |
|  118 |       8 | Best   |    0.14483 |            563.53 |         0.14483 |          0.1454 |          svm | BoxConstraint:                   2.2083 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     4.5125 |
|  119 |       8 | Accept |    0.15403 |            340.78 |         0.14483 |         0.14559 |          svm | BoxConstraint:                    0.843 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                     26.671 |
|  120 |       8 | Accept |    0.27575 |            4752.3 |         0.14483 |         0.14532 |          svm | BoxConstraint:                0.0017988 |
|      |         |        |            |                   |                 |                 |              | KernelScale:                    0.21724 |

__________________________________________________________
Optimization completed.
Total iterations: 120
Total elapsed time: 17271.6899 seconds
Total time for training and validation: 116613.8436 seconds

Best observed learner is an svm model with:
	Learner:                   svm
	BoxConstraint:          2.2083
	KernelScale:            4.5125
Observed validation loss: 0.14483
Time for training and validation: 563.5264 seconds

Best estimated learner (returned model) is an svm model with:
	Learner:                   svm
	BoxConstraint:          3.1097
	KernelScale:             5.013
Estimated validation loss: 0.14532
Estimated time for training and validation: 574.6554 seconds

Documentation for fitcauto display

Total elapsed time の値から、ベイズ最適化の実行に時間がかかったことがわかります (約 4.8 時間)。

fitcauto によって返される最終的なモデルが、最適な推定学習器となります。モデルを返す前に、関数は学習データ セット全体 (adultdata)、リストされている Learner (またはモデル) のタイプ、および表示されたハイパーパラメーター値を使用して、モデルの再学習を行います。

ASHA 最適化による自動モデル選択の使用

学習セットの観測値の数が原因でベイズ最適化による fitcauto の実行に長い時間がかかる場合は、代わりに ASHA 最適化による fitcauto を使用することを検討してください。adultdata に含まれる観測値が 10,000 を超える場合は、ASHA 最適化による fitcauto を使用して適切な分類器を自動的に見つけるよう試します。ASHA 最適化による fitcauto を使用すると、関数はさまざまなハイパーパラメーターの値をもつ複数のモデルを無作為に選択し、学習データの小さいサブセットで学習させます。交差検証の分類誤差 (Validation Loss) に基づいて有望なモデルが見つかると、そのモデルをプロモートし、より多くの学習データで学習させます。このプロセスを繰り返し、データの量を徐々に増やしながら有望なモデルに学習させます。既定の設定では、fitcauto は、最適化のプロット、および最適化の結果の反復表示を提供します。これらの結果を解釈する方法の詳細については、Verbose の表示を参照してください。

観測値の重みを設定し、ASHA 最適化を並列実行するよう指定します。ASHA 最適化は既定のベイズ最適化に比べて反復回数が多くなる場合が多いことに注意してください。時間の制約がある場合は、HyperparameterOptimizationOptions 構造体の MaxTime フィールドを指定して、fitcauto を実行する秒数を制限できます。

ashaOptions = struct("Optimizer","asha","UseParallel",true);
[ashaMdl,ashaResults] = fitcauto(adultdata,"salary","Weights","fnlwgt", ...
   "HyperparameterOptimizationOptions",ashaOptions);
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Copying objective function to workers...
Done copying objective function to workers.
Learner types to explore: ensemble, nb, svm, tree
Total iterations (MaxObjectiveEvaluations): 425
Total time (MaxTime): Inf

|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|    1 |       8 | Best   |     0.1831 |           0.59595 |          0.1831 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|    2 |       7 | Accept |    0.24677 |           0.64255 |          0.1831 |          378 |         tree | MinLeafSize:                       2865 |
|    3 |       7 | Accept |    0.24677 |           0.88001 |          0.1831 |          378 |         tree | MinLeafSize:                       1629 |
|    4 |       7 | Accept |    0.24677 |           0.49804 |          0.1831 |          378 |          svm | BoxConstraint:                   93.722 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0031746 |
|    5 |       8 | Best   |    0.15872 |            2.6522 |         0.15872 |          378 |          svm | BoxConstraint:                   622.14 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     8.3947 |
|    6 |       8 | Accept |    0.17662 |           0.88532 |         0.15872 |         1509 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|    7 |       8 | Accept |    0.17844 |            0.7806 |         0.15872 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|    8 |       7 | Accept |    0.24677 |            1.6871 |         0.15872 |          378 |          svm | BoxConstraint:                   16.149 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     418.84 |
|    9 |       7 | Accept |    0.18401 |           0.70381 |         0.15872 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   10 |       7 | Accept |    0.17244 |           0.31131 |         0.15872 |          378 |         tree | MinLeafSize:                          4 |
|   11 |       8 | Accept |    0.17135 |           0.45227 |         0.15872 |          378 |         tree | MinLeafSize:                         13 |
|   12 |       8 | Accept |    0.24677 |           0.53034 |         0.15872 |          378 |          svm | BoxConstraint:                    272.5 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0013306 |
|   13 |       8 | Accept |    0.24677 |           0.34333 |         0.15872 |          378 |         tree | MinLeafSize:                        701 |
|   14 |       8 | Best   |    0.15344 |            1.0465 |         0.15344 |         1509 |         tree | MinLeafSize:                         13 |
|   15 |       8 | Accept |     0.1538 |            12.371 |         0.15344 |         1509 |          svm | BoxConstraint:                   622.14 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     8.3947 |
|   16 |       8 | Accept |    0.24677 |           0.31034 |         0.15344 |          378 |         tree | MinLeafSize:                       1773 |
|   17 |       8 | Accept |      0.186 |             21.17 |         0.15344 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           9030.3 |
|   18 |       8 | Accept |    0.17043 |            21.308 |         0.15344 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           512.37 |
|   19 |       8 | Accept |    0.18322 |            20.058 |         0.15344 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                            30.45 |
|   20 |       8 | Accept |     0.1802 |            1.1355 |         0.15344 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|   21 |       8 | Accept |    0.18721 |            23.236 |         0.15344 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           9222.9 |
|   22 |       8 | Accept |    0.20066 |            21.173 |         0.15344 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           129.74 |
|   23 |       8 | Accept |     0.2205 |            17.802 |         0.15344 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                            30996 |
|   24 |       8 | Accept |      0.155 |           0.81516 |         0.15344 |         1509 |         tree | MinLeafSize:                          4 |
|   25 |       8 | Best   |    0.14361 |            1.0991 |         0.14361 |         6033 |         tree | MinLeafSize:                         13 |
|   26 |       8 | Accept |    0.19409 |            33.202 |         0.14361 |          378 |          svm | BoxConstraint:                   259.24 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.32997 |
|   27 |       8 | Accept |    0.24677 |            2.6882 |         0.14361 |          378 |          svm | BoxConstraint:                0.0059822 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     713.08 |
|   28 |       8 | Accept |    0.16587 |            1.8275 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   29 |       8 | Accept |    0.18015 |            16.315 |         0.14361 |          378 |          svm | BoxConstraint:                   0.5769 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.36299 |
|   30 |       8 | Accept |    0.17001 |            1.1164 |         0.14361 |         1509 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   31 |       8 | Accept |    0.18193 |           0.55339 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   32 |       8 | Accept |    0.16862 |            24.418 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                            102.7 |
|   33 |       7 | Accept |      0.241 |            2.5999 |         0.14361 |          378 |          svm | BoxConstraint:                   0.6369 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     23.843 |
|   34 |       7 | Accept |     0.1821 |            1.0709 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   35 |       8 | Accept |    0.18464 |            29.794 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           9256.9 |
|   36 |       8 | Accept |    0.24677 |            34.988 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  289 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                        929 |
|   37 |       8 | Accept |    0.16073 |            50.947 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  283 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          5 |
|   38 |       8 | Accept |    0.17086 |            52.897 |         0.14361 |         1509 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           512.37 |
|   39 |       8 | Accept |    0.17374 |           0.96688 |         0.14361 |          378 |         tree | MinLeafSize:                          7 |
|   40 |       8 | Accept |    0.16598 |            37.543 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  281 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|   41 |       8 | Accept |      0.242 |            1.7295 |         0.14361 |          378 |          svm | BoxConstraint:                0.0093388 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     4.7672 |
|   42 |       8 | Accept |    0.20568 |            32.829 |         0.14361 |          378 |          svm | BoxConstraint:                 0.091315 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.30293 |
|   43 |       8 | Accept |    0.16465 |            56.109 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  265 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|   44 |       8 | Accept |    0.16598 |             2.096 |         0.14361 |          378 |          svm | BoxConstraint:                   49.164 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     1.7376 |
|   45 |       8 | Accept |    0.18456 |            52.012 |         0.14361 |         1509 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                            102.7 |
|   46 |       8 | Accept |    0.24677 |           0.77083 |         0.14361 |          378 |         tree | MinLeafSize:                      13706 |
|   47 |       8 | Accept |    0.16477 |            25.694 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  218 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         36 |
|   48 |       8 | Accept |    0.17952 |             1.107 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   49 |       8 | Accept |    0.32627 |            84.486 |         0.14361 |          378 |          svm | BoxConstraint:                    549.6 |
|      |         |        |            |                   |                 |              |              | KernelScale:                   0.036915 |
|   50 |       8 | Accept |    0.16346 |            14.925 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                          0.65733 |
|   51 |       8 | Accept |    0.24677 |            4.6465 |         0.14361 |          378 |          svm | BoxConstraint:                   0.9246 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     398.37 |
|   52 |       8 | Accept |    0.15285 |            42.395 |         0.14361 |         1509 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  283 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          5 |
|   53 |       8 | Accept |    0.24677 |            2.1044 |         0.14361 |          378 |          svm | BoxConstraint:                 0.041664 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     29.281 |
|   54 |       8 | Accept |    0.24677 |            39.272 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  216 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                       1096 |
|   55 |       8 | Accept |    0.15401 |            48.294 |         0.14361 |         1509 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  218 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         36 |
|   56 |       8 | Accept |    0.14704 |            59.661 |         0.14361 |         1509 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  265 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|   57 |       8 | Accept |    0.15196 |            41.835 |         0.14361 |         1509 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                          0.65733 |
|   58 |       8 | Accept |    0.24285 |            97.209 |         0.14361 |          378 |          svm | BoxConstraint:                   62.688 |
|      |         |        |            |                   |                 |              |              | KernelScale:                   0.019392 |
|   59 |       8 | Accept |    0.24677 |           0.75744 |         0.14361 |          378 |         tree | MinLeafSize:                        159 |
|   60 |       8 | Accept |     0.1928 |             24.43 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           24.876 |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|   61 |       8 | Accept |    0.20489 |            21.827 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           6524.9 |
|   62 |       8 | Accept |    0.24677 |            0.5275 |         0.14361 |          378 |          svm | BoxConstraint:                0.0012686 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0019115 |
|   63 |       8 | Accept |    0.24677 |           0.43222 |         0.14361 |          378 |          svm | BoxConstraint:                   68.024 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0023633 |
|   64 |       8 | Accept |    0.26755 |            94.048 |         0.14361 |          378 |          svm | BoxConstraint:                   3.4696 |
|      |         |        |            |                   |                 |              |              | KernelScale:                   0.026518 |
|   65 |       8 | Accept |    0.25611 |            97.151 |         0.14361 |          378 |          svm | BoxConstraint:                   14.395 |
|      |         |        |            |                   |                 |              |              | KernelScale:                   0.018438 |
|   66 |       8 | Accept |    0.24677 |            0.6123 |         0.14361 |          378 |          svm | BoxConstraint:                   6.1487 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0012672 |
|   67 |       8 | Accept |    0.23521 |            38.994 |         0.14361 |          378 |          svm | BoxConstraint:                  0.47847 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.21848 |
|   68 |       8 | Accept |    0.14949 |            75.744 |         0.14361 |         6033 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  283 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          5 |
|   69 |       8 | Accept |     0.1689 |            26.118 |         0.14361 |         1509 |          svm | BoxConstraint:                   49.164 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     1.7376 |
|   70 |       8 | Accept |    0.17936 |              25.8 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  228 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                        164 |
|   71 |       8 | Accept |     0.2466 |            34.448 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  239 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         88 |
|   72 |       8 | Accept |    0.15654 |            2.6753 |         0.14361 |         1509 |         tree | MinLeafSize:                          7 |
|   73 |       8 | Accept |    0.18175 |           0.87169 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   74 |       8 | Accept |    0.16188 |            42.445 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  232 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         14 |
|   75 |       8 | Accept |    0.24677 |            2.9432 |         0.14361 |          378 |          svm | BoxConstraint:                   5.2509 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     566.41 |
|   76 |       8 | Accept |    0.24677 |           0.68866 |         0.14361 |          378 |          svm | BoxConstraint:                   121.38 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0038679 |
|   77 |       8 | Accept |    0.24677 |            1.8912 |         0.14361 |          378 |          svm | BoxConstraint:                   4.2627 |
|      |         |        |            |                   |                 |              |              | KernelScale:                      139.4 |
|   78 |       8 | Accept |    0.21364 |            25.979 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                            69985 |
|   79 |       8 | Accept |    0.16991 |            22.652 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           12.131 |
|   80 |       8 | Accept |    0.24677 |            27.661 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  200 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                        285 |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|   81 |       8 | Accept |    0.16426 |            2.4413 |         0.14361 |          378 |          svm | BoxConstraint:                  0.10011 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     3.7337 |
|   82 |       8 | Accept |    0.17779 |           0.78782 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   83 |       8 | Accept |    0.15315 |            42.755 |         0.14361 |         1509 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  281 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|   84 |       7 | Accept |    0.17499 |            1.8123 |         0.14361 |          378 |          svm | BoxConstraint:                   235.48 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     1.5618 |
|   85 |       7 | Accept |     0.1702 |            1.7494 |         0.14361 |          378 |          svm | BoxConstraint:                   1.7191 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     1.7936 |
|   86 |       8 | Accept |    0.17428 |           0.71717 |         0.14361 |          378 |         tree | MinLeafSize:                          1 |
|   87 |       8 | Accept |    0.24677 |           0.35012 |         0.14361 |          378 |         tree | MinLeafSize:                      10340 |
|   88 |       8 | Accept |    0.15506 |            4.6537 |         0.14361 |         1509 |          svm | BoxConstraint:                  0.10011 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     3.7337 |
|   89 |       8 | Accept |    0.17907 |           0.93664 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   90 |       8 | Accept |    0.15925 |            7.5152 |         0.14361 |         1509 |          svm | BoxConstraint:                   1.7191 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     1.7936 |
|   91 |       8 | Accept |    0.16344 |            30.309 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  213 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         25 |
|   92 |       8 | Accept |    0.18234 |           0.92613 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|   93 |       8 | Accept |    0.24677 |            1.8464 |         0.14361 |          378 |          svm | BoxConstraint:                 0.019224 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     574.94 |
|   94 |       8 | Accept |     0.1566 |            31.765 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  209 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|   95 |       8 | Accept |    0.15317 |            50.355 |         0.14361 |         1509 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  232 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         14 |
|   96 |       8 | Accept |    0.24677 |           0.43111 |         0.14361 |          378 |         tree | MinLeafSize:                         95 |
|   97 |       8 | Accept |    0.16512 |            35.087 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  239 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         37 |
|   98 |       8 | Accept |    0.17052 |           0.45685 |         0.14361 |          378 |         tree | MinLeafSize:                          2 |
|   99 |       8 | Accept |    0.17347 |            1.1812 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  100 |       8 | Accept |    0.16896 |            54.887 |         0.14361 |         1509 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           12.131 |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|  101 |       8 | Accept |    0.18156 |           0.49723 |         0.14361 |          378 |         tree | MinLeafSize:                         59 |
|  102 |       8 | Accept |    0.24677 |           0.68678 |         0.14361 |          378 |          svm | BoxConstraint:                0.0010962 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0023771 |
|  103 |       8 | Accept |    0.24677 |            33.615 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  262 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                       1898 |
|  104 |       8 | Accept |    0.15192 |            81.549 |         0.14361 |         6033 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  265 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|  105 |       8 | Accept |    0.15412 |            32.695 |         0.14361 |         1509 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  209 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|  106 |       8 | Accept |    0.24677 |            2.1129 |         0.14361 |          378 |          svm | BoxConstraint:                  0.51162 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     901.32 |
|  107 |       8 | Accept |    0.24712 |            96.705 |         0.14361 |          378 |          svm | BoxConstraint:                  0.12031 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.03735 |
|  108 |       8 | Accept |    0.24677 |            0.5869 |         0.14361 |          378 |          svm | BoxConstraint:                   878.36 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0032037 |
|  109 |       8 | Accept |    0.15335 |            46.629 |         0.14361 |         1509 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  213 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         25 |
|  110 |       8 | Accept |    0.24677 |            42.807 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  229 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                        133 |
|  111 |       8 | Accept |    0.18895 |            1.9037 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  112 |       8 | Accept |    0.16369 |             2.104 |         0.14361 |         1509 |         tree | MinLeafSize:                          2 |
|  113 |       8 | Accept |    0.16353 |            1.0029 |         0.14361 |          378 |         tree | MinLeafSize:                          3 |
|  114 |       8 | Accept |    0.15127 |            93.189 |         0.14361 |         6033 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                          0.65733 |
|  115 |       8 | Accept |    0.16092 |            25.041 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                          0.59189 |
|  116 |       8 | Accept |    0.24677 |           0.91976 |         0.14361 |          378 |         tree | MinLeafSize:                        123 |
|  117 |       8 | Accept |    0.16083 |            55.658 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  241 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          4 |
|  118 |       8 | Accept |    0.17643 |           0.91472 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  119 |       8 | Accept |    0.15053 |            9.4088 |         0.14361 |        24130 |         tree | MinLeafSize:                         13 |
|  120 |       8 | Accept |    0.18671 |            1.1927 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|  121 |       8 | Accept |    0.15504 |            52.066 |         0.14361 |         1509 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  239 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         37 |
|  122 |       7 | Accept |     0.1873 |           0.88051 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  123 |       7 | Accept |    0.18138 |            1.4255 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  124 |       8 | Accept |     0.1836 |            1.6767 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  125 |       8 | Accept |    0.24677 |           0.53547 |         0.14361 |          378 |          svm | BoxConstraint:                 0.015196 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0050979 |
|  126 |       8 | Accept |    0.16782 |            76.114 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  270 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         21 |
|  127 |       8 | Accept |    0.65702 |             109.5 |         0.14361 |          378 |          svm | BoxConstraint:                   11.102 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.02167 |
|  128 |       8 | Accept |    0.15785 |           0.97717 |         0.14361 |         1509 |         tree | MinLeafSize:                          3 |
|  129 |       8 | Accept |    0.16396 |            43.391 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  206 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         52 |
|  130 |       8 | Accept |    0.20726 |            29.078 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                            71433 |
|  131 |       8 | Accept |    0.15259 |            46.724 |         0.14361 |         1509 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                          0.59189 |
|  132 |       8 | Accept |    0.19003 |            42.596 |         0.14361 |          378 |          svm | BoxConstraint:                   6.2931 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.33148 |
|  133 |       8 | Accept |    0.16091 |             28.84 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           1.1628 |
|  134 |       8 | Accept |    0.17784 |            29.872 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           886.95 |
|  135 |       8 | Accept |    0.15027 |            76.937 |         0.14361 |         1509 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  241 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          4 |
|  136 |       8 | Accept |    0.18327 |            29.845 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           4528.8 |
|  137 |       8 | Accept |    0.24677 |           0.38473 |         0.14361 |          378 |         tree | MinLeafSize:                      10969 |
|  138 |       8 | Accept |    0.15068 |             106.4 |         0.14361 |         6033 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  281 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          2 |
|  139 |       8 | Accept |    0.24677 |            27.445 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  223 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         98 |
|  140 |       8 | Accept |    0.24341 |            100.59 |         0.14361 |          378 |          svm | BoxConstraint:                   2.7209 |
|      |         |        |            |                   |                 |              |              | KernelScale:                   0.010215 |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|  141 |       8 | Accept |    0.24187 |            2.6075 |         0.14361 |          378 |          svm | BoxConstraint:                0.0079129 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     4.8667 |
|  142 |       8 | Accept |    0.17089 |           0.53327 |         0.14361 |          378 |         tree | MinLeafSize:                          3 |
|  143 |       8 | Accept |    0.15068 |            41.021 |         0.14361 |         1509 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           1.1628 |
|  144 |       8 | Accept |    0.24677 |            26.637 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  258 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                       6413 |
|  145 |       8 | Accept |    0.24677 |            26.876 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  283 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                       3673 |
|  146 |       8 | Accept |    0.16076 |            3.2005 |         0.14361 |          378 |          svm | BoxConstraint:                   4.4667 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     13.015 |
|  147 |       8 | Accept |     0.1567 |            2.1777 |         0.14361 |          378 |          svm | BoxConstraint:                  0.35692 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     4.2583 |
|  148 |       8 | Accept |    0.18396 |           0.90132 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  149 |       8 | Accept |    0.15063 |            6.9084 |         0.14361 |         1509 |          svm | BoxConstraint:                  0.35692 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     4.2583 |
|  150 |       8 | Accept |    0.17674 |            1.2601 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  151 |       8 | Accept |      0.156 |            35.953 |         0.14361 |         1509 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  206 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         52 |
|  152 |       8 | Accept |    0.16561 |             27.77 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           905.18 |
|  153 |       8 | Accept |    0.24677 |            34.957 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  295 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                       4587 |
|  154 |       8 | Accept |    0.17262 |            25.102 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           7.0619 |
|  155 |       8 | Accept |     0.1518 |            94.985 |         0.14361 |         6033 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                          0.59189 |
|  156 |       8 | Accept |    0.15191 |            4.8239 |         0.14361 |         1509 |          svm | BoxConstraint:                   4.4667 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     13.015 |
|  157 |       8 | Accept |    0.70666 |             88.91 |         0.14361 |          378 |          svm | BoxConstraint:                   144.03 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0087964 |
|  158 |       8 | Accept |    0.20845 |            1.2899 |         0.14361 |          378 |          svm | BoxConstraint:                   14.266 |
|      |         |        |            |                   |                 |              |              | KernelScale:                    0.56134 |
|  159 |       8 | Accept |    0.16207 |            2.2155 |         0.14361 |          378 |          svm | BoxConstraint:                 0.018808 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     1.4491 |
|  160 |       8 | Accept |    0.17735 |           0.30364 |         0.14361 |          378 |         tree | MinLeafSize:                         16 |
|====================================================================================================================================================|
| Iter | Active  | Eval   | Validation | Time for training | Observed min    | Training set | Learner      | Hyperparameter:                 Value   |
|      | workers | result | loss       | & validation (sec)| validation loss | size         |              |                                         |
|====================================================================================================================================================|
|  161 |       8 | Accept |    0.14814 |            3.7088 |         0.14361 |         1509 |          svm | BoxConstraint:                 0.018808 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     1.4491 |
|  162 |       8 | Accept |    0.15674 |            54.506 |         0.14361 |         1509 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  270 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                         21 |
|  163 |       8 | Accept |    0.18367 |            22.255 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           79.973 |
|  164 |       8 | Accept |     0.1802 |           0.81271 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  165 |       8 | Accept |    0.27102 |            87.837 |         0.14361 |          378 |          svm | BoxConstraint:                   2.3737 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0096356 |
|  166 |       8 | Accept |    0.24677 |            15.367 |         0.14361 |          378 |     ensemble | Method:                      LogitBoost |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  211 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                       8387 |
|  167 |       8 | Accept |    0.16134 |            17.629 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                           1.1013 |
|  168 |       8 | Accept |    0.24677 |            1.5152 |         0.14361 |          378 |          svm | BoxConstraint:                0.0011611 |
|      |         |        |            |                   |                 |              |              | KernelScale:                     20.262 |
|  169 |       8 | Accept |     0.1889 |           0.58768 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  170 |       8 | Accept |    0.20571 |            21.075 |         0.14361 |          378 |           nb | DistributionNames:               kernel |
|      |         |        |            |                   |                 |              |              | Width:                            51434 |
|  171 |       8 | Accept |    0.17171 |           0.54565 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              NaN |
|  172 |       8 | Accept |    0.16136 |            43.429 |         0.14361 |          378 |     ensemble | Method:                             Bag |
|      |         |        |            |                   |                 |              |              | NumLearningCycles:                  264 |
|      |         |        |            |                   |                 |              |              | MinLeafSize:                          1 |
|  173 |       8 | Accept |    0.24677 |           0.74563 |         0.14361 |          378 |          svm | BoxConstraint:                   82.364 |
|      |         |        |            |                   |                 |              |              | KernelScale:                  0.0048723 |
|  174 |       8 | Accept |    0.17774 |           0.82021 |         0.14361 |          378 |           nb | DistributionNames:               normal |
|      |         |        |            |                   |                 |              |              | Width:                              ...

__________________________________________________________
Optimization completed.
Total iterations: 425
Total elapsed time: 1225.1049 seconds
Total time for training and validation: 8476.6632 seconds

Best observed learner is a tree model with:
	Learner:                  tree
	MinLeafSize:                14
Observed validation loss: 0.14138
Time for training and validation: 1.6545 seconds

Documentation for fitcauto display

Total elapsed time の値から、ASHA 最適化の方がベイズ最適化よりも実行時間が短くなったことがわかります (約 0.3 時間)。

fitcauto によって返される最終的なモデルが、観測された最適な学習器となります。モデルを返す前に、関数は学習データ セット全体 (adultdata)、リストされている Learner (またはモデル) のタイプ、および表示されたハイパーパラメーター値を使用して、モデルの再学習を行います。

検定セットのパフォーマンスの評価

混同行列と受信者動作特性 (ROC) 曲線を使用して、返されたモデル bayesianMdlashaMdl の性能を検定セット adulttest で評価します。

各モデルについて、検定セットの予測されるラベルとスコア値を見つけます。

[bayesianLabels,bayesianScores] = predict(bayesianMdl,adulttest);
[ashaLabels,ashaScores] = predict(ashaMdl,adulttest);

検定セットの結果から、混同行列を作成します。対角要素は、特定のクラスの正しく分類されたインスタンスの数を示しています。非対角要素は誤分類した観測値のインスタンスです。1 行 2 列のタイル レイアウトを使用して結果を比較します。

tiledlayout(1,2)

nexttile
confusionchart(adulttest.salary,bayesianLabels)
title("Bayesian Optimization")

nexttile
confusionchart(adulttest.salary,ashaLabels)
title("ASHA Optimization")

各モデルについて、検定セットの分類精度を計算します。精度は、検定セットの正しく分類された観測値の割合です。

bayesianAccuracy = (1-loss(bayesianMdl,adulttest,"salary"))*100
bayesianAccuracy = 85.2062
ashaAccuracy = (1-loss(ashaMdl,adulttest,"salary"))*100
ashaAccuracy = 84.1612

混同行列と精度の値から、検定セットでの性能は bayesianMdl の方が ashaMdl よりもわずかに優れていることがわかります。ただし、どちらのモデルも十分な性能を示しています。

それぞれのモデルについて、ROC 曲線をプロットし、ROC 曲線の下の領域 (AUC) を計算します。ROC 曲線は、分類スコアのさまざまなしきい値についての真陽性率と偽陽性率の関係を示します。しきい値にかかわらず真陽性率が常に 1 の完璧な分類器では、AUC = 1 になります。観測値をランダムにクラスに割り当てるバイナリ分類器では、AUC = 0.5 になります。大きな AUC 値 (1 に近い) は、分類器の性能が高いことを示します。

それぞれのモデルについて、rocmetrics オブジェクトを作成し、ROC 曲線のメトリクスを計算して AUC の値を求めます。

bayesianROC = rocmetrics(adulttest.salary,bayesianScores,bayesianMdl.ClassNames);
ashaROC = rocmetrics(adulttest.salary,ashaScores,ashaMdl.ClassNames);

rocmetrics の関数 plot を使用して、ラベル <=50K の ROC 曲線をプロットします。

figure
[r1,g1] = plot(bayesianROC,"ClassNames","<=50K");
hold on
[r2,g2] = plot(ashaROC,"ClassNames","<=50K");
r1.DisplayName = replace(r1.DisplayName,"<=50K","Bayesian Optimization");
r2.DisplayName = replace(r2.DisplayName,"<=50K","ASHA Optimization");
g1(1).DisplayName = "Bayesian Optimization Model Operating Point";
g2(1).DisplayName = "ASHA Optimization Model Operating Point";
title("ROC Curves for Class <=50K")
hold off

AUC 値を基準にすると、どちらの分類器も検定データに対して適切に機能しています。

参考

| | |

関連するトピック