ベイズ最適化および ASHA 最適化による分類器の自動選択
この例では、fitcauto を使用し、指定した学習予測子と応答データに基づいてさまざまなハイパーパラメーターの値をもつ分類モデルのタイプの選択を自動的に試す方法を示します。既定では、この関数はモデルの選択と評価にベイズ最適化を使用します。学習データ セットに多数の観測値が含まれている場合は、代わりに非同期連続半減アルゴリズム (ASHA) を使用できます。最適化が完了すると、fitcauto は、データ セット全体で学習済みの、新しいデータの分類に最適であると予測したモデルを返します。テスト データに対するモデルの性能をチェックします。
標本データの読み込み
この例では census1994.mat に保存されている 1994 年の国勢調査データを使用します。このデータ セットは、個人の年収が $50,000 を超えるかどうかを予測するために使用できる、米国国勢調査局の人口統計情報から構成されています。
学習データ adultdata およびテスト データ adulttest を含む、標本データ census1994 を読み込みます。学習データセットの最初の数行をプレビューします。
load census1994
head(adultdata)ans=8×15 table
age workClass fnlwgt education education_num marital_status occupation relationship race sex capital_gain capital_loss hours_per_week native_country salary
___ ________________ __________ _________ _____________ _____________________ _________________ _____________ _____ ______ ____________ ____________ ______________ ______________ ______
39 State-gov 77516 Bachelors 13 Never-married Adm-clerical Not-in-family White Male 2174 0 40 United-States <=50K
50 Self-emp-not-inc 83311 Bachelors 13 Married-civ-spouse Exec-managerial Husband White Male 0 0 13 United-States <=50K
38 Private 2.1565e+05 HS-grad 9 Divorced Handlers-cleaners Not-in-family White Male 0 0 40 United-States <=50K
53 Private 2.3472e+05 11th 7 Married-civ-spouse Handlers-cleaners Husband Black Male 0 0 40 United-States <=50K
28 Private 3.3841e+05 Bachelors 13 Married-civ-spouse Prof-specialty Wife Black Female 0 0 40 Cuba <=50K
37 Private 2.8458e+05 Masters 14 Married-civ-spouse Exec-managerial Wife White Female 0 0 40 United-States <=50K
49 Private 1.6019e+05 9th 5 Married-spouse-absent Other-service Not-in-family Black Female 0 0 16 Jamaica <=50K
52 Self-emp-not-inc 2.0964e+05 HS-grad 9 Married-civ-spouse Exec-managerial Husband White Male 0 0 45 United-States >50K
各行には、成人 1 人の人口統計情報が格納されています。最後の列 salary は個人の年収が $50,000 以下か、$50,000 を超えるかどうかを示します。
adultdata と adulttest から欠損値を含む観測値を削除します。
adultdata = rmmissing(adultdata); adulttest = rmmissing(adulttest);
ベイズ最適化による自動モデル選択の使用
fitcauto を使用して、adultdata のデータに適切な分類器を見つけます。既定では、fitcauto はベイズ最適化を使用してモデルとそのハイパーパラメーターの値を選択し、各モデルの交差検証の分類誤差 (Validation loss) を計算します。既定の設定では、fitcauto は、最適化のプロット、および最適化の結果の反復表示を提供します。これらの結果を解釈する方法の詳細については、Verbose の表示を参照してください。
観測値の重みを設定し、ベイズ最適化を並列実行するよう指定します。これには Parallel Computing Toolbox™ が必要です。並列でのタイミングに再現性がないため、並列ベイズ最適化で再現性のある結果が生成されるとは限りません。最適化の複雑度に応じて、特に大きなデータ セットでは、この処理に時間がかかる場合があります。
bayesianOptions = struct("UseParallel",true); [bayesianMdl,bayesianResults] = fitcauto(adultdata,"salary","Weights","fnlwgt", ... "HyperparameterOptimizationOptions",bayesianOptions);
Warning: Data set has more than 10000 observations. Because ASHA optimization often finds good solutions faster than Bayesian optimization for data sets with many observations, try specifying the 'Optimizer' field value as 'asha' in the 'HyperparameterOptimizationOptions' value structure.
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 8). Copying objective function to workers... Done copying objective function to workers. Learner types to explore: ensemble, nb, svm, tree Total iterations (MaxObjectiveEvaluations): 120 Total time (MaxTime): Inf |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 1 | 8 | Best | 0.14546 | 9.0638 | 0.14546 | 0.14546 | tree | MinLeafSize: 118 | | 2 | 7 | Accept | 0.24677 | 11.074 | 0.14546 | 0.14704 | svm | BoxConstraint: 3.8856 | | | | | | | | | | KernelScale: 0.00432 | | 3 | 7 | Accept | 0.14868 | 9.9597 | 0.14546 | 0.14704 | tree | MinLeafSize: 26 | | 4 | 8 | Accept | 0.15457 | 156.73 | 0.14546 | 0.14704 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 203 | | | | | | | | | | MinLeafSize: 105 | | 5 | 8 | Accept | 0.18015 | 153.77 | 0.14546 | 0.14704 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 239 | | | | | | | | | | MinLeafSize: 2417 | | 6 | 8 | Accept | 0.14921 | 233.35 | 0.14546 | 0.14704 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 258 | | | | | | | | | | MinLeafSize: 542 | | 7 | 8 | Accept | 0.24677 | 113.54 | 0.14546 | 0.14704 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 298 | | | | | | | | | | MinLeafSize: 11733 | | 8 | 8 | Best | 0.14517 | 318.06 | 0.14517 | 0.14704 | svm | BoxConstraint: 0.10777 | | | | | | | | | | KernelScale: 4.1321 | | 9 | 8 | Accept | 0.15414 | 196.23 | 0.14517 | 0.14704 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 267 | | | | | | | | | | MinLeafSize: 1 | | 10 | 8 | Accept | 0.24677 | 382.69 | 0.14517 | 0.14704 | svm | BoxConstraint: 0.043508 | | | | | | | | | | KernelScale: 931.39 | | 11 | 8 | Accept | 0.24677 | 393.3 | 0.14517 | 0.14704 | svm | BoxConstraint: 0.043508 | | | | | | | | | | KernelScale: 931.39 | | 12 | 8 | Accept | 0.16391 | 11.028 | 0.14517 | 0.1523 | tree | MinLeafSize: 5 | | 13 | 8 | Accept | 0.17328 | 2.6893 | 0.14517 | 0.15762 | tree | MinLeafSize: 846 | | 14 | 8 | Accept | 0.24677 | 1.4486 | 0.14517 | 0.17323 | tree | MinLeafSize: 8098 | | 15 | 8 | Accept | 0.24169 | 425.67 | 0.14517 | 0.17323 | svm | BoxConstraint: 0.15557 | | | | | | | | | | KernelScale: 72.871 | | 16 | 8 | Accept | 0.24677 | 1.0011 | 0.14517 | 0.16798 | tree | MinLeafSize: 11604 | | 17 | 8 | Accept | 0.1511 | 88.912 | 0.14517 | 0.1511 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.67978 | | 18 | 8 | Accept | 0.18099 | 162.32 | 0.14517 | 0.16694 | nb | DistributionNames: kernel | | | | | | | | | | Width: 232.3 | | 19 | 8 | Accept | 0.18403 | 2.8724 | 0.14517 | 0.16363 | tree | MinLeafSize: 2226 | | 20 | 8 | Accept | 0.24677 | 9.3034 | 0.14517 | 0.16363 | svm | BoxConstraint: 0.019712 | | | | | | | | | | KernelScale: 0.0047701 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 21 | 8 | Accept | 0.15364 | 141.13 | 0.14517 | 0.16209 | nb | DistributionNames: kernel | | | | | | | | | | Width: 3.2128 | | 22 | 8 | Accept | 0.16345 | 351.87 | 0.14517 | 0.16209 | svm | BoxConstraint: 0.0017882 | | | | | | | | | | KernelScale: 3.4862 | | 23 | 8 | Accept | 0.1585 | 391.52 | 0.14517 | 0.16209 | svm | BoxConstraint: 19.675 | | | | | | | | | | KernelScale: 205.34 | | 24 | 8 | Accept | 0.18472 | 180.24 | 0.14517 | 0.16363 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1676.9 | | 25 | 8 | Accept | 0.15005 | 209.17 | 0.14517 | 0.15973 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 218 | | | | | | | | | | MinLeafSize: 34 | | 26 | 8 | Accept | 0.15429 | 172.14 | 0.14517 | 0.15973 | nb | DistributionNames: kernel | | | | | | | | | | Width: 3.6464 | | 27 | 8 | Accept | 0.168 | 1.2833 | 0.14517 | 0.15973 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 28 | 8 | Accept | 0.15045 | 225.03 | 0.14517 | 0.15445 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 235 | | | | | | | | | | MinLeafSize: 7 | | 29 | 8 | Accept | 0.168 | 2.4337 | 0.14517 | 0.15445 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 30 | 8 | Accept | 0.168 | 2.0997 | 0.14517 | 0.15445 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 31 | 8 | Accept | 0.15761 | 198.66 | 0.14517 | 0.15445 | nb | DistributionNames: kernel | | | | | | | | | | Width: 5.6802 | | 32 | 8 | Accept | 0.15055 | 214.5 | 0.14517 | 0.1515 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 202 | | | | | | | | | | MinLeafSize: 2 | | 33 | 8 | Accept | 0.15429 | 143.42 | 0.14517 | 0.1515 | nb | DistributionNames: kernel | | | | | | | | | | Width: 3.6279 | | 34 | 8 | Accept | 0.14984 | 211.73 | 0.14517 | 0.15084 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 271 | | | | | | | | | | MinLeafSize: 13 | | 35 | 8 | Accept | 0.15455 | 182.76 | 0.14517 | 0.15058 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 261 | | | | | | | | | | MinLeafSize: 1 | | 36 | 8 | Accept | 0.24182 | 435.16 | 0.14517 | 0.15058 | svm | BoxConstraint: 0.088843 | | | | | | | | | | KernelScale: 60.035 | | 37 | 8 | Accept | 0.15528 | 150.14 | 0.14517 | 0.15062 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 244 | | | | | | | | | | MinLeafSize: 3 | | 38 | 8 | Accept | 0.15115 | 5.0809 | 0.14517 | 0.15062 | tree | MinLeafSize: 262 | | 39 | 8 | Accept | 0.14982 | 154.77 | 0.14517 | 0.15003 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 218 | | | | | | | | | | MinLeafSize: 19 | | 40 | 8 | Accept | 0.15044 | 145.66 | 0.14517 | 0.1501 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 201 | | | | | | | | | | MinLeafSize: 1 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 41 | 8 | Accept | 0.24677 | 0.61497 | 0.14517 | 0.1501 | tree | MinLeafSize: 12837 | | 42 | 8 | Accept | 0.17342 | 8.1501 | 0.14517 | 0.1501 | tree | MinLeafSize: 3 | | 43 | 8 | Accept | 0.15795 | 137.62 | 0.14517 | 0.15019 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 224 | | | | | | | | | | MinLeafSize: 2353 | | 44 | 8 | Accept | 0.17696 | 1.4365 | 0.14517 | 0.15019 | tree | MinLeafSize: 1170 | | 45 | 8 | Accept | 0.1683 | 144.68 | 0.14517 | 0.15019 | nb | DistributionNames: kernel | | | | | | | | | | Width: 18.063 | | 46 | 8 | Accept | 0.2011 | 125.96 | 0.14517 | 0.15019 | nb | DistributionNames: kernel | | | | | | | | | | Width: 58214 | | 47 | 8 | Accept | 0.1883 | 134.06 | 0.14517 | 0.15019 | nb | DistributionNames: kernel | | | | | | | | | | Width: 2612.2 | | 48 | 8 | Accept | 0.1494 | 865.49 | 0.14517 | 0.15019 | svm | BoxConstraint: 0.028795 | | | | | | | | | | KernelScale: 1.4357 | | 49 | 8 | Accept | 0.17342 | 7.5968 | 0.14517 | 0.15019 | tree | MinLeafSize: 3 | | 50 | 8 | Accept | 0.15016 | 3.9429 | 0.14517 | 0.15019 | tree | MinLeafSize: 171 | | 51 | 8 | Accept | 0.16106 | 170.49 | 0.14517 | 0.1505 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 248 | | | | | | | | | | MinLeafSize: 358 | | 52 | 8 | Accept | 0.17901 | 9.2773 | 0.14517 | 0.1505 | tree | MinLeafSize: 2 | | 53 | 8 | Accept | 0.17487 | 146.78 | 0.14517 | 0.1505 | nb | DistributionNames: kernel | | | | | | | | | | Width: 65.783 | | 54 | 8 | Accept | 0.15025 | 243.33 | 0.14517 | 0.15019 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 293 | | | | | | | | | | MinLeafSize: 37 | | 55 | 8 | Accept | 0.23138 | 406.36 | 0.14517 | 0.15019 | svm | BoxConstraint: 0.064193 | | | | | | | | | | KernelScale: 38.054 | | 56 | 8 | Accept | 0.15038 | 245.69 | 0.14517 | 0.15032 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 300 | | | | | | | | | | MinLeafSize: 16 | | 57 | 8 | Accept | 0.14853 | 224.07 | 0.14517 | 0.15057 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 259 | | | | | | | | | | MinLeafSize: 684 | | 58 | 8 | Accept | 0.15075 | 210.06 | 0.14517 | 0.15016 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 296 | | | | | | | | | | MinLeafSize: 33 | | 59 | 8 | Accept | 0.14981 | 221.88 | 0.14517 | 0.15016 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 300 | | | | | | | | | | MinLeafSize: 18 | | 60 | 8 | Accept | 0.15111 | 192.41 | 0.14517 | 0.15004 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 250 | | | | | | | | | | MinLeafSize: 1571 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 61 | 8 | Accept | 0.15018 | 219.2 | 0.14517 | 0.15006 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 297 | | | | | | | | | | MinLeafSize: 29 | | 62 | 8 | Accept | 0.14983 | 207.92 | 0.14517 | 0.14994 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 272 | | | | | | | | | | MinLeafSize: 553 | | 63 | 8 | Accept | 0.15006 | 208.62 | 0.14517 | 0.15004 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 299 | | | | | | | | | | MinLeafSize: 13 | | 64 | 8 | Accept | 0.15116 | 199.41 | 0.14517 | 0.14986 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 256 | | | | | | | | | | MinLeafSize: 1574 | | 65 | 8 | Accept | 0.14934 | 214.65 | 0.14517 | 0.14923 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 264 | | | | | | | | | | MinLeafSize: 1469 | | 66 | 8 | Accept | 0.16747 | 359.01 | 0.14517 | 0.14923 | svm | BoxConstraint: 0.033521 | | | | | | | | | | KernelScale: 12.508 | | 67 | 8 | Accept | 0.14883 | 219.75 | 0.14517 | 0.14919 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 254 | | | | | | | | | | MinLeafSize: 986 | | 68 | 8 | Accept | 0.14959 | 207 | 0.14517 | 0.14912 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 259 | | | | | | | | | | MinLeafSize: 1173 | | 69 | 8 | Accept | 0.15016 | 207.07 | 0.14517 | 0.14918 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 290 | | | | | | | | | | MinLeafSize: 1897 | | 70 | 8 | Accept | 0.14754 | 275.92 | 0.14517 | 0.14918 | svm | BoxConstraint: 0.10603 | | | | | | | | | | KernelScale: 5.0157 | | 71 | 8 | Accept | 0.14794 | 282.2 | 0.14517 | 0.14918 | svm | BoxConstraint: 0.11374 | | | | | | | | | | KernelScale: 5.4724 | | 72 | 8 | Accept | 0.15093 | 190.3 | 0.14517 | 0.14914 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 267 | | | | | | | | | | MinLeafSize: 1707 | | 73 | 8 | Accept | 0.14908 | 300.63 | 0.14517 | 0.14914 | svm | BoxConstraint: 0.087041 | | | | | | | | | | KernelScale: 6.0336 | | 74 | 8 | Accept | 0.14875 | 246.04 | 0.14517 | 0.14913 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 299 | | | | | | | | | | MinLeafSize: 934 | | 75 | 8 | Accept | 0.16303 | 345.34 | 0.14517 | 0.14913 | svm | BoxConstraint: 0.10529 | | | | | | | | | | KernelScale: 18.53 | | 76 | 8 | Accept | 0.14828 | 274.59 | 0.14517 | 0.14913 | svm | BoxConstraint: 0.060389 | | | | | | | | | | KernelScale: 4.7508 | | 77 | 8 | Accept | 0.15908 | 326.16 | 0.14517 | 0.14913 | svm | BoxConstraint: 0.11 | | | | | | | | | | KernelScale: 16.289 | | 78 | 8 | Accept | 0.15976 | 297.48 | 0.14517 | 0.14852 | svm | BoxConstraint: 0.26147 | | | | | | | | | | KernelScale: 25.792 | | 79 | 8 | Accept | 0.14857 | 279.1 | 0.14517 | 0.14802 | svm | BoxConstraint: 0.048355 | | | | | | | | | | KernelScale: 4.4186 | | 80 | 8 | Accept | 0.14944 | 302.39 | 0.14517 | 0.14845 | svm | BoxConstraint: 0.041852 | | | | | | | | | | KernelScale: 4.9836 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 81 | 8 | Accept | 0.19487 | 401.11 | 0.14517 | 0.14861 | svm | BoxConstraint: 0.078452 | | | | | | | | | | KernelScale: 30.022 | | 82 | 8 | Accept | 0.1461 | 315.95 | 0.14517 | 0.14876 | svm | BoxConstraint: 0.15302 | | | | | | | | | | KernelScale: 4.7768 | | 83 | 8 | Accept | 0.15583 | 333.62 | 0.14517 | 0.14716 | svm | BoxConstraint: 0.0071375 | | | | | | | | | | KernelScale: 4.1809 | | 84 | 8 | Accept | 0.14761 | 297.08 | 0.14517 | 0.1478 | svm | BoxConstraint: 0.070313 | | | | | | | | | | KernelScale: 4.4613 | | 85 | 8 | Accept | 0.14531 | 326.83 | 0.14517 | 0.14706 | svm | BoxConstraint: 0.73357 | | | | | | | | | | KernelScale: 5.3351 | | 86 | 8 | Accept | 0.14691 | 303.06 | 0.14517 | 0.14643 | svm | BoxConstraint: 0.089237 | | | | | | | | | | KernelScale: 4.479 | | 87 | 8 | Accept | 0.15009 | 313.15 | 0.14517 | 0.14635 | svm | BoxConstraint: 0.028044 | | | | | | | | | | KernelScale: 4.6944 | | 88 | 8 | Accept | 0.1578 | 212.73 | 0.14517 | 0.14635 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 283 | | | | | | | | | | MinLeafSize: 2103 | | 89 | 8 | Accept | 0.14521 | 456.62 | 0.14517 | 0.14617 | svm | BoxConstraint: 1.9049 | | | | | | | | | | KernelScale: 4.7834 | | 90 | 8 | Accept | 0.15028 | 224.72 | 0.14517 | 0.14617 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 271 | | | | | | | | | | MinLeafSize: 1953 | | 91 | 8 | Accept | 0.15213 | 316.5 | 0.14517 | 0.14614 | svm | BoxConstraint: 0.016203 | | | | | | | | | | KernelScale: 4.3045 | | 92 | 8 | Accept | 0.14533 | 405.52 | 0.14517 | 0.14618 | svm | BoxConstraint: 2.5334 | | | | | | | | | | KernelScale: 5.0734 | | 93 | 8 | Accept | 0.4955 | 5808 | 0.14517 | 0.14629 | svm | BoxConstraint: 0.0055052 | | | | | | | | | | KernelScale: 0.11946 | | 94 | 8 | Accept | 0.14599 | 421.4 | 0.14517 | 0.14552 | svm | BoxConstraint: 0.1423 | | | | | | | | | | KernelScale: 2.5078 | | 95 | 8 | Accept | 0.15696 | 334.19 | 0.14517 | 0.1459 | svm | BoxConstraint: 0.0086085 | | | | | | | | | | KernelScale: 4.7011 | | 96 | 8 | Accept | 0.16125 | 335.61 | 0.14517 | 0.14572 | svm | BoxConstraint: 0.0043914 | | | | | | | | | | KernelScale: 4.3671 | | 97 | 8 | Accept | 0.24245 | 6787.2 | 0.14517 | 0.14594 | svm | BoxConstraint: 11.252 | | | | | | | | | | KernelScale: 0.48339 | | 98 | 8 | Accept | 0.16294 | 354.47 | 0.14517 | 0.14602 | svm | BoxConstraint: 0.0062158 | | | | | | | | | | KernelScale: 5.3367 | | 99 | 8 | Accept | 0.16271 | 339.55 | 0.14517 | 0.14604 | svm | BoxConstraint: 0.0046539 | | | | | | | | | | KernelScale: 4.7345 | | 100 | 8 | Accept | 0.17184 | 5978.5 | 0.14517 | 0.14589 | svm | BoxConstraint: 7.6879 | | | | | | | | | | KernelScale: 1.6339 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 101 | 8 | Accept | 0.16142 | 355.72 | 0.14517 | 0.14624 | svm | BoxConstraint: 0.0057866 | | | | | | | | | | KernelScale: 4.8082 | | 102 | 8 | Accept | 0.15067 | 1030.9 | 0.14517 | 0.14639 | svm | BoxConstraint: 0.0050117 | | | | | | | | | | KernelScale: 1.0093 | | 103 | 8 | Best | 0.14513 | 501.17 | 0.14513 | 0.1457 | svm | BoxConstraint: 3.1097 | | | | | | | | | | KernelScale: 5.013 | | 104 | 8 | Accept | 0.35434 | 7976.9 | 0.14513 | 0.14584 | svm | BoxConstraint: 0.06472 | | | | | | | | | | KernelScale: 0.0099007 | | 105 | 8 | Accept | 0.26591 | 8022.4 | 0.14513 | 0.14609 | svm | BoxConstraint: 3.0956 | | | | | | | | | | KernelScale: 0.013802 | | 106 | 8 | Accept | 0.35499 | 5418.2 | 0.14513 | 0.14564 | svm | BoxConstraint: 0.37919 | | | | | | | | | | KernelScale: 0.14009 | | 107 | 8 | Accept | 0.167 | 6198.1 | 0.14513 | 0.14563 | svm | BoxConstraint: 659.16 | | | | | | | | | | KernelScale: 4.3029 | | 108 | 8 | Accept | 0.14539 | 747.98 | 0.14513 | 0.14574 | svm | BoxConstraint: 5.9153 | | | | | | | | | | KernelScale: 4.9842 | | 109 | 8 | Accept | 0.14543 | 586.07 | 0.14513 | 0.14532 | svm | BoxConstraint: 3.9618 | | | | | | | | | | KernelScale: 4.9501 | | 110 | 8 | Accept | 0.17056 | 5739.7 | 0.14513 | 0.1457 | svm | BoxConstraint: 0.0024422 | | | | | | | | | | KernelScale: 0.46361 | | 111 | 8 | Accept | 0.29426 | 4906.9 | 0.14513 | 0.14557 | svm | BoxConstraint: 0.0064138 | | | | | | | | | | KernelScale: 0.31177 | | 112 | 8 | Accept | 0.1717 | 5907.9 | 0.14513 | 0.14571 | svm | BoxConstraint: 0.10504 | | | | | | | | | | KernelScale: 0.82207 | | 113 | 8 | Accept | 0.17241 | 5909.3 | 0.14513 | 0.14549 | svm | BoxConstraint: 0.028608 | | | | | | | | | | KernelScale: 0.64777 | | 114 | 8 | Accept | 0.17806 | 6444.5 | 0.14513 | 0.14558 | svm | BoxConstraint: 0.34511 | | | | | | | | | | KernelScale: 0.72366 | | 115 | 8 | Accept | 0.16555 | 5481 | 0.14513 | 0.14533 | svm | BoxConstraint: 0.17641 | | | | | | | | | | KernelScale: 1.0964 | | 116 | 8 | Accept | 0.14788 | 1249.1 | 0.14513 | 0.14565 | svm | BoxConstraint: 11.881 | | | | | | | | | | KernelScale: 4.8553 | | 117 | 8 | Accept | 0.24664 | 5351.2 | 0.14513 | 0.14631 | svm | BoxConstraint: 0.0048502 | | | | | | | | | | KernelScale: 0.38595 | | 118 | 8 | Best | 0.14483 | 563.53 | 0.14483 | 0.1454 | svm | BoxConstraint: 2.2083 | | | | | | | | | | KernelScale: 4.5125 | | 119 | 8 | Accept | 0.15403 | 340.78 | 0.14483 | 0.14559 | svm | BoxConstraint: 0.843 | | | | | | | | | | KernelScale: 26.671 | | 120 | 8 | Accept | 0.27575 | 4752.3 | 0.14483 | 0.14532 | svm | BoxConstraint: 0.0017988 | | | | | | | | | | KernelScale: 0.21724 |
__________________________________________________________ Optimization completed. Total iterations: 120 Total elapsed time: 17271.6899 seconds Total time for training and validation: 116613.8436 seconds Best observed learner is an svm model with: Learner: svm BoxConstraint: 2.2083 KernelScale: 4.5125 Observed validation loss: 0.14483 Time for training and validation: 563.5264 seconds Best estimated learner (returned model) is an svm model with: Learner: svm BoxConstraint: 3.1097 KernelScale: 5.013 Estimated validation loss: 0.14532 Estimated time for training and validation: 574.6554 seconds Documentation for fitcauto display
Total elapsed time の値から、ベイズ最適化の実行に時間がかかったことがわかります (約 4.8 時間)。
fitcauto によって返される最終的なモデルが、最適な推定学習器となります。モデルを返す前に、関数は学習データ セット全体 (adultdata)、リストされている Learner (またはモデル) のタイプ、および表示されたハイパーパラメーター値を使用して、モデルの再学習を行います。
ASHA 最適化による自動モデル選択の使用
学習セットの観測値の数が原因でベイズ最適化による fitcauto の実行に長い時間がかかる場合は、代わりに ASHA 最適化による fitcauto を使用することを検討してください。adultdata に含まれる観測値が 10,000 を超える場合は、ASHA 最適化による fitcauto を使用して適切な分類器を自動的に見つけるよう試します。ASHA 最適化による fitcauto を使用すると、関数はさまざまなハイパーパラメーターの値をもつ複数のモデルを無作為に選択し、学習データの小さいサブセットで学習させます。交差検証の分類誤差 (Validation Loss) に基づいて有望なモデルが見つかると、そのモデルをプロモートし、より多くの学習データで学習させます。このプロセスを繰り返し、データの量を徐々に増やしながら有望なモデルに学習させます。既定の設定では、fitcauto は、最適化のプロット、および最適化の結果の反復表示を提供します。これらの結果を解釈する方法の詳細については、Verbose の表示を参照してください。
観測値の重みを設定し、ASHA 最適化を並列実行するよう指定します。ASHA 最適化は既定のベイズ最適化に比べて反復回数が多くなる場合が多いことに注意してください。時間の制約がある場合は、HyperparameterOptimizationOptions 構造体の MaxTime フィールドを指定して、fitcauto を実行する秒数を制限できます。
ashaOptions = struct("Optimizer","asha","UseParallel",true); [ashaMdl,ashaResults] = fitcauto(adultdata,"salary","Weights","fnlwgt", ... "HyperparameterOptimizationOptions",ashaOptions);
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Copying objective function to workers... Done copying objective function to workers. Learner types to explore: ensemble, nb, svm, tree Total iterations (MaxObjectiveEvaluations): 425 Total time (MaxTime): Inf |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 1 | 8 | Best | 0.1831 | 0.59595 | 0.1831 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 2 | 7 | Accept | 0.24677 | 0.64255 | 0.1831 | 378 | tree | MinLeafSize: 2865 | | 3 | 7 | Accept | 0.24677 | 0.88001 | 0.1831 | 378 | tree | MinLeafSize: 1629 | | 4 | 7 | Accept | 0.24677 | 0.49804 | 0.1831 | 378 | svm | BoxConstraint: 93.722 | | | | | | | | | | KernelScale: 0.0031746 | | 5 | 8 | Best | 0.15872 | 2.6522 | 0.15872 | 378 | svm | BoxConstraint: 622.14 | | | | | | | | | | KernelScale: 8.3947 | | 6 | 8 | Accept | 0.17662 | 0.88532 | 0.15872 | 1509 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 7 | 8 | Accept | 0.17844 | 0.7806 | 0.15872 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 8 | 7 | Accept | 0.24677 | 1.6871 | 0.15872 | 378 | svm | BoxConstraint: 16.149 | | | | | | | | | | KernelScale: 418.84 | | 9 | 7 | Accept | 0.18401 | 0.70381 | 0.15872 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 10 | 7 | Accept | 0.17244 | 0.31131 | 0.15872 | 378 | tree | MinLeafSize: 4 | | 11 | 8 | Accept | 0.17135 | 0.45227 | 0.15872 | 378 | tree | MinLeafSize: 13 | | 12 | 8 | Accept | 0.24677 | 0.53034 | 0.15872 | 378 | svm | BoxConstraint: 272.5 | | | | | | | | | | KernelScale: 0.0013306 | | 13 | 8 | Accept | 0.24677 | 0.34333 | 0.15872 | 378 | tree | MinLeafSize: 701 | | 14 | 8 | Best | 0.15344 | 1.0465 | 0.15344 | 1509 | tree | MinLeafSize: 13 | | 15 | 8 | Accept | 0.1538 | 12.371 | 0.15344 | 1509 | svm | BoxConstraint: 622.14 | | | | | | | | | | KernelScale: 8.3947 | | 16 | 8 | Accept | 0.24677 | 0.31034 | 0.15344 | 378 | tree | MinLeafSize: 1773 | | 17 | 8 | Accept | 0.186 | 21.17 | 0.15344 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 9030.3 | | 18 | 8 | Accept | 0.17043 | 21.308 | 0.15344 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 512.37 | | 19 | 8 | Accept | 0.18322 | 20.058 | 0.15344 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 30.45 | | 20 | 8 | Accept | 0.1802 | 1.1355 | 0.15344 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 21 | 8 | Accept | 0.18721 | 23.236 | 0.15344 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 9222.9 | | 22 | 8 | Accept | 0.20066 | 21.173 | 0.15344 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 129.74 | | 23 | 8 | Accept | 0.2205 | 17.802 | 0.15344 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 30996 | | 24 | 8 | Accept | 0.155 | 0.81516 | 0.15344 | 1509 | tree | MinLeafSize: 4 | | 25 | 8 | Best | 0.14361 | 1.0991 | 0.14361 | 6033 | tree | MinLeafSize: 13 | | 26 | 8 | Accept | 0.19409 | 33.202 | 0.14361 | 378 | svm | BoxConstraint: 259.24 | | | | | | | | | | KernelScale: 0.32997 | | 27 | 8 | Accept | 0.24677 | 2.6882 | 0.14361 | 378 | svm | BoxConstraint: 0.0059822 | | | | | | | | | | KernelScale: 713.08 | | 28 | 8 | Accept | 0.16587 | 1.8275 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 29 | 8 | Accept | 0.18015 | 16.315 | 0.14361 | 378 | svm | BoxConstraint: 0.5769 | | | | | | | | | | KernelScale: 0.36299 | | 30 | 8 | Accept | 0.17001 | 1.1164 | 0.14361 | 1509 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 31 | 8 | Accept | 0.18193 | 0.55339 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 32 | 8 | Accept | 0.16862 | 24.418 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 102.7 | | 33 | 7 | Accept | 0.241 | 2.5999 | 0.14361 | 378 | svm | BoxConstraint: 0.6369 | | | | | | | | | | KernelScale: 23.843 | | 34 | 7 | Accept | 0.1821 | 1.0709 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 35 | 8 | Accept | 0.18464 | 29.794 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 9256.9 | | 36 | 8 | Accept | 0.24677 | 34.988 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 289 | | | | | | | | | | MinLeafSize: 929 | | 37 | 8 | Accept | 0.16073 | 50.947 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 283 | | | | | | | | | | MinLeafSize: 5 | | 38 | 8 | Accept | 0.17086 | 52.897 | 0.14361 | 1509 | nb | DistributionNames: kernel | | | | | | | | | | Width: 512.37 | | 39 | 8 | Accept | 0.17374 | 0.96688 | 0.14361 | 378 | tree | MinLeafSize: 7 | | 40 | 8 | Accept | 0.16598 | 37.543 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 281 | | | | | | | | | | MinLeafSize: 2 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 41 | 8 | Accept | 0.242 | 1.7295 | 0.14361 | 378 | svm | BoxConstraint: 0.0093388 | | | | | | | | | | KernelScale: 4.7672 | | 42 | 8 | Accept | 0.20568 | 32.829 | 0.14361 | 378 | svm | BoxConstraint: 0.091315 | | | | | | | | | | KernelScale: 0.30293 | | 43 | 8 | Accept | 0.16465 | 56.109 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 265 | | | | | | | | | | MinLeafSize: 2 | | 44 | 8 | Accept | 0.16598 | 2.096 | 0.14361 | 378 | svm | BoxConstraint: 49.164 | | | | | | | | | | KernelScale: 1.7376 | | 45 | 8 | Accept | 0.18456 | 52.012 | 0.14361 | 1509 | nb | DistributionNames: kernel | | | | | | | | | | Width: 102.7 | | 46 | 8 | Accept | 0.24677 | 0.77083 | 0.14361 | 378 | tree | MinLeafSize: 13706 | | 47 | 8 | Accept | 0.16477 | 25.694 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 218 | | | | | | | | | | MinLeafSize: 36 | | 48 | 8 | Accept | 0.17952 | 1.107 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 49 | 8 | Accept | 0.32627 | 84.486 | 0.14361 | 378 | svm | BoxConstraint: 549.6 | | | | | | | | | | KernelScale: 0.036915 | | 50 | 8 | Accept | 0.16346 | 14.925 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.65733 | | 51 | 8 | Accept | 0.24677 | 4.6465 | 0.14361 | 378 | svm | BoxConstraint: 0.9246 | | | | | | | | | | KernelScale: 398.37 | | 52 | 8 | Accept | 0.15285 | 42.395 | 0.14361 | 1509 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 283 | | | | | | | | | | MinLeafSize: 5 | | 53 | 8 | Accept | 0.24677 | 2.1044 | 0.14361 | 378 | svm | BoxConstraint: 0.041664 | | | | | | | | | | KernelScale: 29.281 | | 54 | 8 | Accept | 0.24677 | 39.272 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 216 | | | | | | | | | | MinLeafSize: 1096 | | 55 | 8 | Accept | 0.15401 | 48.294 | 0.14361 | 1509 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 218 | | | | | | | | | | MinLeafSize: 36 | | 56 | 8 | Accept | 0.14704 | 59.661 | 0.14361 | 1509 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 265 | | | | | | | | | | MinLeafSize: 2 | | 57 | 8 | Accept | 0.15196 | 41.835 | 0.14361 | 1509 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.65733 | | 58 | 8 | Accept | 0.24285 | 97.209 | 0.14361 | 378 | svm | BoxConstraint: 62.688 | | | | | | | | | | KernelScale: 0.019392 | | 59 | 8 | Accept | 0.24677 | 0.75744 | 0.14361 | 378 | tree | MinLeafSize: 159 | | 60 | 8 | Accept | 0.1928 | 24.43 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 24.876 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 61 | 8 | Accept | 0.20489 | 21.827 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 6524.9 | | 62 | 8 | Accept | 0.24677 | 0.5275 | 0.14361 | 378 | svm | BoxConstraint: 0.0012686 | | | | | | | | | | KernelScale: 0.0019115 | | 63 | 8 | Accept | 0.24677 | 0.43222 | 0.14361 | 378 | svm | BoxConstraint: 68.024 | | | | | | | | | | KernelScale: 0.0023633 | | 64 | 8 | Accept | 0.26755 | 94.048 | 0.14361 | 378 | svm | BoxConstraint: 3.4696 | | | | | | | | | | KernelScale: 0.026518 | | 65 | 8 | Accept | 0.25611 | 97.151 | 0.14361 | 378 | svm | BoxConstraint: 14.395 | | | | | | | | | | KernelScale: 0.018438 | | 66 | 8 | Accept | 0.24677 | 0.6123 | 0.14361 | 378 | svm | BoxConstraint: 6.1487 | | | | | | | | | | KernelScale: 0.0012672 | | 67 | 8 | Accept | 0.23521 | 38.994 | 0.14361 | 378 | svm | BoxConstraint: 0.47847 | | | | | | | | | | KernelScale: 0.21848 | | 68 | 8 | Accept | 0.14949 | 75.744 | 0.14361 | 6033 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 283 | | | | | | | | | | MinLeafSize: 5 | | 69 | 8 | Accept | 0.1689 | 26.118 | 0.14361 | 1509 | svm | BoxConstraint: 49.164 | | | | | | | | | | KernelScale: 1.7376 | | 70 | 8 | Accept | 0.17936 | 25.8 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 228 | | | | | | | | | | MinLeafSize: 164 | | 71 | 8 | Accept | 0.2466 | 34.448 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 239 | | | | | | | | | | MinLeafSize: 88 | | 72 | 8 | Accept | 0.15654 | 2.6753 | 0.14361 | 1509 | tree | MinLeafSize: 7 | | 73 | 8 | Accept | 0.18175 | 0.87169 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 74 | 8 | Accept | 0.16188 | 42.445 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 232 | | | | | | | | | | MinLeafSize: 14 | | 75 | 8 | Accept | 0.24677 | 2.9432 | 0.14361 | 378 | svm | BoxConstraint: 5.2509 | | | | | | | | | | KernelScale: 566.41 | | 76 | 8 | Accept | 0.24677 | 0.68866 | 0.14361 | 378 | svm | BoxConstraint: 121.38 | | | | | | | | | | KernelScale: 0.0038679 | | 77 | 8 | Accept | 0.24677 | 1.8912 | 0.14361 | 378 | svm | BoxConstraint: 4.2627 | | | | | | | | | | KernelScale: 139.4 | | 78 | 8 | Accept | 0.21364 | 25.979 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 69985 | | 79 | 8 | Accept | 0.16991 | 22.652 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 12.131 | | 80 | 8 | Accept | 0.24677 | 27.661 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 200 | | | | | | | | | | MinLeafSize: 285 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 81 | 8 | Accept | 0.16426 | 2.4413 | 0.14361 | 378 | svm | BoxConstraint: 0.10011 | | | | | | | | | | KernelScale: 3.7337 | | 82 | 8 | Accept | 0.17779 | 0.78782 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 83 | 8 | Accept | 0.15315 | 42.755 | 0.14361 | 1509 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 281 | | | | | | | | | | MinLeafSize: 2 | | 84 | 7 | Accept | 0.17499 | 1.8123 | 0.14361 | 378 | svm | BoxConstraint: 235.48 | | | | | | | | | | KernelScale: 1.5618 | | 85 | 7 | Accept | 0.1702 | 1.7494 | 0.14361 | 378 | svm | BoxConstraint: 1.7191 | | | | | | | | | | KernelScale: 1.7936 | | 86 | 8 | Accept | 0.17428 | 0.71717 | 0.14361 | 378 | tree | MinLeafSize: 1 | | 87 | 8 | Accept | 0.24677 | 0.35012 | 0.14361 | 378 | tree | MinLeafSize: 10340 | | 88 | 8 | Accept | 0.15506 | 4.6537 | 0.14361 | 1509 | svm | BoxConstraint: 0.10011 | | | | | | | | | | KernelScale: 3.7337 | | 89 | 8 | Accept | 0.17907 | 0.93664 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 90 | 8 | Accept | 0.15925 | 7.5152 | 0.14361 | 1509 | svm | BoxConstraint: 1.7191 | | | | | | | | | | KernelScale: 1.7936 | | 91 | 8 | Accept | 0.16344 | 30.309 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | | MinLeafSize: 25 | | 92 | 8 | Accept | 0.18234 | 0.92613 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 93 | 8 | Accept | 0.24677 | 1.8464 | 0.14361 | 378 | svm | BoxConstraint: 0.019224 | | | | | | | | | | KernelScale: 574.94 | | 94 | 8 | Accept | 0.1566 | 31.765 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 209 | | | | | | | | | | MinLeafSize: 2 | | 95 | 8 | Accept | 0.15317 | 50.355 | 0.14361 | 1509 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 232 | | | | | | | | | | MinLeafSize: 14 | | 96 | 8 | Accept | 0.24677 | 0.43111 | 0.14361 | 378 | tree | MinLeafSize: 95 | | 97 | 8 | Accept | 0.16512 | 35.087 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 239 | | | | | | | | | | MinLeafSize: 37 | | 98 | 8 | Accept | 0.17052 | 0.45685 | 0.14361 | 378 | tree | MinLeafSize: 2 | | 99 | 8 | Accept | 0.17347 | 1.1812 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 100 | 8 | Accept | 0.16896 | 54.887 | 0.14361 | 1509 | nb | DistributionNames: kernel | | | | | | | | | | Width: 12.131 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 101 | 8 | Accept | 0.18156 | 0.49723 | 0.14361 | 378 | tree | MinLeafSize: 59 | | 102 | 8 | Accept | 0.24677 | 0.68678 | 0.14361 | 378 | svm | BoxConstraint: 0.0010962 | | | | | | | | | | KernelScale: 0.0023771 | | 103 | 8 | Accept | 0.24677 | 33.615 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 262 | | | | | | | | | | MinLeafSize: 1898 | | 104 | 8 | Accept | 0.15192 | 81.549 | 0.14361 | 6033 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 265 | | | | | | | | | | MinLeafSize: 2 | | 105 | 8 | Accept | 0.15412 | 32.695 | 0.14361 | 1509 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 209 | | | | | | | | | | MinLeafSize: 2 | | 106 | 8 | Accept | 0.24677 | 2.1129 | 0.14361 | 378 | svm | BoxConstraint: 0.51162 | | | | | | | | | | KernelScale: 901.32 | | 107 | 8 | Accept | 0.24712 | 96.705 | 0.14361 | 378 | svm | BoxConstraint: 0.12031 | | | | | | | | | | KernelScale: 0.03735 | | 108 | 8 | Accept | 0.24677 | 0.5869 | 0.14361 | 378 | svm | BoxConstraint: 878.36 | | | | | | | | | | KernelScale: 0.0032037 | | 109 | 8 | Accept | 0.15335 | 46.629 | 0.14361 | 1509 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | | MinLeafSize: 25 | | 110 | 8 | Accept | 0.24677 | 42.807 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 229 | | | | | | | | | | MinLeafSize: 133 | | 111 | 8 | Accept | 0.18895 | 1.9037 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 112 | 8 | Accept | 0.16369 | 2.104 | 0.14361 | 1509 | tree | MinLeafSize: 2 | | 113 | 8 | Accept | 0.16353 | 1.0029 | 0.14361 | 378 | tree | MinLeafSize: 3 | | 114 | 8 | Accept | 0.15127 | 93.189 | 0.14361 | 6033 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.65733 | | 115 | 8 | Accept | 0.16092 | 25.041 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.59189 | | 116 | 8 | Accept | 0.24677 | 0.91976 | 0.14361 | 378 | tree | MinLeafSize: 123 | | 117 | 8 | Accept | 0.16083 | 55.658 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 241 | | | | | | | | | | MinLeafSize: 4 | | 118 | 8 | Accept | 0.17643 | 0.91472 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 119 | 8 | Accept | 0.15053 | 9.4088 | 0.14361 | 24130 | tree | MinLeafSize: 13 | | 120 | 8 | Accept | 0.18671 | 1.1927 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 121 | 8 | Accept | 0.15504 | 52.066 | 0.14361 | 1509 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 239 | | | | | | | | | | MinLeafSize: 37 | | 122 | 7 | Accept | 0.1873 | 0.88051 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 123 | 7 | Accept | 0.18138 | 1.4255 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 124 | 8 | Accept | 0.1836 | 1.6767 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 125 | 8 | Accept | 0.24677 | 0.53547 | 0.14361 | 378 | svm | BoxConstraint: 0.015196 | | | | | | | | | | KernelScale: 0.0050979 | | 126 | 8 | Accept | 0.16782 | 76.114 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 270 | | | | | | | | | | MinLeafSize: 21 | | 127 | 8 | Accept | 0.65702 | 109.5 | 0.14361 | 378 | svm | BoxConstraint: 11.102 | | | | | | | | | | KernelScale: 0.02167 | | 128 | 8 | Accept | 0.15785 | 0.97717 | 0.14361 | 1509 | tree | MinLeafSize: 3 | | 129 | 8 | Accept | 0.16396 | 43.391 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 206 | | | | | | | | | | MinLeafSize: 52 | | 130 | 8 | Accept | 0.20726 | 29.078 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 71433 | | 131 | 8 | Accept | 0.15259 | 46.724 | 0.14361 | 1509 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.59189 | | 132 | 8 | Accept | 0.19003 | 42.596 | 0.14361 | 378 | svm | BoxConstraint: 6.2931 | | | | | | | | | | KernelScale: 0.33148 | | 133 | 8 | Accept | 0.16091 | 28.84 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.1628 | | 134 | 8 | Accept | 0.17784 | 29.872 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 886.95 | | 135 | 8 | Accept | 0.15027 | 76.937 | 0.14361 | 1509 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 241 | | | | | | | | | | MinLeafSize: 4 | | 136 | 8 | Accept | 0.18327 | 29.845 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 4528.8 | | 137 | 8 | Accept | 0.24677 | 0.38473 | 0.14361 | 378 | tree | MinLeafSize: 10969 | | 138 | 8 | Accept | 0.15068 | 106.4 | 0.14361 | 6033 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 281 | | | | | | | | | | MinLeafSize: 2 | | 139 | 8 | Accept | 0.24677 | 27.445 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 223 | | | | | | | | | | MinLeafSize: 98 | | 140 | 8 | Accept | 0.24341 | 100.59 | 0.14361 | 378 | svm | BoxConstraint: 2.7209 | | | | | | | | | | KernelScale: 0.010215 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 141 | 8 | Accept | 0.24187 | 2.6075 | 0.14361 | 378 | svm | BoxConstraint: 0.0079129 | | | | | | | | | | KernelScale: 4.8667 | | 142 | 8 | Accept | 0.17089 | 0.53327 | 0.14361 | 378 | tree | MinLeafSize: 3 | | 143 | 8 | Accept | 0.15068 | 41.021 | 0.14361 | 1509 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.1628 | | 144 | 8 | Accept | 0.24677 | 26.637 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 258 | | | | | | | | | | MinLeafSize: 6413 | | 145 | 8 | Accept | 0.24677 | 26.876 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 283 | | | | | | | | | | MinLeafSize: 3673 | | 146 | 8 | Accept | 0.16076 | 3.2005 | 0.14361 | 378 | svm | BoxConstraint: 4.4667 | | | | | | | | | | KernelScale: 13.015 | | 147 | 8 | Accept | 0.1567 | 2.1777 | 0.14361 | 378 | svm | BoxConstraint: 0.35692 | | | | | | | | | | KernelScale: 4.2583 | | 148 | 8 | Accept | 0.18396 | 0.90132 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 149 | 8 | Accept | 0.15063 | 6.9084 | 0.14361 | 1509 | svm | BoxConstraint: 0.35692 | | | | | | | | | | KernelScale: 4.2583 | | 150 | 8 | Accept | 0.17674 | 1.2601 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 151 | 8 | Accept | 0.156 | 35.953 | 0.14361 | 1509 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 206 | | | | | | | | | | MinLeafSize: 52 | | 152 | 8 | Accept | 0.16561 | 27.77 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 905.18 | | 153 | 8 | Accept | 0.24677 | 34.957 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 295 | | | | | | | | | | MinLeafSize: 4587 | | 154 | 8 | Accept | 0.17262 | 25.102 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 7.0619 | | 155 | 8 | Accept | 0.1518 | 94.985 | 0.14361 | 6033 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.59189 | | 156 | 8 | Accept | 0.15191 | 4.8239 | 0.14361 | 1509 | svm | BoxConstraint: 4.4667 | | | | | | | | | | KernelScale: 13.015 | | 157 | 8 | Accept | 0.70666 | 88.91 | 0.14361 | 378 | svm | BoxConstraint: 144.03 | | | | | | | | | | KernelScale: 0.0087964 | | 158 | 8 | Accept | 0.20845 | 1.2899 | 0.14361 | 378 | svm | BoxConstraint: 14.266 | | | | | | | | | | KernelScale: 0.56134 | | 159 | 8 | Accept | 0.16207 | 2.2155 | 0.14361 | 378 | svm | BoxConstraint: 0.018808 | | | | | | | | | | KernelScale: 1.4491 | | 160 | 8 | Accept | 0.17735 | 0.30364 | 0.14361 | 378 | tree | MinLeafSize: 16 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 161 | 8 | Accept | 0.14814 | 3.7088 | 0.14361 | 1509 | svm | BoxConstraint: 0.018808 | | | | | | | | | | KernelScale: 1.4491 | | 162 | 8 | Accept | 0.15674 | 54.506 | 0.14361 | 1509 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 270 | | | | | | | | | | MinLeafSize: 21 | | 163 | 8 | Accept | 0.18367 | 22.255 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 79.973 | | 164 | 8 | Accept | 0.1802 | 0.81271 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 165 | 8 | Accept | 0.27102 | 87.837 | 0.14361 | 378 | svm | BoxConstraint: 2.3737 | | | | | | | | | | KernelScale: 0.0096356 | | 166 | 8 | Accept | 0.24677 | 15.367 | 0.14361 | 378 | ensemble | Method: LogitBoost | | | | | | | | | | NumLearningCycles: 211 | | | | | | | | | | MinLeafSize: 8387 | | 167 | 8 | Accept | 0.16134 | 17.629 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.1013 | | 168 | 8 | Accept | 0.24677 | 1.5152 | 0.14361 | 378 | svm | BoxConstraint: 0.0011611 | | | | | | | | | | KernelScale: 20.262 | | 169 | 8 | Accept | 0.1889 | 0.58768 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 170 | 8 | Accept | 0.20571 | 21.075 | 0.14361 | 378 | nb | DistributionNames: kernel | | | | | | | | | | Width: 51434 | | 171 | 8 | Accept | 0.17171 | 0.54565 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 172 | 8 | Accept | 0.16136 | 43.429 | 0.14361 | 378 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 264 | | | | | | | | | | MinLeafSize: 1 | | 173 | 8 | Accept | 0.24677 | 0.74563 | 0.14361 | 378 | svm | BoxConstraint: 82.364 | | | | | | | | | | KernelScale: 0.0048723 | | 174 | 8 | Accept | 0.17774 | 0.82021 | 0.14361 | 378 | nb | DistributionNames: normal | | | | | | | | | | Width: ...
__________________________________________________________ Optimization completed. Total iterations: 425 Total elapsed time: 1225.1049 seconds Total time for training and validation: 8476.6632 seconds Best observed learner is a tree model with: Learner: tree MinLeafSize: 14 Observed validation loss: 0.14138 Time for training and validation: 1.6545 seconds Documentation for fitcauto display
Total elapsed time の値から、ASHA 最適化の方がベイズ最適化よりも実行時間が短くなったことがわかります (約 0.3 時間)。
fitcauto によって返される最終的なモデルが、観測された最適な学習器となります。モデルを返す前に、関数は学習データ セット全体 (adultdata)、リストされている Learner (またはモデル) のタイプ、および表示されたハイパーパラメーター値を使用して、モデルの再学習を行います。
テスト セットのパフォーマンスの評価
混同行列と受信者動作特性 (ROC) 曲線を使用して、返されたモデル bayesianMdl と ashaMdl の性能をテスト セット adulttest で評価します。
各モデルについて、テスト セットの予測されるラベルとスコア値を見つけます。
[bayesianLabels,bayesianScores] = predict(bayesianMdl,adulttest); [ashaLabels,ashaScores] = predict(ashaMdl,adulttest);
テスト セットの結果から、混同行列を作成します。対角要素は、特定のクラスの正しく分類されたインスタンスの数を示しています。非対角要素は誤分類した観測値のインスタンスです。1 行 2 列のタイル レイアウトを使用して結果を比較します。
tiledlayout(1,2) nexttile confusionchart(adulttest.salary,bayesianLabels) title("Bayesian Optimization") nexttile confusionchart(adulttest.salary,ashaLabels) title("ASHA Optimization")
各モデルについて、テスト セットの分類精度を計算します。精度は、テスト セットの正しく分類された観測値の割合です。
bayesianAccuracy = (1-loss(bayesianMdl,adulttest,"salary"))*100bayesianAccuracy = 85.2062
ashaAccuracy = (1-loss(ashaMdl,adulttest,"salary"))*100ashaAccuracy = 84.1612
混同行列と精度の値から、テスト セットでの性能は bayesianMdl の方が ashaMdl よりもわずかに優れていることがわかります。ただし、どちらのモデルも十分な性能を示しています。
それぞれのモデルについて、ROC 曲線をプロットし、ROC 曲線の下の領域 (AUC) を計算します。ROC 曲線は、分類スコアのさまざまなしきい値についての真陽性率と偽陽性率の関係を示します。しきい値にかかわらず真陽性率が常に 1 の完璧な分類器では、AUC = 1 になります。観測値をランダムにクラスに割り当てるバイナリ分類器では、AUC = 0.5 になります。大きな AUC 値 (1 に近い) は、分類器の性能が高いことを示します。
それぞれのモデルについて、rocmetrics オブジェクトを作成し、ROC 曲線のメトリクスを計算して AUC の値を求めます。
bayesianROC = rocmetrics(adulttest.salary,bayesianScores,bayesianMdl.ClassNames); ashaROC = rocmetrics(adulttest.salary,ashaScores,ashaMdl.ClassNames);
rocmetrics の関数 plot を使用して、ラベル <=50K の ROC 曲線をプロットします。
figure [r1,g1] = plot(bayesianROC,"ClassNames","<=50K"); hold on [r2,g2] = plot(ashaROC,"ClassNames","<=50K"); r1.DisplayName = replace(r1.DisplayName,"<=50K","Bayesian Optimization"); r2.DisplayName = replace(r2.DisplayName,"<=50K","ASHA Optimization"); g1(1).DisplayName = "Bayesian Optimization Model Operating Point"; g2(1).DisplayName = "ASHA Optimization Model Operating Point"; title("ROC Curves for Class <=50K") hold off
AUC 値を基準にすると、どちらの分類器もテスト データに対して適切に機能しています。
参考
fitcauto | confusionchart | perfcurve | BayesianOptimization
関連するトピック
You can also select a web site from the following list:
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)