このページの翻訳は最新ではありません。ここをクリックして、英語の最新版を参照してください。
fitcauto
構文
説明
fitcauto は、指定した予測子と応答データに基づいて、さまざまなハイパーパラメーターの値をもつ分類モデルのタイプの選択を自動的に試します。既定では、この関数は、ベイズ最適化を使用してモデルとそのハイパーパラメーターの値を選択し、各モデルの交差検証の分類誤差を計算します。最適化が完了すると、fitcauto は、データセット全体で学習済みの、新しいデータの分類に最適であると予測したモデルを返します。返されたモデルのオブジェクト関数 predict と loss を使用して、新しいデータの分類、およびテスト セットの分類誤差の計算をそれぞれ行うことができます。
データに最適な分類器のタイプがわからない場合は、fitcauto を使用します。分類モデルのハイパーパラメーターを調整するための代替方法については、代替機能を参照してください。
データに含まれる観測値が 10,000 を超える場合は、fitcauto を実行するとき、ベイズ最適化の代わりに非同期連続半減アルゴリズム (ASHA) の使用を検討してください。データ セットの観測値が多いと、多くの場合、ベイズ最適化よりも ASHA 最適化の方が優れた解を速く見つけます。
Mdl = fitcauto(Tbl,ResponseVarName)Mdl を返します。テーブル Tbl には予測子変数と応答変数を格納します。ResponseVarName は応答変数の名前です。
Mdl = fitcauto(___,Name,Value)HyperparameterOptimizationOptions を使用して、ベイズ最適化 (既定) または非同期連続半減アルゴリズム (ASHA) を使用するかどうかを指定します。ASHA 最適化を使用するには、"HyperparameterOptimizationOptions",struct("Optimizer","asha") を指定します。この構造体に追加フィールドを含めて、その他の最適化の側面を制御できます。
[ は、Mdl,OptimizationResults] = fitcauto(___)OptimizationResults も返します。これには、モデル選択とハイパーパラメーター調整のプロセスの結果が含まれます。この出力は、ベイズ最適化を使用している場合は BayesianOptimization オブジェクトで、ASHA 最適化を使用している場合は table です。
例
table データを使用した分類器の自動選択
fitcauto を使用し、table で指定した予測子データと応答データに基づいて、最適化されたハイパーパラメーターをもつ分類モデルを自動的に選択します。
データの読み込み
carbig データセットを読み込みます。このデータセットには、1970 年代と 1980 年代初期に製造された自動車の測定値が格納されています。
load carbig米国製かどうかに基づいて、自動車を分類します。
Origin = categorical(cellstr(Origin)); Origin = mergecats(Origin,["France","Japan","Germany", ... "Sweden","Italy","England"],"NotUSA");
Acceleration、Displacement などの予測子変数と応答変数 Origin が格納された table を作成します。
cars = table(Acceleration,Displacement,Horsepower, ...
Model_Year,MPG,Weight,Origin);データの分割
データを学習セットとテスト セットに分割します。モデル選択とハイパーパラメーター調整のプロセスに観測値の約 80% を使用し、fitcauto によって返された最終モデルのパフォーマンスのテストに観測値の 20% を使用します。cvpartition を使用してデータを分割します。
rng("default") % For reproducibility of the data partition c = cvpartition(Origin,"Holdout",0.2); trainingIdx = training(c); % Training set indices carsTrain = cars(trainingIdx,:); testIdx = test(c); % Test set indices carsTest = cars(testIdx,:);
fitcauto の実行
学習データを fitcauto に渡します。既定では、fitcauto は、試行する適切なモデルのタイプを決定し、ベイズ最適化を使用して適切なハイパーパラメーター値を求め、パフォーマンスが最大となることが期待される学習済みモデル Mdl を返します。さらに、fitcauto は、最適化のプロット、および最適化の結果の反復表示を提供します。これらの結果を解釈する方法の詳細については、Verbose の表示を参照してください。
このプロセスにいくらか時間がかかることを見込んでください。Parallel Computing Toolbox™ のライセンスがある場合、最適化を並列で実行するよう指定して最適化プロセスを高速化することを検討してください。これを行うには、名前と値の引数 "HyperparameterOptimizationOptions",struct("UseParallel",true) を fitcauto に渡します。
Mdl = fitcauto(carsTrain,"Origin");Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Learner types to explore: ensemble, knn, nb, net, svm, tree Total iterations (MaxObjectiveEvaluations): 180 Total time (MaxTime): Inf |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 1 | Best | 0.37179 | 0.62903 | 0.37179 | 0.37179 | svm | BoxConstraint: 0.11704 | | | | | | | | | KernelScale: 0.004903 | | 2 | Best | 0.22769 | 0.42586 | 0.22769 | 0.22769 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 3 | Best | 0.19231 | 0.42729 | 0.19231 | 0.19231 | knn | NumNeighbors: 3 | | 4 | Accept | 0.22769 | 0.1005 | 0.19231 | 0.19231 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 5 | Best | 0.1891 | 0.13361 | 0.1891 | 0.19096 | knn | NumNeighbors: 12 | | 6 | Best | 0.10154 | 0.28324 | 0.10154 | 0.10154 | tree | MinLeafSize: 5 | | 7 | Accept | 0.16026 | 7.2743 | 0.10154 | 0.10154 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.025856 | | | | | | | | | LayerSizes: [ 286 51 3 ] | | 8 | Accept | 0.37179 | 0.096727 | 0.10154 | 0.10154 | svm | BoxConstraint: 1.2607 | | | | | | | | | KernelScale: 97.75 | | 9 | Accept | 0.37179 | 0.25096 | 0.10154 | 0.10154 | net | Activations: relu | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 211.47 | | | | | | | | | LayerSizes: [ 102 222 ] | | 10 | Accept | 0.19231 | 0.070249 | 0.10154 | 0.10154 | knn | NumNeighbors: 15 | | 11 | Accept | 0.22769 | 0.067816 | 0.10154 | 0.10154 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 12 | Accept | 0.15077 | 9.2891 | 0.10154 | 0.10154 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 249 | | | | | | | | | MinLeafSize: 25 | | 13 | Accept | 0.22769 | 0.063862 | 0.10154 | 0.10154 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 14 | Accept | 0.37179 | 0.092315 | 0.10154 | 0.10154 | svm | BoxConstraint: 9.3148 | | | | | | | | | KernelScale: 0.0017736 | | 15 | Accept | 0.24615 | 0.64539 | 0.10154 | 0.10154 | nb | DistributionNames: kernel | | | | | | | | | Width: 1.2125 | | 16 | Accept | 0.12615 | 0.074982 | 0.10154 | 0.11409 | tree | MinLeafSize: 7 | | 17 | Accept | 0.16308 | 9.4331 | 0.10154 | 0.11409 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 284 | | | | | | | | | MinLeafSize: 89 | | 18 | Accept | 0.16923 | 0.062929 | 0.10154 | 0.13272 | tree | MinLeafSize: 81 | | 19 | Accept | 0.37179 | 0.096521 | 0.10154 | 0.13272 | svm | BoxConstraint: 1.6219 | | | | | | | | | KernelScale: 0.0011185 | | 20 | Accept | 0.25321 | 0.069954 | 0.10154 | 0.13272 | knn | NumNeighbors: 124 | |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 21 | Accept | 0.37179 | 0.081257 | 0.10154 | 0.13272 | svm | BoxConstraint: 0.0011787 | | | | | | | | | KernelScale: 1.1427 | | 22 | Accept | 0.22769 | 0.062348 | 0.10154 | 0.13272 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 23 | Accept | 0.13846 | 9.7413 | 0.10154 | 0.13272 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 279 | | | | | | | | | MinLeafSize: 2 | | 24 | Accept | 0.25231 | 0.19127 | 0.10154 | 0.13272 | nb | DistributionNames: kernel | | | | | | | | | Width: 1.6084 | | 25 | Accept | 0.22769 | 0.062537 | 0.10154 | 0.13272 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 26 | Accept | 0.19872 | 0.079713 | 0.10154 | 0.13272 | knn | NumNeighbors: 1 | | 27 | Accept | 0.23397 | 3.666 | 0.10154 | 0.13272 | net | Activations: tanh | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 1.1283e-06 | | | | | | | | | LayerSizes: [ 102 1 ] | | 28 | Accept | 0.13538 | 0.070656 | 0.10154 | 0.1338 | tree | MinLeafSize: 19 | | 29 | Accept | 0.19551 | 0.064085 | 0.10154 | 0.1338 | knn | NumNeighbors: 26 | | 30 | Accept | 0.37179 | 0.09049 | 0.10154 | 0.1338 | svm | BoxConstraint: 3.391 | | | | | | | | | KernelScale: 0.021864 | | 31 | Accept | 0.1891 | 2.712 | 0.10154 | 0.1338 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 1.0513e-06 | | | | | | | | | LayerSizes: [ 2 2 ] | | 32 | Accept | 0.21154 | 0.063578 | 0.10154 | 0.1338 | knn | NumNeighbors: 2 | | 33 | Accept | 0.10154 | 0.070667 | 0.10154 | 0.12751 | tree | MinLeafSize: 5 | | 34 | Accept | 0.37179 | 0.096094 | 0.10154 | 0.12751 | svm | BoxConstraint: 469.1 | | | | | | | | | KernelScale: 0.0089806 | | 35 | Accept | 0.14462 | 8.4121 | 0.10154 | 0.12751 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 241 | | | | | | | | | MinLeafSize: 1 | | 36 | Accept | 0.11385 | 0.066814 | 0.10154 | 0.11727 | tree | MinLeafSize: 11 | | 37 | Best | 0.098462 | 5.7806 | 0.098462 | 0.11727 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 218 | | | | | | | | | MinLeafSize: 48 | | 38 | Accept | 0.22769 | 0.061168 | 0.098462 | 0.11727 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 39 | Accept | 0.37179 | 0.44849 | 0.098462 | 0.11727 | net | Activations: tanh | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 29.705 | | | | | | | | | LayerSizes: 118 | | 40 | Accept | 0.24923 | 0.19982 | 0.098462 | 0.11727 | nb | DistributionNames: kernel | | | | | | | | | Width: 3.9774 | |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 41 | Accept | 0.18769 | 0.063134 | 0.098462 | 0.11494 | tree | MinLeafSize: 112 | | 42 | Accept | 0.10769 | 5.1672 | 0.098462 | 0.11494 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | MinLeafSize: 6 | | 43 | Accept | 0.17628 | 0.064608 | 0.098462 | 0.11494 | knn | NumNeighbors: 41 | | 44 | Accept | 0.37231 | 0.065944 | 0.098462 | 0.11788 | tree | MinLeafSize: 152 | | 45 | Accept | 0.22769 | 0.055901 | 0.098462 | 0.11788 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 46 | Accept | 0.37179 | 0.070447 | 0.098462 | 0.11788 | svm | BoxConstraint: 0.017639 | | | | | | | | | KernelScale: 1.8123 | | 47 | Accept | 0.37179 | 0.22159 | 0.098462 | 0.11788 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 2.6201 | | | | | | | | | LayerSizes: [ 134 10 240 ] | | 48 | Accept | 0.37179 | 0.16389 | 0.098462 | 0.11788 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.12107 | | | | | | | | | LayerSizes: [ 253 1 ] | | 49 | Accept | 0.13141 | 0.077293 | 0.098462 | 0.11788 | svm | BoxConstraint: 31.426 | | | | | | | | | KernelScale: 1.6379 | | 50 | Accept | 0.22769 | 0.055984 | 0.098462 | 0.11788 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 51 | Accept | 0.14769 | 9.3995 | 0.098462 | 0.11788 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 272 | | | | | | | | | MinLeafSize: 1 | | 52 | Accept | 0.12923 | 0.063907 | 0.098462 | 0.11796 | tree | MinLeafSize: 3 | | 53 | Accept | 0.37179 | 0.10353 | 0.098462 | 0.11796 | svm | BoxConstraint: 20.907 | | | | | | | | | KernelScale: 0.0030163 | | 54 | Accept | 0.15385 | 8.3016 | 0.098462 | 0.11796 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 243 | | | | | | | | | MinLeafSize: 18 | | 55 | Accept | 0.17628 | 0.061456 | 0.098462 | 0.11796 | knn | NumNeighbors: 41 | | 56 | Accept | 0.16667 | 0.7857 | 0.098462 | 0.11796 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.00041104 | | | | | | | | | LayerSizes: 2 | | 57 | Accept | 0.22769 | 0.058204 | 0.098462 | 0.11796 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 58 | Accept | 0.17949 | 4.0585 | 0.098462 | 0.11796 | net | Activations: sigmoid | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 3.7419e-06 | | | | | | | | | LayerSizes: [ 8 37 ] | | 59 | Accept | 0.23385 | 0.22143 | 0.098462 | 0.11796 | nb | DistributionNames: kernel | | | | | | | | | Width: 561.16 | | 60 | Accept | 0.19551 | 0.06451 | 0.098462 | 0.11796 | knn | NumNeighbors: 26 | |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 61 | Accept | 0.37231 | 6.2084 | 0.098462 | 0.11796 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 272 | | | | | | | | | MinLeafSize: 161 | | 62 | Accept | 0.37179 | 0.090047 | 0.098462 | 0.11796 | svm | BoxConstraint: 29.741 | | | | | | | | | KernelScale: 0.045927 | | 63 | Accept | 0.20513 | 6.8138 | 0.098462 | 0.11796 | net | Activations: sigmoid | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 5.1269e-08 | | | | | | | | | LayerSizes: [ 175 2 4 ] | | 64 | Accept | 0.1891 | 0.05843 | 0.098462 | 0.11796 | knn | NumNeighbors: 9 | | 65 | Accept | 0.19231 | 0.067392 | 0.098462 | 0.11796 | knn | NumNeighbors: 15 | | 66 | Accept | 0.12923 | 0.075809 | 0.098462 | 0.11512 | tree | MinLeafSize: 3 | | 67 | Accept | 0.37179 | 0.073588 | 0.098462 | 0.11512 | svm | BoxConstraint: 0.018753 | | | | | | | | | KernelScale: 0.38262 | | 68 | Accept | 0.17308 | 0.06212 | 0.098462 | 0.11512 | knn | NumNeighbors: 4 | | 69 | Accept | 0.14769 | 0.074561 | 0.098462 | 0.11688 | tree | MinLeafSize: 2 | | 70 | Accept | 0.13538 | 7.9953 | 0.098462 | 0.11674 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 222 | | | | | | | | | MinLeafSize: 1 | | 71 | Accept | 0.13538 | 0.062477 | 0.098462 | 0.11674 | tree | MinLeafSize: 19 | | 72 | Accept | 0.12923 | 0.063529 | 0.098462 | 0.11674 | tree | MinLeafSize: 8 | | 73 | Accept | 0.15064 | 1.8048 | 0.098462 | 0.11674 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.00059411 | | | | | | | | | LayerSizes: [ 5 4 ] | | 74 | Accept | 0.22769 | 0.058621 | 0.098462 | 0.11674 | nb | DistributionNames: normal | | | | | | | | | Width: NaN | | 75 | Accept | 0.16987 | 5.3533 | 0.098462 | 0.11674 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.00022289 | | | | | | | | | LayerSizes: [ 81 9 ] | | 76 | Accept | 0.14154 | 7.4446 | 0.098462 | 0.11813 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 214 | | | | | | | | | MinLeafSize: 7 | | 77 | Accept | 0.33846 | 0.059384 | 0.098462 | 0.1167 | tree | MinLeafSize: 130 | | 78 | Accept | 0.13231 | 4.9246 | 0.098462 | 0.1167 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 212 | | | | | | | | | MinLeafSize: 64 | | 79 | Accept | 0.37179 | 0.12859 | 0.098462 | 0.1167 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 5.8307 | | | | | | | | | LayerSizes: [ 20 1 ] | | 80 | Accept | 0.14154 | 7.5942 | 0.098462 | 0.1167 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 219 | | | | | | | | | MinLeafSize: 7 | |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 81 | Accept | 0.19872 | 0.063107 | 0.098462 | 0.1167 | knn | NumNeighbors: 1 | | 82 | Accept | 0.37179 | 0.14405 | 0.098462 | 0.1167 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 15.587 | | | | | | | | | LayerSizes: [ 182 4 ] | | 83 | Accept | 0.37179 | 0.3951 | 0.098462 | 0.1167 | net | Activations: none | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 0.00026401 | | | | | | | | | LayerSizes: [ 1 79 ] | | 84 | Accept | 0.14154 | 6.1198 | 0.098462 | 0.1167 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 266 | | | | | | | | | MinLeafSize: 110 | | 85 | Accept | 0.14423 | 0.10373 | 0.098462 | 0.1167 | svm | BoxConstraint: 322.43 | | | | | | | | | KernelScale: 1.7393 | | 86 | Accept | 0.12 | 0.073943 | 0.098462 | 0.11425 | tree | MinLeafSize: 4 | | 87 | Accept | 0.37179 | 0.089384 | 0.098462 | 0.11425 | svm | BoxConstraint: 0.0026322 | | | | | | | | | KernelScale: 0.004006 | | 88 | Accept | 0.14154 | 9.5099 | 0.098462 | 0.11425 | ensemble | Method: Bag | | | | | | | | | NumLearningCycles: 276 | | | | | | | | | MinLeafSize: 2 | | 89 | Accept | 0.37179 | 0.088251 | 0.098462 | 0.11425 | svm | BoxConstraint: 25.201 | | | | | | | | | KernelScale: 0.019423 | | 90 | Accept | 0.1891 | 0.072839 | 0.098462 | 0.11425 | knn | NumNeighbors: 13 | | 91 | Accept | 0.12615 | 4.8775 | 0.098462 | 0.11425 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 211 | | | | | | | | | MinLeafSize: 75 | | 92 | Accept | 0.14154 | 4.9062 | 0.098462 | 0.11425 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | MinLeafSize: 96 | | 93 | Accept | 0.11077 | 6.4978 | 0.098462 | 0.11425 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 277 | | | | | | | | | MinLeafSize: 4 | | 94 | Accept | 0.10154 | 6.4944 | 0.098462 | 0.11048 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 274 | | | | | | | | | MinLeafSize: 16 | | 95 | Accept | 0.12615 | 6.6536 | 0.098462 | 0.11217 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 289 | | | | | | | | | MinLeafSize: 53 | | 96 | Accept | 0.14462 | 4.5605 | 0.098462 | 0.1093 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 201 | | | | | | | | | MinLeafSize: 91 | | 97 | Best | 0.089231 | 5.7135 | 0.089231 | 0.10371 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 242 | | | | | | | | | MinLeafSize: 13 | | 98 | Accept | 0.10769 | 5.5323 | 0.089231 | 0.10353 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 233 | | | | | | | | | MinLeafSize: 10 | | 99 | Accept | 0.12 | 5.9044 | 0.089231 | 0.10351 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 253 | | | | | | | | | MinLeafSize: 26 | | 100 | Accept | 0.10154 | 6.4575 | 0.089231 | 0.10122 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 272 | | | | | | | | | MinLeafSize: 13 | |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 101 | Accept | 0.16026 | 7.4685 | 0.089231 | 0.10122 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.020619 | | | | | | | | | LayerSizes: [ 117 160 2 ] | | 102 | Accept | 0.20513 | 3.5845 | 0.089231 | 0.10122 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.052211 | | | | | | | | | LayerSizes: [ 18 182 163 ] | | 103 | Best | 0.086154 | 4.9401 | 0.086154 | 0.095252 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 208 | | | | | | | | | MinLeafSize: 15 | | 104 | Accept | 0.095385 | 6.4925 | 0.086154 | 0.096118 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 274 | | | | | | | | | MinLeafSize: 14 | | 105 | Accept | 0.092308 | 4.8125 | 0.086154 | 0.093255 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 201 | | | | | | | | | MinLeafSize: 14 | | 106 | Accept | 0.37231 | 4.7781 | 0.086154 | 0.092615 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 207 | | | | | | | | | MinLeafSize: 134 | | 107 | Accept | 0.14462 | 4.7742 | 0.086154 | 0.097454 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 206 | | | | | | | | | MinLeafSize: 93 | | 108 | Accept | 0.092308 | 4.9755 | 0.086154 | 0.092405 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 206 | | | | | | | | | MinLeafSize: 14 | | 109 | Accept | 0.092308 | 4.9658 | 0.086154 | 0.091949 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 208 | | | | | | | | | MinLeafSize: 20 | | 110 | Accept | 0.10154 | 5.0332 | 0.086154 | 0.092013 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 207 | | | | | | | | | MinLeafSize: 16 | | 111 | Accept | 0.17846 | 4.6286 | 0.086154 | 0.092219 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 201 | | | | | | | | | MinLeafSize: 120 | | 112 | Accept | 0.18462 | 5.3291 | 0.086154 | 0.092663 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 234 | | | | | | | | | MinLeafSize: 114 | | 113 | Accept | 0.10154 | 4.9436 | 0.086154 | 0.091972 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 210 | | | | | | | | | MinLeafSize: 21 | | 114 | Accept | 0.10154 | 4.8767 | 0.086154 | 0.092793 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 208 | | | | | | | | | MinLeafSize: 23 | | 115 | Accept | 0.10769 | 4.8384 | 0.086154 | 0.09189 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 203 | | | | | | | | | MinLeafSize: 27 | | 116 | Accept | 0.089231 | 5.1892 | 0.086154 | 0.091881 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 216 | | | | | | | | | MinLeafSize: 18 | | 117 | Accept | 0.095385 | 5.2157 | 0.086154 | 0.092387 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 220 | | | | | | | | | MinLeafSize: 15 | | 118 | Accept | 0.095385 | 5.0879 | 0.086154 | 0.092544 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | MinLeafSize: 15 | | 119 | Accept | 0.10154 | 5.6188 | 0.086154 | 0.092332 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 235 | | | | | | | | | MinLeafSize: 17 | | 120 | Accept | 0.37179 | 0.197 | 0.086154 | 0.092332 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.063953 | | | | | | | | | LayerSizes: [ 220 3 ] | |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 121 | Accept | 0.37179 | 0.1961 | 0.086154 | 0.092332 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.055033 | | | | | | | | | LayerSizes: [ 97 13 ] | | 122 | Accept | 0.14103 | 1.4229 | 0.086154 | 0.092332 | net | Activations: none | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.00085152 | | | | | | | | | LayerSizes: [ 197 20 2 ] | | 123 | Accept | 0.37179 | 0.22892 | 0.086154 | 0.092332 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.051445 | | | | | | | | | LayerSizes: [ 247 6 ] | | 124 | Accept | 0.13782 | 0.38245 | 0.086154 | 0.092332 | net | Activations: none | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0087893 | | | | | | | | | LayerSizes: [ 199 2 ] | | 125 | Accept | 0.26282 | 1.0954 | 0.086154 | 0.092332 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.016624 | | | | | | | | | LayerSizes: [ 115 10 ] | | 126 | Accept | 0.18269 | 0.24182 | 0.086154 | 0.092332 | net | Activations: none | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.035571 | | | | | | | | | LayerSizes: [ 224 9 ] | | 127 | Accept | 0.095385 | 5.0171 | 0.086154 | 0.091895 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 208 | | | | | | | | | MinLeafSize: 2 | | 128 | Accept | 0.37179 | 0.13063 | 0.086154 | 0.091895 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0424 | | | | | | | | | LayerSizes: [ 4 2 3 ] | | 129 | Accept | 0.14103 | 2.0229 | 0.086154 | 0.091895 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0028328 | | | | | | | | | LayerSizes: [ 1 8 1 ] | | 130 | Accept | 0.18269 | 1.0404 | 0.086154 | 0.091895 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.037003 | | | | | | | | | LayerSizes: [ 153 3 ] | | 131 | Accept | 0.14423 | 2.7718 | 0.086154 | 0.091895 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0058072 | | | | | | | | | LayerSizes: [ 1 72 ] | | 132 | Accept | 0.14103 | 9.7891 | 0.086154 | 0.091895 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0040232 | | | | | | | | | LayerSizes: [ 1 88 57 ] | | 133 | Accept | 0.14103 | 0.44563 | 0.086154 | 0.091895 | net | Activations: none | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0020791 | | | | | | | | | LayerSizes: [ 3 5 2 ] | | 134 | Accept | 0.37179 | 0.13512 | 0.086154 | 0.091895 | net | Activations: relu | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 0.027533 | | | | | | | | | LayerSizes: [ 2 2 1 ] | | 135 | Accept | 0.18269 | 1.6723 | 0.086154 | 0.091895 | net | Activations: relu | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0042357 | | | | | | | | | LayerSizes: [ 5 3 1 ] | | 136 | Accept | 0.23397 | 3.4642 | 0.086154 | 0.091895 | net | Activations: tanh | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 0.00033175 | | | | | | | | | LayerSizes: [ 1 26 1 ] | | 137 | Accept | 0.21154 | 3.0298 | 0.086154 | 0.091895 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.030788 | | | | | | | | | LayerSizes: [ 1 130 48 ] | | 138 | Accept | 0.086154 | 5.109 | 0.086154 | 0.091384 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 210 | | | | | | | | | MinLeafSize: 15 | | 139 | Accept | 0.14103 | 5.9512 | 0.086154 | 0.091384 | net | Activations: tanh | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0029983 | | | | | | | | | LayerSizes: [ 1 11 104 ] | | 140 | Accept | 0.14744 | 3.0068 | 0.086154 | 0.091384 | net | Activations: relu | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.007429 | | | | | | | | | LayerSizes: [ 1 102 25 ] | |=============================================================================================================================================| | Iter | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | result | loss | & validation (sec)| validation loss | validation loss | | | |=============================================================================================================================================| | 141 | Accept | 0.22436 | 2.6641 | 0.086154 | 0.091384 | net | Activations: none | | | | | | | | | Standardize: false | | | | | | | | | Lambda: 0.001718 | | | | | | | | | LayerSizes: [ 3 7 125 ] | | 142 | Accept | 0.089231 | 5.0047 | 0.086154 | 0.089265 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 209 | | | | | | | | | MinLeafSize: 15 | | 143 | Accept | 0.14154 | 5.1754 | 0.086154 | 0.0895 | ensemble | Method: LogitBoost | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | MinLeafSize: 100 | | 144 | Accept | 0.37179 | 0.1594 | 0.086154 | 0.0895 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.011755 | | | | | | | | | LayerSizes: [ 5 1 2 ] | | 145 | Accept | 0.37179 | 0.37807 | 0.086154 | 0.0895 | net | Activations: sigmoid | | | | | | | | | Standardize: true | | | | | | | | | Lambda: 0.0033755 | | | | | | | | | Lay...
__________________________________________________________ Optimization completed. Total iterations: 180 Total elapsed time: 699.614 seconds Total time for training and validation: 493.3351 seconds Best observed learner is an ensemble model with: Learner: ensemble Method: LogitBoost NumLearningCycles: 208 MinLeafSize: 15 Observed validation loss: 0.086154 Time for training and validation: 4.9401 seconds Best estimated learner (returned model) is an ensemble model with: Learner: ensemble Method: LogitBoost NumLearningCycles: 209 MinLeafSize: 15 Estimated validation loss: 0.089192 Estimated time for training and validation: 5.0288 seconds Documentation for fitcauto display
fitcauto によって返される最終的なモデルが、最適な推定学習器となります。モデルを返す前に、関数は学習データ全体 (carsTrain)、リストされている Learner (またはモデル) のタイプ、および表示されたハイパーパラメーター値を使用して、モデルの再学習を行います。
テスト セットのパフォーマンスの評価
テスト セットに対するモデルのパフォーマンスを評価します。
testAccuracy = 1 - loss(Mdl,carsTest,"Origin")testAccuracy = 0.9263
confusionchart(carsTest.Origin,predict(Mdl,carsTest))
行列データを使用した分類器の自動選択
fitcauto を使用し、個別の変数で指定した予測子データと応答データに基づいて、最適化されたハイパーパラメーターをもつ分類モデルを自動的に選択します。
データの読み込み
humanactivity データセットを読み込みます。このデータセットには、人間の次の 5 種類の身体動作についての 24,075 個の観測値が含まれています。座る (1)、立つ (2)、歩く (3)、走る (4)、踊る (5)。各観測値には、スマートフォンの加速度センサーによって測定された加速度データから抽出した 60 個の特徴量が含まれています。変数 feat には、24,075 個の観測値に関する 60 個の特徴量から成る予測子データ行列が格納され、応答変数 actid には、観測値の身体動作 ID が整数として格納されています。
load humanactivityデータの分割
データを学習セットとテスト セットに分割します。モデルの選択に観測値の 90% を使用し、fitcauto によって返された最終モデルの検証に観測値の 10% を使用します。cvpartition を使用して、観測値の 10% をテスト用に確保します。
rng("default") % For reproducibility of the partition c = cvpartition(actid,"Holdout",0.10); trainingIndices = training(c); % Indices for the training set XTrain = feat(trainingIndices,:); YTrain = actid(trainingIndices); testIndices = test(c); % Indices for the test set XTest = feat(testIndices,:); YTest = actid(testIndices);
fitcauto の実行
学習データを fitcauto に渡します。学習データ XTrain に含まれる観測値が 10,000 を超えるため、ベイズ最適化ではなく ASHA 最適化を使用します。関数 fitcauto は、さまざまなハイパーパラメーターの値をもつ適切なモデル (または学習器) のタイプを無作為に選択し、学習データの小さいサブセットでこのモデルに学習させ、適切に機能するモデルをプロモートして、プロモートされたモデルを徐々に大きくした学習データのセットで再学習させます。この関数は、最適な交差検証性能をもち、すべての学習データで再学習済みのモデル、および最適化の詳細を含む table を返します。最適化を並列実行するよう指定します (Parallel Computing Toolbox™ が必要)。
既定の設定では、fitcauto は、最適化のプロット、および最適化の結果の反復表示を提供します。これらの結果を解釈する方法の詳細については、Verbose の表示を参照してください。
options = struct("Optimizer","asha","UseParallel",true); [Mdl,OptimizationResults] = fitcauto(XTrain,YTrain,"HyperparameterOptimizationOptions",options);
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Starting parallel pool (parpool) using the 'local' profile ... Connected to the parallel pool (number of workers: 8). Copying objective function to workers... Done copying objective function to workers. Learner types to explore: ensemble, knn, nb, net, svm, tree Total iterations (MaxObjectiveEvaluations): 595 Total time (MaxTime): Inf |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 1 | 8 | Best | 0.74165 | 2.2322 | 0.74165 | 271 | tree | MinLeafSize: 945 | | 2 | 7 | Accept | 0.74165 | 9.0692 | 0.049289 | 271 | knn | NumNeighbors: 1726 | | 3 | 7 | Best | 0.049289 | 3.3616 | 0.049289 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 4 | 7 | Accept | 0.74165 | 9.2877 | 0.049289 | 271 | knn | NumNeighbors: 3072 | | 5 | 8 | Best | 0.046566 | 0.81486 | 0.046566 | 1084 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 6 | 8 | Accept | 0.13379 | 3.0947 | 0.046566 | 271 | knn | NumNeighbors: 46 | | 7 | 8 | Accept | 0.066457 | 13.692 | 0.046566 | 271 | net | Activations: sigmoid | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.927e-08 | | | | | | | | | | LayerSizes: [ 10 56 28 ] | | 8 | 8 | Accept | 0.096225 | 3.801 | 0.046566 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 8.8825 | | | | | | | | | | KernelScale: 73.89 | | 9 | 7 | Accept | 0.73962 | 14.971 | 0.046566 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 17.562 | | | | | | | | | | KernelScale: 0.0082394 | | 10 | 7 | Accept | 0.73925 | 14.98 | 0.046566 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 1.3419 | | | | | | | | | | KernelScale: 0.027033 | | 11 | 8 | Accept | 0.74165 | 5.7328 | 0.046566 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.040932 | | | | | | | | | | KernelScale: 949.09 | | 12 | 8 | Best | 0.04472 | 3.1496 | 0.04472 | 271 | knn | NumNeighbors: 2 | | 13 | 8 | Accept | 0.74165 | 6.4341 | 0.04472 | 271 | knn | NumNeighbors: 1240 | | 14 | 8 | Best | 0.041536 | 2.2078 | 0.041536 | 271 | knn | NumNeighbors: 3 | | 15 | 8 | Accept | 0.051828 | 2.211 | 0.041536 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 16 | 8 | Accept | 0.74165 | 12.471 | 0.041536 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0079065 | | | | | | | | | | KernelScale: 0.039442 | | 17 | 8 | Accept | 0.74165 | 0.88796 | 0.041536 | 271 | tree | MinLeafSize: 252 | | 18 | 8 | Best | 0.029629 | 6.9518 | 0.029629 | 1084 | knn | NumNeighbors: 2 | | 19 | 8 | Accept | 0.030044 | 6.2852 | 0.029629 | 1084 | knn | NumNeighbors: 3 | | 20 | 8 | Accept | 0.74165 | 6.2132 | 0.029629 | 271 | knn | NumNeighbors: 8117 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 21 | 8 | Accept | 0.1811 | 2.2815 | 0.029629 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.0052814 | | | | | | | | | | KernelScale: 546.04 | | 22 | 8 | Accept | 0.34996 | 4.8703 | 0.029629 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.0486 | | | | | | | | | | KernelScale: 2.323 | | 23 | 8 | Accept | 0.74165 | 5.7248 | 0.029629 | 271 | knn | NumNeighbors: 524 | | 24 | 8 | Accept | 0.046243 | 1.9627 | 0.029629 | 1084 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 25 | 7 | Accept | 0.73057 | 53.598 | 0.029629 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.0633e-13 | | 26 | 7 | Accept | 0.04209 | 38.18 | 0.029629 | 1084 | net | Activations: sigmoid | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.927e-08 | | | | | | | | | | LayerSizes: [ 10 56 28 ] | | 27 | 8 | Accept | 0.035998 | 31.431 | 0.029629 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.4417e-06 | | | | | | | | | | LayerSizes: [ 202 11 ] | | 28 | 8 | Accept | 0.082103 | 2.5818 | 0.029629 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 1.0839 | | | | | | | | | | KernelScale: 39.504 | | 29 | 8 | Accept | 0.032721 | 20.812 | 0.029629 | 271 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 257 | | | | | | | | | | MinLeafSize: 4 | | | | | | | | | | MaxNumSplits: 13 | | 30 | 8 | Accept | 0.74165 | 7.0246 | 0.029629 | 271 | knn | NumNeighbors: 384 | | 31 | 8 | Accept | 0.73034 | 49.099 | 0.029629 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 2.0942e-12 | | 32 | 8 | Accept | 0.052843 | 1.4947 | 0.029629 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 33 | 8 | Best | 0.019522 | 19.771 | 0.019522 | 4334 | knn | NumNeighbors: 2 | | 34 | 8 | Accept | 0.74072 | 10.11 | 0.019522 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.44626 | | | | | | | | | | KernelScale: 0.089894 | | 35 | 8 | Accept | 0.10218 | 4.6984 | 0.019522 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 2.8638 | | | | | | | | | | KernelScale: 201.68 | | 36 | 8 | Accept | 0.71774 | 68.103 | 0.019522 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.00032006 | | 37 | 8 | Accept | 0.6913 | 5.0575 | 0.019522 | 271 | knn | NumNeighbors: 184 | | 38 | 8 | Accept | 0.10689 | 2.2859 | 0.019522 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.036995 | | | | | | | | | | KernelScale: 13.878 | | 39 | 8 | Accept | 0.72983 | 56.397 | 0.019522 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 2.3638e-05 | | 40 | 8 | Accept | 0.035121 | 5.8369 | 0.019522 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 2.7559e-07 | | | | | | | | | | LayerSizes: [ 32 93 ] | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 41 | 8 | Accept | 0.048459 | 0.72565 | 0.019522 | 1084 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 42 | 8 | Accept | 0.045136 | 0.89703 | 0.019522 | 271 | tree | MinLeafSize: 6 | | 43 | 8 | Accept | 0.11427 | 20.922 | 0.019522 | 271 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 240 | | | | | | | | | | MinLeafSize: 49 | | | | | | | | | | MaxNumSplits: 97 | | 44 | 8 | Accept | 0.041674 | 2.8829 | 0.019522 | 271 | knn | NumNeighbors: 2 | | 45 | 8 | Accept | 0.74165 | 5.746 | 0.019522 | 271 | knn | NumNeighbors: 4410 | | 46 | 8 | Accept | 0.019799 | 3.4571 | 0.019522 | 1084 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 2.7559e-07 | | | | | | | | | | LayerSizes: [ 32 93 ] | | 47 | 8 | Accept | 0.030414 | 23.366 | 0.019522 | 1084 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 257 | | | | | | | | | | MinLeafSize: 4 | | | | | | | | | | MaxNumSplits: 13 | | 48 | 8 | Accept | 0.049243 | 2.1016 | 0.019522 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 49 | 8 | Accept | 0.041767 | 0.83736 | 0.019522 | 271 | tree | MinLeafSize: 3 | | 50 | 8 | Accept | 0.032813 | 4.5548 | 0.019522 | 271 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 223 | | | | | | | | | | MinLeafSize: 1 | | | | | | | | | | MaxNumSplits: 75 | | 51 | 8 | Accept | 0.7362 | 11.413 | 0.019522 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 760.57 | | | | | | | | | | KernelScale: 0.34067 | | 52 | 8 | Accept | 0.055843 | 0.7704 | 0.019522 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 53 | 8 | Accept | 0.028937 | 6.754 | 0.019522 | 1084 | knn | NumNeighbors: 2 | | 54 | 8 | Accept | 0.74165 | 5.6993 | 0.019522 | 271 | knn | NumNeighbors: 9124 | | 55 | 8 | Accept | 0.054689 | 0.75579 | 0.019522 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 56 | 8 | Accept | 0.019799 | 8.4671 | 0.019522 | 1084 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 223 | | | | | | | | | | MinLeafSize: 1 | | | | | | | | | | MaxNumSplits: 75 | | 57 | 8 | Best | 0.010338 | 22.501 | 0.010338 | 4334 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 2.7559e-07 | | | | | | | | | | LayerSizes: [ 32 93 ] | | 58 | 8 | Accept | 0.12599 | 2.4672 | 0.010338 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.41895 | | | | | | | | | | KernelScale: 96.491 | | 59 | 8 | Accept | 0.70357 | 60.43 | 0.010338 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.00045097 | | 60 | 8 | Accept | 0.5737 | 73.531 | 0.010338 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.0018132 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 61 | 8 | Accept | 0.74165 | 2.1874 | 0.010338 | 271 | net | Activations: sigmoid | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 0.019205 | | | | | | | | | | LayerSizes: [ 3 3 ] | | 62 | 8 | Accept | 0.050212 | 1.0089 | 0.010338 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 63 | 8 | Accept | 0.027506 | 77.499 | 0.010338 | 1084 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.4417e-06 | | | | | | | | | | LayerSizes: [ 202 11 ] | | 64 | 8 | Accept | 0.030137 | 2.1967 | 0.010338 | 1084 | tree | MinLeafSize: 3 | | 65 | 8 | Accept | 0.74165 | 6.9907 | 0.010338 | 271 | knn | NumNeighbors: 7694 | | 66 | 8 | Accept | 0.73025 | 46.328 | 0.010338 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 7.594e-10 | | 67 | 8 | Accept | 0.71742 | 63.01 | 0.010338 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.00036683 | | 68 | 8 | Accept | 0.029906 | 1.3903 | 0.010338 | 1084 | tree | MinLeafSize: 6 | | 69 | 8 | Accept | 0.090502 | 5.9339 | 0.010338 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 2.6095e-08 | | | | | | | | | | LayerSizes: [ 3 16 4 ] | | 70 | 8 | Accept | 0.061381 | 2.8978 | 0.010338 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.66386 | | | | | | | | | | KernelScale: 11.66 | | 71 | 8 | Accept | 0.037705 | 19.775 | 0.010338 | 271 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 258 | | | | | | | | | | MinLeafSize: 11 | | | | | | | | | | MaxNumSplits: 10 | | 72 | 8 | Accept | 0.03849 | 19.504 | 0.010338 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 6.8643e-05 | | | | | | | | | | LayerSizes: [ 24 101 14 ] | | 73 | 8 | Accept | 0.037152 | 20.796 | 0.010338 | 271 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 275 | | | | | | | | | | MinLeafSize: 2 | | | | | | | | | | MaxNumSplits: 27 | | 74 | 8 | Accept | 0.048689 | 2.1741 | 0.010338 | 271 | knn | NumNeighbors: 4 | | 75 | 8 | Accept | 0.55335 | 4.3695 | 0.010338 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.011209 | | | | | | | | | | KernelScale: 1.0514 | | 76 | 8 | Accept | 0.032352 | 22.88 | 0.010338 | 1084 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 258 | | | | | | | | | | MinLeafSize: 11 | | | | | | | | | | MaxNumSplits: 10 | | 77 | 8 | Best | 0.010061 | 42.264 | 0.010061 | 4334 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 223 | | | | | | | | | | MinLeafSize: 1 | | | | | | | | | | MaxNumSplits: 75 | | 78 | 8 | Accept | 0.13125 | 5.0693 | 0.010061 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.5998e-06 | | | | | | | | | | LayerSizes: [ 1 3 ] | | 79 | 8 | Accept | 0.74165 | 14.599 | 0.010061 | 271 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 216 | | | | | | | | | | MinLeafSize: 421 | | | | | | | | | | MaxNumSplits: 59 | | 80 | 8 | Accept | 0.73131 | 46.536 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.0852e-11 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 81 | 8 | Accept | 0.026121 | 27.416 | 0.010061 | 1084 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 275 | | | | | | | | | | MinLeafSize: 2 | | | | | | | | | | MaxNumSplits: 27 | | 82 | 8 | Accept | 0.07818 | 2.9308 | 0.010061 | 271 | knn | NumNeighbors: 15 | | 83 | 8 | Accept | 0.64501 | 150.1 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 422.98 | | 84 | 8 | Accept | 0.048966 | 0.97418 | 0.010061 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 85 | 8 | Accept | 0.74165 | 5.9347 | 0.010061 | 271 | knn | NumNeighbors: 2792 | | 86 | 8 | Accept | 0.050858 | 0.89733 | 0.010061 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 87 | 8 | Accept | 0.035444 | 5.8934 | 0.010061 | 1084 | knn | NumNeighbors: 4 | | 88 | 8 | Accept | 0.74165 | 156.96 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1679.5 | | 89 | 8 | Accept | 0.043982 | 1.8291 | 0.010061 | 271 | tree | MinLeafSize: 3 | | 90 | 8 | Accept | 0.080857 | 144.55 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.87308 | | 91 | 8 | Accept | 0.72956 | 47.076 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 2.7893e-08 | | 92 | 8 | Accept | 0.029721 | 1.4922 | 0.010061 | 1084 | tree | MinLeafSize: 3 | | 93 | 8 | Accept | 0.74165 | 0.12917 | 0.010061 | 271 | tree | MinLeafSize: 7511 | | 94 | 7 | Accept | 0.74165 | 6.1437 | 0.010061 | 271 | knn | NumNeighbors: 9834 | | 95 | 7 | Accept | 0.59359 | 73.466 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.0015851 | | 96 | 8 | Accept | 0.040336 | 0.69244 | 0.010061 | 271 | tree | MinLeafSize: 1 | | 97 | 7 | Accept | 0.042874 | 2.1452 | 0.010061 | 271 | net | Activations: relu | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 3.2315e-08 | | | | | | | | | | LayerSizes: 3 | | 98 | 7 | Accept | 0.74165 | 1.2052 | 0.010061 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 0.3822 | | | | | | | | | | LayerSizes: [ 27 9 ] | | 99 | 8 | Accept | 0.026121 | 1.2028 | 0.010061 | 1084 | tree | MinLeafSize: 1 | | 100 | 8 | Accept | 0.74165 | 0.66984 | 0.010061 | 271 | tree | MinLeafSize: 1551 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 101 | 8 | Accept | 0.74165 | 10.343 | 0.010061 | 271 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 223 | | | | | | | | | | MinLeafSize: 9461 | | | | | | | | | | MaxNumSplits: 11 | | 102 | 8 | Accept | 0.051505 | 0.82579 | 0.010061 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 103 | 8 | Accept | 0.028244 | 46.019 | 0.010061 | 1084 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 6.8643e-05 | | | | | | | | | | LayerSizes: [ 24 101 14 ] | | 104 | 8 | Accept | 0.05612 | 1.9299 | 0.010061 | 271 | knn | NumNeighbors: 5 | | 105 | 8 | Accept | 0.73006 | 44.245 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.203e-13 | | 106 | 7 | Accept | 0.74165 | 6.0627 | 0.010061 | 271 | knn | NumNeighbors: 323 | | 107 | 7 | Accept | 0.041167 | 3.0942 | 0.010061 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 7.2382e-05 | | | | | | | | | | LayerSizes: [ 3 2 ] | | 108 | 8 | Accept | 0.66841 | 4.8715 | 0.010061 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 8.4445 | | | | | | | | | | KernelScale: 0.70923 | | 109 | 8 | Accept | 0.027321 | 5.5757 | 0.010061 | 1084 | net | Activations: relu | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 3.2315e-08 | | | | | | | | | | LayerSizes: 3 | | 110 | 8 | Accept | 0.1343 | 4.1806 | 0.010061 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 2.4202 | | | | | | | | | | KernelScale: 3.4213 | | 111 | 8 | Accept | 0.055612 | 0.90595 | 0.010061 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 112 | 8 | Accept | 0.019153 | 1.3107 | 0.010061 | 4334 | tree | MinLeafSize: 1 | | 113 | 8 | Accept | 0.048597 | 0.70833 | 0.010061 | 1084 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 114 | 8 | Accept | 0.59175 | 0.84241 | 0.010061 | 271 | net | Activations: sigmoid | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.0815 | | | | | | | | | | LayerSizes: 150 | | 115 | 8 | Accept | 0.54006 | 4.17 | 0.010061 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.029391 | | | | | | | | | | KernelScale: 1.121 | | 116 | 8 | Accept | 0.74165 | 6.5172 | 0.010061 | 271 | knn | NumNeighbors: 2873 | | 117 | 8 | Accept | 0.024091 | 7.1168 | 0.010061 | 1084 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 7.2382e-05 | | | | | | | | | | LayerSizes: [ 3 2 ] | | 118 | 8 | Accept | 0.027368 | 10.135 | 0.010061 | 271 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 216 | | | | | | | | | | MinLeafSize: 2 | | | | | | | | | | MaxNumSplits: 42 | | 119 | 8 | Accept | 0.027598 | 46.175 | 0.010061 | 4334 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 275 | | | | | | | | | | MinLeafSize: 2 | | | | | | | | | | MaxNumSplits: 27 | | 120 | 8 | Accept | 0.12701 | 3.4389 | 0.010061 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 19.581 | | | | | | | | | | KernelScale: 2.5698 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 121 | 8 | Accept | 0.74165 | 5.7497 | 0.010061 | 271 | knn | NumNeighbors: 3004 | | 122 | 8 | Accept | 0.051551 | 0.98458 | 0.010061 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 123 | 8 | Accept | 0.73934 | 10.379 | 0.010061 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 6.66 | | | | | | | | | | KernelScale: 0.0075876 | | 124 | 8 | Accept | 0.74165 | 6.0464 | 0.010061 | 271 | knn | NumNeighbors: 538 | | 125 | 8 | Accept | 0.051643 | 0.94588 | 0.010061 | 1084 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 126 | 8 | Accept | 0.050074 | 0.66389 | 0.010061 | 271 | tree | MinLeafSize: 8 | | 127 | 8 | Accept | 0.51948 | 11.015 | 0.010061 | 271 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 245 | | | | | | | | | | MinLeafSize: 103 | | | | | | | | | | MaxNumSplits: 30 | | 128 | 8 | Accept | 0.054966 | 0.59455 | 0.010061 | 271 | tree | MinLeafSize: 19 | | 129 | 8 | Accept | 0.068765 | 2.1334 | 0.010061 | 271 | knn | NumNeighbors: 12 | | 130 | 8 | Accept | 0.031613 | 1.5598 | 0.010061 | 1084 | tree | MinLeafSize: 8 | | 131 | 8 | Accept | 0.018414 | 20.732 | 0.010061 | 1084 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 216 | | | | | | | | | | MinLeafSize: 2 | | | | | | | | | | MaxNumSplits: 42 | | 132 | 8 | Accept | 0.73874 | 9.856 | 0.010061 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.37029 | | | | | | | | | | KernelScale: 0.28264 | | 133 | 8 | Accept | 0.044813 | 1.4812 | 0.010061 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 187.62 | | | | | | | | | | KernelScale: 55.462 | | 134 | 8 | Accept | 0.74165 | 0.39159 | 0.010061 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.3833 | | | | | | | | | | LayerSizes: [ 7 3 ] | | 135 | 8 | Accept | 0.69134 | 7.5927 | 0.010061 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.90862 | | | | | | | | | | KernelScale: 1.0317 | | 136 | 8 | Accept | 0.022568 | 2.2974 | 0.010061 | 1084 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 187.62 | | | | | | | | | | KernelScale: 55.462 | | 137 | 8 | Accept | 0.029352 | 19.821 | 0.010061 | 271 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 268 | | | | | | | | | | MinLeafSize: 1 | | | | | | | | | | MaxNumSplits: 79 | | 138 | 8 | Accept | 0.74165 | 5.6724 | 0.010061 | 271 | knn | NumNeighbors: 4444 | | 139 | 8 | Accept | 0.056443 | 0.6542 | 0.010061 | 271 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 140 | 8 | Accept | 0.74165 | 8.0385 | 0.010061 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0043989 | | | | | | | | | | KernelScale: 0.36557 | |====================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Training set | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | size | | | |====================================================================================================================================================| | 141 | 8 | Accept | 0.73066 | 46.903 | 0.010061 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 1.1477e-12 | | 142 | 8 | Accept | 0.015784 | 31.487 | 0.010061 | 4334 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 7.2382e-05 | | | | | | | | | | LayerSizes: [ 3 2 ] | | 143 | 8 | Accept | 0.028567 | 13.713 | 0.010061 | 271 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 213 | | | | | | | | | | MinLeafSize: 7 | | | | | | | | | | MaxNumSplits: 82 | | 144 | 8 | Accept | 0.62622 | 3.8393 | 0.010061 | 271 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.0037069 | | | | | | | | | | KernelScale: 0.81773 | | 145 | 8 | Accept | 0.025014 | 27.536 | 0.010061 | 1084 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 268 | | | | | | | | | | MinLeafSize: 1 | | | | | | | | | | MaxNumSplits: 79 | | 146 | 8 | Best | 0.0038767 | 74.175 | 0.0038767 | 17335 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 223 | | | | | | | | | | MinLeafSize: 1 | | | | | | | | | | MaxNumSplits: 75 | | 147 | 8 | Accept | 0.74165 | 4.4671 | 0.0038767 | 271 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.052342 | | | | | | | | | | KernelScale: 305.83 | | 148 | 8 | Accept | 0.082795 | 2.2053 | 0.0038767 | 271 | knn | NumNeighbors: 20 | | 149 | 8 | Accept | 0.57647 | 153.17 | 0.0038767 | 271 | nb | DistributionNames: kernel | | | | | | | | | | Width: 200.41 | | 150 | 8 | Accept | 0.041767 | 7.58 | 0.0038767 | 271 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 0.0095919 | | | | | | | | | | LayerSizes: [ 89 3 ] | | 151 | 8 | Accept | 0.018276 | 18.316 | 0.0038767 | 1084 | ensemble | Method: AdaBoostM2 | | | ...
__________________________________________________________ Optimization completed. Total iterations: 595 Total elapsed time: 1276.4375 seconds Total time for training and validation: 9777.0453 seconds Best observed learner is an ensemble model with: Learner: ensemble Method: AdaBoostM2 NumLearningCycles: 223 MinLeafSize: 1 MaxNumSplits: 75 Observed validation loss: 0.0038767 Time for training and validation: 74.175 seconds Documentation for fitcauto display
fitcauto によって返される最終的なモデルが、観測された最適な学習器となります。モデルを返す前に、関数はすべての学習データ (XTrain と YTrain)、リストされている Learner (またはモデル) のタイプ、および表示されたハイパーパラメーター値を使用して、モデルの再学習を行います。
テスト セットのパフォーマンスの評価
テスト データ セットに対する最終モデルのパフォーマンスを評価します。
testAccuracy = 1 - loss(Mdl,XTest,YTest)
testAccuracy = 0.9958
最終モデルは、99% を超える観測値を正しく分類しています。
特徴選択と分類器自動選択の組み合わせ
fitcauto を使用し、table で指定した予測子データと応答データに基づいて、最適化されたハイパーパラメーターをもつ分類モデルを自動的に選択します。データを fitcauto に渡す前に特徴選択を実行して、重要でない予測子をデータセットから削除します。
データの読み込みと分割
標本ファイル CreditRating_Historical.dat を table に読み取ります。予測子データは、法人顧客リストの財務比率と業種の情報で構成されます。応答変数は、格付機関が割り当てた格付けから構成されます。データセットの最初の数行をプレビューします。
creditrating = readtable("CreditRating_Historical.dat");
head(creditrating)ans=8×8 table
ID WC_TA RE_TA EBIT_TA MVE_BVTD S_TA Industry Rating
_____ ______ ______ _______ ________ _____ ________ _______
62394 0.013 0.104 0.036 0.447 0.142 3 {'BB' }
48608 0.232 0.335 0.062 1.969 0.281 8 {'A' }
42444 0.311 0.367 0.074 1.935 0.366 1 {'A' }
48631 0.194 0.263 0.062 1.017 0.228 4 {'BBB'}
43768 0.121 0.413 0.057 3.647 0.466 12 {'AAA'}
39255 -0.117 -0.799 0.01 0.179 0.082 4 {'CCC'}
62236 0.087 0.158 0.049 0.816 0.324 2 {'BBB'}
39354 0.005 0.181 0.034 2.597 0.388 7 {'AA' }
変数 ID の各値は一意の顧客 ID であるため (つまり、length(unique(creditrating.ID)) は creditrating に含まれる観測値の数に等しい)、変数 ID は予測子としては適切ではありません。変数 ID を table から削除し、変数 Industry を categorical 変数に変換します。
creditrating = removevars(creditrating,"ID");
creditrating.Industry = categorical(creditrating.Industry);データを学習セットとテスト セットに分割します。モデル選択とハイパーパラメーター調整のプロセスに観測値の約 85% を使用し、fitcauto によって返された、新しいデータに対する最終モデルのパフォーマンスのテストに観測値の 15% を使用します。cvpartition を使用してデータを分割します。
rng("default") % For reproducibility of the partition c = cvpartition(creditrating.Rating,"Holdout",0.15); trainingIndices = training(c); % Indices for the training set testIndices = test(c); % Indices for the test set creditTrain = creditrating(trainingIndices,:); creditTest = creditrating(testIndices,:);
特徴選択の実行
学習データを fitcauto に渡す前に、関数 fscchi2 を使用して重要な予測子を見つけます。関数 bar を使用して、予測子スコアを可視化します。Inf となるスコアがあっても、bar は Inf 値を破棄するため、最初に有限のスコアをプロットしてから、Inf スコアを有限値として別の色でプロットします。
[idx,scores] = fscchi2(creditTrain,"Rating"); bar(scores(idx)) % Represents finite scores hold on veryImportant = isinf(scores); finiteMax = max(scores(~veryImportant)); bar(finiteMax*veryImportant(idx)) % Represents Inf scores hold off xticklabels(strrep(creditTrain.Properties.VariableNames(idx),"_","\_")) xtickangle(45) legend(["Finite Scores","Inf Scores"])
予測子 Industry のスコアは低く、0.05 より大きい p 値に相当します。これは、Industry が重要な特徴ではない可能性があることを示しています。学習データ セットとテスト データ セットから、特徴量 Industry を削除します。
creditTrain = removevars(creditTrain,'Industry'); creditTest = removevars(creditTest,'Industry');
fitcauto の実行
学習データを fitcauto に渡します。この関数は、ベイズ最適化を使用してモデルとそのハイパーパラメーター値を選択し、パフォーマンスが最大となることが期待される学習済みモデル Mdl を返します。使用可能なすべての学習器のタイプを試すよう指定し、最適化を並列実行するよう指定します (Parallel Computing Toolbox™ が必要)。ベイズ最適化の詳細が格納された 2 番目の出力 Results を返します。
このプロセスにいくらか時間がかかることを見込んでください。既定の設定では、fitcauto は、最適化のプロット、および最適化の結果の反復表示を提供します。これらの結果を解釈する方法の詳細については、Verbose の表示を参照してください。
options = struct("UseParallel",true); [Mdl,Results] = fitcauto(creditTrain,"Rating", ... "Learners","all","HyperparameterOptimizationOptions",options);
Warning: It is recommended that you first standardize all numeric predictors when optimizing the Naive Bayes 'Width' parameter. Ignore this warning if you have done that.
Copying objective function to workers...
Warning: Files that have already been attached are being ignored. To see which files are attached see the 'AttachedFiles' property of the parallel pool.
Done copying objective function to workers. Learner types to explore: discr, ensemble, kernel, knn, linear, nb, net, svm, tree Total iterations (MaxObjectiveEvaluations): 270 Total time (MaxTime): Inf |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 1 | 7 | Best | 0.31798 | 0.78014 | 0.31798 | 0.31798 | tree | MinLeafSize: 1 | | 2 | 7 | Accept | 0.47622 | 0.76358 | 0.31798 | 0.31798 | knn | NumNeighbors: 63 | | | | | | | | | | Distance: correlation | | 3 | 3 | Accept | 0.42896 | 0.95281 | 0.31798 | 0.31798 | discr | Delta: 1.5003e-06 | | | | | | | | | | Gamma: 0.47392 | | 4 | 3 | Accept | 0.74185 | 0.99139 | 0.31798 | 0.31798 | discr | Delta: 389.85 | | | | | | | | | | Gamma: 0.29596 | | 5 | 3 | Accept | 0.7236 | 1.0177 | 0.31798 | 0.31798 | knn | NumNeighbors: 599 | | | | | | | | | | Distance: hamming | | 6 | 3 | Accept | 0.5848 | 2.195 | 0.31798 | 0.31798 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.84619 | | | | | | | | | | KernelScale: 19.346 | | 7 | 3 | Accept | 0.74185 | 2.3486 | 0.31798 | 0.31798 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.0052084 | | | | | | | | | | KernelScale: 128.61 | | 8 | 8 | Best | 0.28059 | 0.30573 | 0.28059 | 0.28059 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 9 | 6 | Accept | 0.74215 | 6.4474 | 0.28059 | 0.28059 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.028123 | | | | | | | | | | Lambda: 0.016312 | | 10 | 6 | Accept | 0.74185 | 1.0242 | 0.28059 | 0.28059 | knn | NumNeighbors: 1052 | | | | | | | | | | Distance: jaccard | | 11 | 6 | Accept | 0.38618 | 1.1643 | 0.28059 | 0.28059 | knn | NumNeighbors: 8 | | | | | | | | | | Distance: mahalanobis | | 12 | 5 | Accept | 0.76967 | 1.9214 | 0.28059 | 0.29713 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 14 | | | | | | | | | | LearnRate: 0.0069636 | | | | | | | | | | MinLeafSize: 230 | | 13 | 5 | Accept | 0.3096 | 2.4447 | 0.28059 | 0.29713 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.45485 | | 14 | 8 | Accept | 0.40532 | 1.6213 | 0.28059 | 0.29713 | linear | Coding: onevsall | | | | | | | | | | Lambda: 0.00047099 | | | | | | | | | | Learner: logistic | | 15 | 7 | Accept | 0.74185 | 4.2042 | 0.28059 | 0.29713 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 20.46 | | | | | | | | | | Lambda: 0.081223 | | 16 | 7 | Accept | 0.48011 | 0.83491 | 0.28059 | 0.29713 | linear | Coding: onevsall | | | | | | | | | | Lambda: 0.0012929 | | | | | | | | | | Learner: svm | | 17 | 4 | Accept | 0.379 | 7.0624 | 0.24978 | 0.31798 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 156.99 | | | | | | | | | | Lambda: 7.336e-06 | | 18 | 4 | Best | 0.24978 | 2.7256 | 0.24978 | 0.31798 | linear | Coding: onevsone | | | | | | | | | | Lambda: 1.3809e-08 | | | | | | | | | | Learner: logistic | | 19 | 4 | Accept | 0.25067 | 1.5777 | 0.24978 | 0.31798 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 80.452 | | | | | | | | | | KernelScale: 20.931 | | 20 | 4 | Accept | 0.68112 | 2.197 | 0.24978 | 0.31798 | nb | DistributionNames: kernel | | | | | | | | | | Width: 3.3045 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 21 | 8 | Best | 0.24319 | 2.5177 | 0.24319 | 0.31798 | linear | Coding: onevsone | | | | | | | | | | Lambda: 7.4269e-05 | | | | | | | | | | Learner: svm | | 22 | 5 | Accept | 0.24559 | 2.5222 | 0.24319 | 0.31798 | linear | Coding: onevsone | | | | | | | | | | Lambda: 9.6858e-06 | | | | | | | | | | Learner: svm | | 23 | 5 | Accept | 0.26653 | 0.28511 | 0.24319 | 0.31798 | tree | MinLeafSize: 61 | | 24 | 5 | Accept | 0.66647 | 0.40489 | 0.24319 | 0.31798 | knn | NumNeighbors: 81 | | | | | | | | | | Distance: hamming | | 25 | 5 | Accept | 0.28059 | 0.19167 | 0.24319 | 0.31798 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 26 | 7 | Accept | 0.24499 | 2.1437 | 0.24319 | 0.31798 | linear | Coding: onevsone | | | | | | | | | | Lambda: 0.0014257 | | | | | | | | | | Learner: svm | | 27 | 7 | Accept | 0.28059 | 0.19273 | 0.24319 | 0.31798 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 28 | 6 | Accept | 0.26503 | 0.31492 | 0.24319 | 0.31798 | tree | MinLeafSize: 41 | | 29 | 6 | Accept | 0.74185 | 0.86991 | 0.24319 | 0.31798 | net | Activations: relu | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 19.947 | | | | | | | | | | LayerSizes: [ 12 9 162 ] | | 30 | 6 | Accept | 0.71523 | 1.7617 | 0.24319 | 0.27687 | linear | Coding: onevsone | | | | | | | | | | Lambda: 7.8161 | | | | | | | | | | Learner: svm | | 31 | 7 | Accept | 0.31289 | 0.29991 | 0.24319 | 0.27687 | knn | NumNeighbors: 12 | | | | | | | | | | Distance: seuclidean | | 32 | 7 | Accept | 0.31289 | 0.23823 | 0.24319 | 0.27687 | knn | NumNeighbors: 12 | | | | | | | | | | Distance: seuclidean | | 33 | 6 | Accept | 0.77984 | 11.017 | 0.24319 | 0.27687 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.0048668 | | | | | | | | | | Lambda: 6.1392e-06 | | 34 | 6 | Accept | 0.31289 | 0.18884 | 0.24319 | 0.27687 | knn | NumNeighbors: 12 | | | | | | | | | | Distance: seuclidean | | 35 | 5 | Accept | 0.78612 | 3.896 | 0.24319 | 0.27687 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.0016754 | | | | | | | | | | Lambda: 6.6323e-06 | | 36 | 5 | Accept | 0.74185 | 0.47636 | 0.24319 | 0.27687 | discr | Delta: 30.053 | | | | | | | | | | Gamma: 0.73742 | | 37 | 8 | Accept | 0.26443 | 0.12865 | 0.24319 | 0.27532 | tree | MinLeafSize: 33 | | 38 | 5 | Accept | 0.32276 | 0.13117 | 0.24319 | 0.27532 | tree | MinLeafSize: 328 | | 39 | 5 | Accept | 0.28059 | 0.17681 | 0.24319 | 0.27532 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 40 | 5 | Accept | 0.71702 | 0.44235 | 0.24319 | 0.27532 | discr | Delta: 4.3736 | | | | | | | | | | Gamma: 0.086011 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 41 | 5 | Accept | 0.25755 | 0.30003 | 0.24319 | 0.27532 | knn | NumNeighbors: 17 | | | | | | | | | | Distance: cityblock | | 42 | 8 | Accept | 0.58989 | 1.8892 | 0.24319 | 0.27532 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 2.3154 | | | | | | | | | | Lambda: 0.17755 | | 43 | 8 | Best | 0.23961 | 1.8068 | 0.23961 | 0.27532 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 1.6804 | | | | | | | | | | KernelScale: 0.69915 | | 44 | 8 | Accept | 0.52019 | 3.4078 | 0.23961 | 0.27532 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.0020203 | | | | | | | | | | KernelScale: 4.9323 | | 45 | 8 | Accept | 0.39844 | 1.2275 | 0.23961 | 0.27532 | linear | Coding: onevsall | | | | | | | | | | Lambda: 0.000245 | | | | | | | | | | Learner: logistic | | 46 | 7 | Accept | 0.29345 | 3.2203 | 0.23961 | 0.28001 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 11 | | | | | | | | | | LearnRate: 0.0037912 | | | | | | | | | | MinLeafSize: 3 | | 47 | 7 | Accept | 0.28597 | 0.24422 | 0.23961 | 0.28001 | tree | MinLeafSize: 10 | | 48 | 8 | Accept | 0.24828 | 2.1892 | 0.23961 | 0.26397 | linear | Coding: onevsone | | | | | | | | | | Lambda: 3.0856e-06 | | | | | | | | | | Learner: logistic | | 49 | 7 | Accept | 0.42776 | 0.87452 | 0.23961 | 0.26397 | discr | Delta: 0.00024246 | | | | | | | | | | Gamma: 0.94088 | | 50 | 7 | Accept | 0.42776 | 0.73959 | 0.23961 | 0.26397 | discr | Delta: 0.00024246 | | | | | | | | | | Gamma: 0.94088 | | 51 | 6 | Accept | 0.85372 | 22.106 | 0.23961 | 0.26397 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 335 | | | | | | | | | | LearnRate: 0.0029421 | | | | | | | | | | MinLeafSize: 418 | | 52 | 6 | Accept | 0.26982 | 0.13679 | 0.23961 | 0.26397 | tree | MinLeafSize: 17 | | 53 | 6 | Accept | 0.74185 | 0.39262 | 0.23961 | 0.26397 | net | Activations: sigmoid | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 18.22 | | | | | | | | | | LayerSizes: [ 18 7 ] | | 54 | 7 | Accept | 0.28059 | 0.16664 | 0.23961 | 0.26397 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 55 | 7 | Accept | 0.28059 | 0.12859 | 0.23961 | 0.26397 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 56 | 7 | Accept | 0.28059 | 0.11931 | 0.23961 | 0.26397 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 57 | 7 | Accept | 0.43703 | 0.12446 | 0.23961 | 0.26397 | discr | Delta: 0.38801 | | | | | | | | | | Gamma: 0.19442 | | 58 | 6 | Accept | 0.49028 | 18.139 | 0.23961 | 0.26397 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 1.0284 | | | | | | | | | | KernelScale: 0.066897 | | 59 | 6 | Accept | 0.74185 | 0.41221 | 0.23961 | 0.26397 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.1378 | | | | | | | | | | LayerSizes: [ 58 2 ] | | 60 | 6 | Accept | 0.49447 | 0.84836 | 0.23961 | 0.27351 | linear | Coding: onevsall | | | | | | | | | | Lambda: 3.1837e-07 | | | | | | | | | | Learner: svm | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 61 | 7 | Accept | 0.70117 | 0.17095 | 0.23961 | 0.27351 | knn | NumNeighbors: 1 | | | | | | | | | | Distance: hamming | | 62 | 7 | Accept | 0.70117 | 0.17044 | 0.23961 | 0.27351 | knn | NumNeighbors: 1 | | | | | | | | | | Distance: hamming | | 63 | 7 | Accept | 0.4819 | 0.16752 | 0.23961 | 0.27351 | knn | NumNeighbors: 15 | | | | | | | | | | Distance: correlation | | 64 | 8 | Accept | 0.24469 | 2.4156 | 0.23961 | 0.26436 | linear | Coding: onevsone | | | | | | | | | | Lambda: 1.0606e-06 | | | | | | | | | | Learner: svm | | 65 | 7 | Accept | 0.27251 | 1.4945 | 0.23961 | 0.26436 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.016406 | | 66 | 7 | Accept | 0.27251 | 1.5369 | 0.23961 | 0.26436 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.016406 | | 67 | 8 | Accept | 0.3102 | 1.6327 | 0.23961 | 0.26436 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.66555 | | | | | | | | | | KernelScale: 7.7425 | | 68 | 7 | Accept | 0.31798 | 0.31217 | 0.23961 | 0.26436 | tree | MinLeafSize: 1 | | 69 | 7 | Accept | 0.31798 | 0.1989 | 0.23961 | 0.26436 | tree | MinLeafSize: 1 | | 70 | 8 | Accept | 0.7239 | 2.2998 | 0.23961 | 0.26436 | nb | DistributionNames: kernel | | | | | | | | | | Width: 7.994 | | 71 | 8 | Accept | 0.26982 | 0.16298 | 0.23961 | 0.26436 | tree | MinLeafSize: 17 | | 72 | 7 | Accept | 0.4843 | 2.8168 | 0.23961 | 0.26436 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.033141 | | | | | | | | | | KernelScale: 0.28115 | | 73 | 7 | Accept | 0.4843 | 2.8684 | 0.23961 | 0.26436 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.033141 | | | | | | | | | | KernelScale: 0.28115 | | 74 | 8 | Accept | 0.25157 | 17.917 | 0.23961 | 0.26436 | net | Activations: tanh | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 1.7369e-06 | | | | | | | | | | LayerSizes: 8 | | 75 | 8 | Accept | 0.28059 | 0.14319 | 0.23961 | 0.26436 | nb | DistributionNames: normal | | | | | | | | | | Width: NaN | | 76 | 8 | Accept | 0.27401 | 5.2171 | 0.23961 | 0.26436 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 6.8476 | | | | | | | | | | Lambda: 0.00036546 | | 77 | 8 | Accept | 0.24678 | 5.4758 | 0.23961 | 0.26436 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 1.6458 | | | | | | | | | | Lambda: 0.00029076 | | 78 | 8 | Accept | 0.24768 | 15.469 | 0.23961 | 0.26436 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 1.6458 | | | | | | | | | | Lambda: 0.00029076 | | 79 | 8 | Accept | 0.74185 | 2.8161 | 0.23961 | 0.26436 | nb | DistributionNames: kernel | | | | | | | | | | Width: 57.408 | | 80 | 8 | Accept | 0.26683 | 16.599 | 0.23961 | 0.26436 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 356 | | | | | | | | | | LearnRate: 0.4849 | | | | | | | | | | MinLeafSize: 190 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 81 | 8 | Accept | 0.43045 | 0.14072 | 0.23961 | 0.26436 | discr | Delta: 5.5966e-05 | | | | | | | | | | Gamma: 0.72927 | | 82 | 8 | Accept | 0.42806 | 0.11498 | 0.23961 | 0.26436 | discr | Delta: 0.014062 | | | | | | | | | | Gamma: 0.11412 | | 83 | 7 | Accept | 0.34699 | 17.983 | 0.23961 | 0.26436 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.15793 | | | | | | | | | | Lambda: 3.4157e-07 | | 84 | 7 | Accept | 0.26713 | 0.14254 | 0.23961 | 0.26436 | tree | MinLeafSize: 37 | | 85 | 8 | Accept | 0.25067 | 0.34385 | 0.23961 | 0.26436 | knn | NumNeighbors: 107 | | | | | | | | | | Distance: euclidean | | 86 | 8 | Accept | 0.25067 | 0.3674 | 0.23961 | 0.26436 | knn | NumNeighbors: 107 | | | | | | | | | | Distance: euclidean | | 87 | 8 | Accept | 0.25456 | 0.12393 | 0.23961 | 0.26436 | knn | NumNeighbors: 21 | | | | | | | | | | Distance: euclidean | | 88 | 8 | Accept | 0.42866 | 0.16366 | 0.23961 | 0.26436 | discr | Delta: 0.010446 | | | | | | | | | | Gamma: 0.45787 | | 89 | 8 | Accept | 0.42746 | 7.739 | 0.23961 | 0.26436 | net | Activations: none | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 0.012033 | | | | | | | | | | LayerSizes: [ 36 71 224 ] | | 90 | 8 | Accept | 0.42148 | 0.1231 | 0.23961 | 0.26436 | discr | Delta: 6.5069e-06 | | | | | | | | | | Gamma: 0.035196 | | 91 | 8 | Accept | 0.2402 | 4.845 | 0.23961 | 0.26436 | net | Activations: none | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 0.00031695 | | | | | | | | | | LayerSizes: [ 1 1 4 ] | | 92 | 7 | Accept | 0.25337 | 4.0368 | 0.23961 | 0.26436 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 87 | | | | | | | | | | LearnRate: 0.72655 | | | | | | | | | | MinLeafSize: 8 | | 93 | 7 | Accept | 0.29076 | 1.476 | 0.23961 | 0.26436 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 6.1463 | | | | | | | | | | KernelScale: 11.586 | | 94 | 8 | Accept | 0.24858 | 2.4734 | 0.23961 | 0.25191 | linear | Coding: onevsone | | | | | | | | | | Lambda: 5.8564e-07 | | | | | | | | | | Learner: logistic | | 95 | 8 | Accept | 0.75381 | 2.7208 | 0.23961 | 0.25191 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.0011931 | | | | | | | | | | Lambda: 0.0021196 | | 96 | 8 | Accept | 0.74514 | 2.5576 | 0.23961 | 0.25191 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.0011931 | | | | | | | | | | Lambda: 0.0021196 | | 97 | 8 | Accept | 0.2423 | 1.4238 | 0.23961 | 0.25191 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.037756 | | | | | | | | | | KernelScale: 0.054479 | | 98 | 7 | Accept | 0.24529 | 72.651 | 0.23961 | 0.25191 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.092471 | | | | | | | | | | KernelScale: 0.0035999 | | 99 | 7 | Accept | 0.32605 | 1.0318 | 0.23961 | 0.25191 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.002311 | | 100 | 7 | Accept | 0.26683 | 0.11759 | 0.23961 | 0.25191 | tree | MinLeafSize: 60 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 101 | 7 | Accept | 0.31738 | 0.73567 | 0.23961 | 0.25191 | knn | NumNeighbors: 440 | | | | | | | | | | Distance: minkowski | | 102 | 8 | Accept | 0.24619 | 2.8761 | 0.23961 | 0.25191 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.051313 | | | | | | | | | | KernelScale: 0.018609 | | 103 | 8 | Accept | 0.24619 | 2.8825 | 0.23961 | 0.25191 | svm | Coding: onevsone | | | | | | | | | | BoxConstraint: 0.051313 | | | | | | | | | | KernelScale: 0.018609 | | 104 | 8 | Accept | 0.31798 | 0.18993 | 0.23961 | 0.25191 | knn | NumNeighbors: 30 | | | | | | | | | | Distance: seuclidean | | 105 | 8 | Accept | 0.27251 | 7.5016 | 0.23961 | 0.25191 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 2.0044 | | | | | | | | | | Lambda: 0.00044252 | | 106 | 8 | Accept | 0.25606 | 2.4574 | 0.23961 | 0.25191 | ensemble | Method: AdaBoostM2 | | | | | | | | | | NumLearningCycles: 47 | | | | | | | | | | LearnRate: 0.37809 | | | | | | | | | | MinLeafSize: 56 | | 107 | 8 | Accept | 0.71702 | 0.11134 | 0.23961 | 0.25191 | discr | Delta: 4.4097 | | | | | | | | | | Gamma: 0.8883 | | 108 | 8 | Accept | 0.43195 | 0.10201 | 0.23961 | 0.25191 | discr | Delta: 3.4149e-06 | | | | | | | | | | Gamma: 0.66863 | | 109 | 8 | Accept | 0.5851 | 0.16296 | 0.23961 | 0.25191 | knn | NumNeighbors: 1 | | | | | | | | | | Distance: correlation | | 110 | 8 | Accept | 0.3102 | 120.87 | 0.23961 | 0.25191 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 1.2141e-07 | | | | | | | | | | LayerSizes: 157 | | 111 | 7 | Accept | 0.79958 | 4.0886 | 0.23961 | 0.25431 | kernel | Coding: onevsall | | | | | | | | | | KernelScale: 0.005193 | | | | | | | | | | Lambda: 8.7114e-07 | | 112 | 7 | Accept | 0.48998 | 1.3636 | 0.23961 | 0.25431 | linear | Coding: onevsall | | | | | | | | | | Lambda: 1.0966e-07 | | | | | | | | | | Learner: svm | | 113 | 8 | Accept | 0.36464 | 126.9 | 0.23961 | 0.25431 | net | Activations: tanh | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 7.078e-08 | | | | | | | | | | LayerSizes: [ 85 74 ] | | 114 | 8 | Accept | 0.45319 | 139.01 | 0.23961 | 0.25431 | svm | Coding: onevsall | | | | | | | | | | BoxConstraint: 0.049562 | | | | | | | | | | KernelScale: 0.019349 | | 115 | 7 | Accept | 0.24379 | 18.913 | 0.23961 | 0.25431 | net | Activations: sigmoid | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 5.0497e-07 | | | | | | | | | | LayerSizes: [ 1 4 5 ] | | 116 | 7 | Accept | 0.42836 | 1.076 | 0.23961 | 0.25431 | discr | Delta: 0.0016971 | | | | | | | | | | Gamma: 0.35943 | | 117 | 6 | Accept | 0.24319 | 20.917 | 0.23961 | 0.25622 | net | Activations: sigmoid | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 5.0497e-07 | | | | | | | | | | LayerSizes: [ 1 4 5 ] | | 118 | 6 | Accept | 0.46814 | 1.0997 | 0.23961 | 0.25622 | linear | Coding: onevsall | | | | | | | | | | Lambda: 7.5025e-05 | | | | | | | | | | Learner: svm | | 119 | 8 | Accept | 0.74065 | 2.2714 | 0.23961 | 0.25622 | nb | DistributionNames: kernel | | | | | | | | | | Width: 19.139 | | 120 | 8 | Accept | 0.30302 | 0.17067 | 0.23961 | 0.25622 | tree | MinLeafSize: 191 | |=======================================================================================================================================================| | Iter | Active | Eval | Validation | Time for training | Observed min | Estimated min | Learner | Hyperparameter: Value | | | workers | result | loss | & validation (sec)| validation loss | validation loss | | | |=======================================================================================================================================================| | 121 | 8 | Accept | 0.26593 | 0.11582 | 0.23961 | 0.25622 | tree | MinLeafSize: 40 | | 122 | 8 | Accept | 0.47712 | 0.68686 | 0.23961 | 0.25178 | linear | Coding: onevsall | | | | | | | | | | Lambda: 0.2622 | | | | | | | | | | Learner: svm | | 123 | 8 | Accept | 0.24798 | 8.1465 | 0.23961 | 0.25178 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 85 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 14 | | 124 | 6 | Accept | 0.29554 | 48.462 | 0.23961 | 0.25178 | net | Activations: relu | | | | | | | | | | Standardize: true | | | | | | | | | | Lambda: 5.8633e-09 | | | | | | | | | | LayerSizes: 115 | | 125 | 6 | Accept | 0.24978 | 8.4739 | 0.23961 | 0.25178 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 85 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 14 | | 126 | 6 | Accept | 0.25157 | 8.422 | 0.23961 | 0.25178 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 85 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 14 | | 127 | 6 | Accept | 0.25815 | 1.9238 | 0.23961 | 0.25178 | nb | DistributionNames: kernel | | | | | | | | | | Width: 0.062941 | | 128 | 6 | Accept | 0.24948 | 2.291 | 0.23961 | 0.25148 | linear | Coding: onevsone | | | | | | | | | | Lambda: 1.2391e-07 | | | | | | | | | | Learner: logistic | | 129 | 8 | Accept | 0.28118 | 5.9878 | 0.23961 | 0.25148 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 82 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 153 | | 130 | 8 | Accept | 0.29644 | 8.8037 | 0.23961 | 0.25148 | ensemble | Method: RUSBoost | | | | | | | | | | NumLearningCycles: 85 | | | | | | | | | | LearnRate: 0.56002 | | | | | | | | | | MinLeafSize: 8 | | 131 | 7 | Accept | 0.39785 | 8.9125 | 0.23961 | 0.25148 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.10138 | | | | | | | | | | Lambda: 0.00010013 | | 132 | 7 | Accept | 0.28298 | 6.7573 | 0.23961 | 0.25148 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 81 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 151 | | 133 | 8 | Accept | 0.27371 | 6.3282 | 0.23961 | 0.25148 | ensemble | Method: Bag | | | | | | | | | | NumLearningCycles: 84 | | | | | | | | | | LearnRate: NaN | | | | | | | | | | MinLeafSize: 129 | | 134 | 8 | Accept | 0.24559 | 76.492 | 0.23961 | 0.25148 | net | Activations: sigmoid | | | | | | | | | | Standardize: false | | | | | | | | | | Lambda: 2.4405e-07 | | | | | | | | | | LayerSizes: [ 33 17 82 ] | | 135 | 7 | Accept | 0.34819 | 16.667 | 0.23961 | 0.25148 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.12039 | | | | | | | | | | Lambda: 0.00010483 | | 136 | 7 | Accept | 0.42477 | 8.2081 | 0.23961 | 0.25148 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.095799 | | | | | | | | | | Lambda: 8.6717e-05 | | 137 | 6 | Accept | 0.25007 | 4.8241 | 0.23961 | 0.25148 | kernel | Coding: onevsone | | | | | | | | | | KernelScale: 0.54593 | | | | | | | | | | Lambda: 0.0017131 | | 138 | 6 | Accept | 0.74185 | 0.75998 | 0.23961 | 0.25148 | net | Activations: sigmoid | | | | | | ...
__________________________________________________________ Optimization completed. Total iterations: 271 Total elapsed time: 907.7117 seconds Total time for training and validation: 5400.9893 seconds Best observed learner is a net model with: Learner: net Activations: relu Standardize: false Lambda: 0.0004658 LayerSizes: [1 31 23] Observed validation loss: 0.23811 Time for training and validation: 21.2958 seconds Best estimated learner (returned model) is a net model with: Learner: net Activations: none Standardize: false Lambda: 0.00036647 LayerSizes: [1 6 10] Estimated validation loss: 0.24112 Estimated time for training and validation: 5.894 seconds Documentation for fitcauto display
fitcauto によって返される最終的なモデルが、最適な推定学習器となります。モデルを返す前に、関数は学習データ全体 (creditTrain)、リストされている Learner (またはモデル) のタイプ、および表示されたハイパーパラメーター値を使用して、モデルの再学習を行います。
テスト セットのパフォーマンスの評価
モデル Mdl は、基準 "min-visited-mean" に従った、ベイズ最適化における最良の点に対応します。新しいデータに対するモデルのパフォーマンスを評価するには、モデルで観測された交差検証精度 (cvAccuracy)、およびベイズ最適化に基づき推定されたモデルの全体的なパフォーマンス (estimatedAccuracy) を確認します。
[x,~,iteration] = bestPoint(Results,"Criterion","min-visited-mean"); cvError = Results.ObjectiveTrace(iteration); cvAccuracy = 1 - cvError
cvAccuracy = 0.7595
estimatedError = predictObjective(Results,x); estimatedAccuracy = 1 - estimatedError
estimatedAccuracy = 0.7589
テスト セットに対するモデルのパフォーマンスを評価します。結果から混同行列を作成し、混同行列におけるクラスの順序を指定します。
testAccuracy = 1 - loss(Mdl,creditTest,"Rating")testAccuracy = 0.7437
cm = confusionchart(creditTest.Rating,predict(Mdl,creditTest)); sortClasses(cm,["AAA","AA","A","BBB","BB","B","CCC"])
入力引数
出力引数
詳細
ヒント
データ セットのサイズ、指定した学習器の数、および選択した最適化手法によっては、
fitcautoの処理に時間がかかる場合があります。Parallel Computing Toolbox のライセンスがある場合、最適化を並列実行して計算を高速化することができます。これを行うには、
"HyperparameterOptimizationOptions",struct("UseParallel",true)を指定します。この構造体に追加フィールドを含めて、その他の最適化の側面を制御できます。HyperparameterOptimizationOptionsを参照してください。学習セットの観測値の数が原因 (たとえば 10,000 を超える場合) でベイズ最適化による
fitcautoの実行に長い時間がかかる場合は、代わりに ASHA 最適化によるfitcautoを使用することを検討してください。データ セットの観測値が多いと、多くの場合、ベイズ最適化よりも ASHA 最適化の方が優れた解を速く見つけます。ASHA 最適化を使用するには、"HyperparameterOptimizationOptions",struct("Optimizer","asha")を指定します。この構造体に追加フィールドを含めて、その他の最適化の側面を制御できます。特に、時間の制約がある場合は、HyperparameterOptimizationOptions構造体のMaxTimeフィールドを指定して、fitcautoを実行する秒数を制限します。
アルゴリズム
代替機能
使用するデータ セットに最適なモデルがわからない場合は、分類学習器アプリを代わりに使用することができます。このアプリを使用すると、さまざまなモデルについてハイパーパラメーターを調整して、パフォーマンスが最も高い最適化済みモデルを選択できます。分類学習器では、モデルのハイパーパラメーターを調整する前に特定のモデルを選択しなければなりませんが、最適化可能なハイパーパラメーターの選択とハイパーパラメーター値の設定をより柔軟に行うことができます。ただし、このアプリでは、最適化を並列実行することや、学習器として
"linear"または"kernel"を最適化すること、観測値の重みを指定すること、事前確率を指定すること、ASHA 最適化を使用することはできません。詳細は、分類学習器アプリのハイパーパラメーターの最適化を参照してください。使用するデータに適合するモデルがわかっている場合は、対応するモデル近似関数を使用し、名前と値の引数
OptimizeHyperparametersを指定して、ハイパーパラメーターを調整することもできます。複数のモデルの結果を比較して最適な分類器を選択できます。このプロセスの例については、ベイズ最適化を使用したモデル選択の自動化への移行を参照してください。
参照
[1] Li, Liam, Kevin Jamieson, Afshin Rostamizadeh, Ekaterina Gonina, Moritz Hardt, Benjamin Recht, and Ameet Talwalkar. “A System for Massively Parallel Hyperparameter Tuning.” ArXiv:1810.05934v5 [Cs], March 16, 2020. https://arxiv.org/abs/1810.05934v5.
拡張機能
バージョン履歴
R2020a で導入
You can also select a web site from the following list:
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)