How to compute Gini impurity in random forest (treebagger)?

Hello. I am doing classification using treebagger random forest. I should compute the gini index or gini impurity to understand each feature importance in classification. Can any one help me with that? thanks. Here is my random forest classifier code:

nTrees=50;
B2 = TreeBagger(nTrees,train_data,train_label, 'Method', 'classification');
predChar2 = B2.predict(test_data);
predictedClass2 = str2double(predChar2);

Answer 1

[Shubham]

Hi Alex,

To compute feature importance using the Gini index (or Gini impurity) in a random forest model with MATLAB's TreeBagger, you can use the OOBPermutedPredictorDeltaError property. This property provides a measure of feature importance based on the increase in prediction error when the values of a given feature are permuted. Here's how you can do it:

Step-by-Step Guide to Compute Feature Importance

1. Train Your Random Forest Model: You already have this step covered with your existing code.
2. Calculate Feature Importance: Use the OOBPermutedPredictorDeltaError property to get the importance of each feature.

Here's how you can modify your code to include feature importance calculation:

% Number of trees
nTrees = 50;

% Train the TreeBagger model
B2 = TreeBagger(nTrees, train_data, train_label, 'Method', 'classification', 'OOBPredictorImportance', 'on');

% Predict the test data
predChar2 = B2.predict(test_data);
predictedClass2 = str2double(predChar2);

% Compute feature importance
featureImportance = B2.OOBPermutedPredictorDeltaError;

% Display feature importance
fprintf('Feature Importance (using Gini index):\n');
for i = 1:length(featureImportance)
    fprintf('Feature %d: %.4f\n', i, featureImportance(i));
end

% Plot feature importance
figure;
bar(featureImportance);
title('Feature Importance (Gini Index)');
xlabel('Feature Number');
ylabel('Importance');