Predictor Importance feature for Tree Ensemble (Random Forest) method

Hello, It seems that MATLAB package has two approaches for calculating variable importance:

The first is "predictorImportance": http://www.mathworks.com/help/stats/compactregressionensemble.predictorimportance.html

The second is permutation method: http://www.mathworks.com/help/stats/treebagger.oobpermutedvardeltaerror.html

I'm wondering what are the difference between the two approaches, and which is preferred?

Thanks!

Answer 1

[Prashanth Ravindran]

This query was asked back in 2013. I will try to answer for those people who might be looking for the answer.

predictorImportance. This function has input as the ensemble created by the fitensembe function. And this function can be used to create many different kinds of ensembles such as boosting trees, bagging trees, etc..

treebagger.oobpermutedvardeltaerror: Yes this is an output from the Treebagger function in matlab which implements random forests. This can also be used to implement baggin trees by setting the 'NumPredictorsToSample' to 'all'.

You see the basic algorithms are different for the two functions and hence the outputs may be different.

[pinkong]

I want to know why the value of feature importance is negative sometimes.and how to calculate it?

[Cjb_User Broderick]

Can anyone suggest why feature importance may be negative

[Zainab Al-RubayezayMATH]

Hi

I got a negative result of feature importance as well when I used Treebagger. However, I got a positive result when I try to know what are the most important features of the same dataset by applying predictorImportance for the model result from ensemble.

Does anyone know the reasons?

Thanks Zainab

[Shanning Bao]

For why the feature importance may be negative:

https://groups.google.com/forum/#!topic/comp.soft-sys.matlab/U4gyAhKhHvg

Seems useful