How to apply Explainable AI on user defined classification models (without using inbuilt classifiers)?

I am working on a classification model where I have used LIBSVM for classication. I want to investigate the key features responsible for classification using Explainable AI. Kindly suggest me a solution.

Answer 1

[Drew]

You can use the MATLAB explainable AI functions shapley, lime, and partialDependence while specifying the model using a function handle to the predict method for your LIBSVM model. In more detail:

• The MATLAB shapley function can take a function handle for model prediction. See https://www.mathworks.com/help/stats/shapley.html and the example section "Specify Blackbox Model Using Function Handle" openExample('stats/SpecifyBlackboxModelUsingFunctionHandleExample') . So, the shapley function could be used with a prediction function handle for your LIBSVM model. However, the shapley calculations will be much faster if they take advantage of the linear structure of the SVM model ( https://www.mathworks.com/help/stats/shapley-values-for-machine-learning-model.html ), as could be done with an in-built SVM classifier from MATLAB fitcsvm.
• The MATLAB LIME (Local interpretable model-agnostic explanations) function can take a function handle for model prediction. So, the LIME function could be used with a prediction function handle for your LIBSVM model. See https://www.mathworks.com/help/stats/lime.html and openExample('stats/SpecifyBlackboxModelAsFunctionHandleExample')
• The MATLAB partialDependence function can take a function handle for model prediction. See https://www.mathworks.com/help/stats/regressiontree.partialdependence.html and openExample('stats/SpecifyModelUsingFunctionHandlePartialDependenceExample')

If this answer helps you, it is recommended to accept the answer.

[Snigdha]

Dear Drew

Thanks for your reply.

LIME Attempted:

I attempted the example and it worked fine. but when I tried it for a classification problem using LIBSVM toolbox (code attached), it failed and prompted the following message with an error:

Accuracy = 100% (100/100) (classification)

Error using lime (line 355)

Unable to predict using the blackbox model.

Error in run_classification_demo_1 (line 9)

results = lime(myPredict,X,'Type','classification');

Kindly note that the result generated by LIBSVM is a vector of predicted labels. Then I attempted to use fitsvm and the same error is obtained since it also generates a vactor of labels.

SHAPLEY attempt

I also tried Shapley (run_classification_demo_shap.m), and the following error is obtained.

Error using shapley

Expected function handle output to be a scalar.

Error in shapley/fit (line 393)

validateattributes(out,{'double','single'}, {'scalar','nonempty'},

mfilename,getString(message('stats:shapley:FunctionHandleOutput')));

Error in run_classification_demo_shap (line 12)

explainer = fit(explainer,X(1,:));

I request you to kindly help me for the same.

[Snigdha]

Dear Walter Roberson

Thanks for your prompt reply but this is not helping us since we are using libsvm and all experiments have been conducted.

So, I request you to kindly look into the error that I shared.

[Walter Roberson]

@Snigdha

I did not reply. I just touched up Drew's reply to convert some of the URLs into clickable links and format the openexample calls as fixed-width font for readability.

[Walter Roberson]

Note that libsvm is a third-party product not supported by Mathworks. There was a period during which Mathworks used a modified version of libsvm, but that was completely replaced about 5 or so years ago.

[Drew]

You will need classifier scores from the predict function of libsvm in order to do shapley or lime analysis. Explainability analysis cannot be done with only the predicted labels.

Here is an example which uses fitcsvm, but with a function handle. Note that the classifier scores from prediction are used. Note that the function definition for the function handle appears at the very bottom of the code here.

% Load Fisher's iris data set. Remove all observed setosa irises.
% Leave versicolor and virginica, which are not linearly separable.
load fisheriris.mat
inds = ~strcmp(species,'setosa');
X = meas(inds,:);
Ylabels = species(inds);

% For classification problems, for each query point, there are
% Shapley Values for each class and for each predictor. 
% When using a function handle, need to get the shapley values
% for one class at a time. Start with the first class, which is versicolor.
rng(1);
mdl = fitcsvm(X,Ylabels); 
f= @(x) getFirstClassScore(mdl,x); % See definition at bottom of this file
explainer = shapley(f,X);
explainer = fit(explainer,X(1,:));
plot(explainer)

% Look at the explainer and ShapleyValues for one class, when using function handle
explainer

explainer = 
  shapley with properties:

            BlackboxModel: @(x)getFirstClassScore(mdl,x)
               QueryPoint: [7 3.2000 4.7000 1.4000]
           BlackboxFitted: 1.7136
            ShapleyValues: [4×2 table]
               NumSubsets: 16
                        X: [100×4 double]
    CategoricalPredictors: []
                   Method: 'interventional-kernel'
                Intercept: -0.0188

% The first class is versicolor, so these are the versicolor shapley values
explainer.ShapleyValues

ans = 4×2 table
    Predictor    ShapleyValue
    _________    ____________

      "x1"          0.4394   
      "x2"         0.32068   
      "x3"          0.4188   
      "x4"         0.55359   

Now, add an example of how this looks when using fitcsvm ClassificationSVM object with shapley:

% Much more convenient to load fisher iris with the predictor labels, and
% use builtin shapley command to get shapley values for both classes at
% once.
t=readtable("fisheriris.csv");
inds = ~strcmp(t{:,5},'setosa');
t_selected=t(inds,:);
rng(1);
mdl=fitcsvm(t_selected,"Species");
explainer=shapley(mdl,t_selected);
explainer=fit(explainer,t_selected(1,:));
plot(explainer)

% Or you can plot shapley values for multiple classes at once
plot(explainer,ClassNames=mdl.ClassNames);

% Look at the explainer and ShapleyValues for both classes when using
% ClassificationSVM
% explainer also reports that it is using the 'interventional-linear'
% method, which is faster than 'interventional-kernel' (especially when
% the number of predictors is large).
explainer

explainer = 
  shapley with properties:

            BlackboxModel: [1×1 ClassificationSVM]
               QueryPoint: [1×5 table]
           BlackboxFitted: {'versicolor'}
            ShapleyValues: [4×3 table]
               NumSubsets: 16
                        X: [100×5 table]
    CategoricalPredictors: []
                   Method: 'interventional-linear'
                Intercept: [-0.0188 0.0188]

explainer.ShapleyValues

ans = 4×3 table
      Predictor      versicolor    virginica
    _____________    __________    _________

    "SepalLength"      0.4394       -0.4394 
    "SepalWidth"      0.32068      -0.32068 
    "PetalLength"      0.4188       -0.4188 
    "PetalWidth"      0.55359      -0.55359 

% Create function to return the score for the first class.
% This is used in the function handle example above.
function score = getFirstClassScore(mdl, X)
    [~, score] = predict(mdl, X);
    score = score(:, 1); % Extract scores of the first class
end

Answer 2

[BHARTI RANA]

Can you help me resolve the following error:

"Error using shapley (line 226)

The value of 'X' is invalid. Expected input to be two-dimensional."

I am trying to run shapley as:

shapley(f,data3D);

where f: function handler

data3D: 3D matrix where 3rd dimension indicates feature. (say n x n x d dimensional, where n are number of samples and d are number of features)

my intent is to predict importance of features.

Kindly suggest.

[Walter Roberson]

https://www.mathworks.com/help/stats/shapley.html#mw_69aaa96e-1f1a-4341-8ad6-9ceb84db0bd6

Predictor data, specified as a numeric matrix or table. Each row of X corresponds to one observation, and each column corresponds to one variable.

• For a numeric matrix:
• The variables that makes up the columns of X must have the same order as the predictor variables that trained blackbox, stored in blackbox.X.
• If you trained blackbox using a table, then X can be a numeric matrix if the table contains all numeric predictor variables.

So you simply cannot pass in a 3D array at that point.

[BHARTI RANA]

Thank You for your reply. Here I am specifying Blackbox Model Using Function Handle.

[Walter Roberson]

When you specify a blackbox model using a function handle, then the data matrix you supply must have rows corresponding to observations and must have colums corresponding to variables. 3D data is not supported.

[BHARTI RANA]

No, I am modelling using Composite Kernel where I am having 3D data (n x n x d), where 3rd dimension corresponds to observations.