Classifica​tionDiscri​minant - How to exclude a predictor for only one group level?

I am using ClassificationDiscriminant.fit to classify data. I seek to predict classes of Y, a nominal categorical variable, by X, a n-by-5 matrix of continous valued measurements.

Because some groups of Y (e.g. class level "l.b") have zero variance, I get the error:

“Error using ClassificationDiscriminant (line 624) Predictor x1 has zero variance for class l.b. Either exclude this predictor or set 'discrimType' to 'pseudoQuadratic' or 'diagQuadratic'.”

However, I cannot find user-guide, help notes or documentation that indicate how to exclude a predictor during this process. I do not want to use 'pseudoQuadratic' or the other, but would like to exclude the predictor causing the problem. Can someone please explain a simple efficient way to accomplish this?

The solution proposed to exclude an entire column of the matrix of predictors is not acceptable. This not a viable solution because this is a case where the predictor variance is zero for only one class level of the grouping variable. Removing the entire predictor variable will eliminate an important factor. What I figure is needed is: 1 ) to reassign that category of the grouping variable to another category; or 2 ) remove observations belonging to that category. Does anyone agree? What I am seeking is to use ClassificationDiscriminant.fit to proceed by excluding these zero-var cases without throwing an error, i.e. automatically exclude them. Assuming that is not possible, how do I access the variance information by category (group level) during ClassificationDiscriminant.fit, so that I can use its calculations to handle these cases prior to it throwing an error? Must I calculate the intra group level variances myself prior to submitting the data to ClassificationDiscriminant.fit?

Thank you.

Answer 1

[Ilya]

Delete the corresponding column in the matrix of predictors you pass to ClassificationDiscriminant.fit:

X(:,1) = [];

[George]

Please see my edited question above, with additional detail. Thank you for your answer.

[Ilya]

The quadratic discriminant model needs to compute a covariance matrix for each class. This covariance matrix then needs to be inverted to compute the discriminant coefficients. If a predictor has zero variance for one class, obviously the covariance matrix for that class is singular (non-invertible).

How you want to proceed depends on the goals of your analysis and your data. You haven't clearly stated the former and I don't have access to the latter.

You certainly can compute a covariance matrix for each group. If you want to build a quadratic discriminant model, you will need to invert these covariance matrices somehow. Popular options for inverting a singular covariance matrix are: Ignore off-diagonal elements (extreme regularization, 'diagQuadratic' option), take a pseudo-inverse ('pseudoQuadratic' option), and regularize by adding small positive values to the main diagonal (not provided in ClassificationDiscriminant for the quadratic model). Since you have already dismissed the first two, you might want to regularize or maybe think of something else.

[George]

Thanks again for your comments. Would you agree that it is also possible to overcome the problem by: reassigning the problematic category of the grouping variable to another category, or removing observations belonging to that category? Also, I recognize that other inversion methods are available. However, if in my code I have chosen to use quadratic discriminant model, then to prevent error, by the time I call that function (ClassificationDiscriminant.fit) I need to have ensured that none of the class levels have zero variance. This seems to be redundant, having to perform the calculations of the discriminant function prior to calling it. I suppose that using a try/catch statement and opting to use diag- or pseudoQuadratic option in the catch realm could work.

[Ilya]

If the predictor no longer has zero variance after you merge two groups or if you remove observations for the offending group, you will be able to carry out quadratic discriminant analysis. I have no opinion on whether merging two groups or removing observations from a certain group is a sound approach for your specific analysis.

Instead of using try/catch, you can always use the 'pseudoQuadratic' option. If a covariance matrix is not singular, its inverse is equal to its pseudo inverse. In this case, however, you won't know if one of the covariance matrices is singular because the analysis type in the returned object will be always set to 'pseudoQuadratic'.