Cholesky decomposition error when matrix is regularized.

I want to perform cholesky facterization to a covariance matrix using chol. As my data matrix has more columns than rows, thus the covariance matrix should be probably positive semi-definite. I added a regularized term to the diagonal of covariance matrix to make sure that it is positive definite, which should be safely facterized by chol. However, the error "Matrix must be positive definite" still occurs. What's the point if I want to use a regularization parameter to make the covariance matrix suitable for using chol?

My regularization term is 0.01*eye(c), where c is the size of covariance matrix.

Thanks.

Answer 1

[Walter Roberson]

Your covariance matrix probably is not exactly symmetric. Consider forcing it to be symmetric by using

A = (A+A.')/2;

[Jorey]

Thanks. I checked this using

max(max(abs(A-A')))

The result is 0, so I think it is surely symmetric. I used function cov to obtain the covariance matrix.

[Walter Roberson]

(A link for my future reference: https://www.quora.com/What-is-an-eigenvector-of-a-covariance-matrix)

[Walter Roberson]

Investigating, I see that the problem is at least sometimes round off error.

If I generate a random matrix, take its covariance, and look at eig() of the covariance, I can see that the eigenvalues are sometimes negative and small absolute magnitude. When I take eig of sym() of the covariance matrix, sometimes the eigenvalues go non-negative, indicating that round-off error played a significant role. I followed one case further back and calculated the covariance itself symbolically, then even though double() of the symbolic answer displayed the same as the numeric covariance, the symbolic eigenvalues of the symbolic covariance were all non-negative where eig(sym()) of the covariance was not enough in that case. Note: sometimes I had to use simplify() with 'step', 20 on the symbolic eigenvalues to have double() not show any complex residue.

I have not investigated far enough to determine whether this is the only difficulty.

[Walter Roberson]

Also I note https://www.mathworks.com/matlabcentral/answers/101132-how-do-i-determine-if-a-matrix-is-positive-definite-using-matlab

[Jorey]

Thanks for your works. It provides very valuable insights into my question. If so, how this can relate to selection of a parameter 'lambda' which can be safely used as the regularization parameter, i.e., A = A+lambda*eye(c)? Is it the larger the better?

[Walter Roberson]

-min(eig(C), 0) to get to at least 0

However, there might still be round-off problems with this.

[Jorey]

Thanks, it works if I used 'lambda' as max(abs(eig(C))), i.e., C+max(abs(eig(C)))*eye(c), where C is my covariance matrix and c is the size of C.