Problem of singular matrix?

how is it possible to solve the problem of getting singular matrix

Answer 1

[Walter Roberson]

Sometimes it is a matter of using high precision calculations such as using the Symbolic Toolbox.

However, most of the time if you are getting a singular matrix, it is because the matrix is inherently singular. In that case, the best way to avoid the singular matrix is to not do that calculation. If the calculation you are doing is inv() then it is pretty unlikely that you should be doing that.

[hussein mazeh]

so how can I solve this problem??

Answer 2

[John D'Errico]

The really simple and short answer is to not create singular matrices.

The slightly more complex answer is to understand WHY your matrix is singular. Then you can understand how to fix it. However, we cannot know what you have done to create this singular matrix. There are lots of silly things you could have done to create such a problem. (Sorry, but this is usually the case.)

As well, there are some cases where the matrix is inherently singular. It just is, and that makes complete sense, SOME OF THE TIME. In that case, it depends what you are doing with the matrix. There are some good and some obscenely bad computational schemes one can use for these problems. Again, we cannot help you without more information.

Finally, there are some matrices that are not truly singular, but only numerically singular, when created and used in floating point arithmetic. Sometimes double precision just is not enough. Almost always, going to a higher precision will be a fools task though, because your data does not itself have sufficient precision to support those high precision computations. So you end up computing something with incredibly high precision, yet those extra digits are completely meaningless. Complete garbage. Only rarely will super high precision in something like this be of any real value. (And that said by the author of HPF, the high precision tool in MATLAB.)

Sometimes (rarely) it is possible and appropriate to repair a singular matrix. For example, if you have a covariance matrix that is numerically singular, but you need to compute Gaussian random numbers with that covariance structure. Then you can repair the problem using a tool like my nearestSPD, which will allow things to work.

There are other things you can do for SOME problems. Tools like pinv are sometimes appropriate.

So, really, we need to understand

1. Where the matrix came from. What are you doing to create this matrix?

2. What you intend to do with it.

Only then can we give you any intelligent advice.

[hussein mazeh]

thank u in advance for ur respectful support, there seems a problem in evaluating the inverse of H in this code, since I received a message "Matrix is singular, close to singular or badly scaled" in the line 47 where it is found

function [wapp, prob, logL,H] = logbinfit(Xapp, zapp, flIn, wini)
% [wapp, prob, logL] = logbinfit(Xapp, zapp, flIn, wini) 
% 
% binary logistic regression for (precise) observations 
% 
% Inputs 
%     Xapp : (n*p) array of instances 
%     zapp : (n*1) vector of targets (0/1) 
%     flIn : boolean indicating whether an intercept should be included in the model 
%            (recommended : 1) 
%     wini : initialization for the weight vector 
%            (recommended : zeros(p+1,1)) 
% 
% Outputs 
%.    wapp : (p*1) vector of optimal weights 
%.    prob : (n*1) vector of posterior probabilities for class 1 
%.    logL : value of the log-likelihood associated with wapp 

[napp, dim] = size(Xapp);

prec = 1e-5;

if (flIn == 1),
    Xapp = [ones(napp,1),Xapp];
    dim = dim+1;
end;

%%Iterate until convergence 

% initialization of the parameters 
stop=0; iter=0;wapp=wini;
% loop 
for i=1:2
%while (stop == 0),
    iter = iter + 1;

      prevweight=wapp;
      % store the old parameters 
  [post]=auxiliary(Xapp,wapp);
  b=post.*(ones(napp,1)-post);
  W=diag(b); %compute the diagonal matrix of posterior probalities
  X1=transpose(Xapp);
  H=(-1)*(X1)*W*(Xapp); %compute the hessian corresponding to the W matrix
  dervlnL=(X1)*(zapp-post); %compute derivative of lnL satisfying W matrix
  wapp=wapp-inv(H)*dervlnL;  %compute new weight vector
      % compute the new parameters 

      error=(wapp)-(prevweight);
      if norm(error,2)< prec 
          stop=1;
      end;   
      % check for convergence 
  logL=transpose(zapp)*log(post);logL=logL+transpose(ones(napp,1)-zapp)*log1p(-post);
      % compute the log-likelihood and display it 

      fprintf('Iteration %d, log-L=%1.3f\n', iter, logL);
  end;

prob=post;

end

[John D'Errico]

The immediate and obvious solution is to use pinv there instead of inv to update wapp.

[ahmad eldeeb]

it don't give the same answers!!

[John D'Errico]

You CANNOT invert a singular matrix. That pinv and inv do not give the same result is a darn good thing, NOT a problem. If they did give the identical result, then why would anybody ever have written pinv in the first place?

If your matrix is singular, then you cannot form the inverse. Period. If your problem is such that pinv is insufficient, and you still need to invert the matrix, then you need to improve your data, or you need to accept that a solution does not exist.

It MIGHT seem that a third option is to work in higher precision, but that would be a fallacy here. While you could in theory do your computations in hundreds or even thousands of digits, your data will not support those computations, as you will never have the information content in your data to support such super high precision computation. You will not have sufficient signal to noise ratio for that to be of any value.

I'm sorry, but you cannot make a silk purse from a sow's ear.