GUI crashes when using eigs with a nonzero shift on a large generalised eigenvalue problem.

I am using matlab's eigs function to compute a few (5-10) eigenvalues close to a complex non-zero shift sigma of a complex valued generalized eigenvalue problem consisting of a system matrix F and a mass matrix M, both stored in sparse format.

The code I am running is the following:

nev = 10;

shift = 0.01 + 2.8*1i;

opts_F.tol = 1e-9;

opts_F.maxit = 1000;

opts_F.p = 3*nev;

opts_F.disp = 0;

[V,D] = eigs(F, -M, nev, shift/2, opts_F);

For a certain parameter setting where my system matrix is of the order 83000 x 83000 and nnz = ~130 million I can solve the problem. F has a size of ~3GB here.

For a slightly larger system matrix (3.7 GB, 99000 x 99000 with nnz = ~160 million) the GUI crashes mid computation for no detectable reason.

I have tried only computing a single eigenvalue for the large problem and this worked; unfortunately I need at least a few eigenvalue. Considering that I am running the computations on my workstation with 128GB RAM and a cpu with 12 cores (intel Core i9-9920X CPU @ 3.50GHz) I do not see that absolute sizes in the order of a few GB are a deal breaker. Is there some restriction on the matrix sizes for eigs? What else might be the reason for the crash? Could it be on the OS side?

I have also tried closing all but the most essential programs on the workstation while matlab was running to no avail.

I have tried using an external function to compute the action of the system matrix F on a vector instead of constructing it but this is not viable given that in order to use such a function in conjunction with eigs I would need to solve the system (F - sigma*M*I)\x at every step of the arnoldi iteration for which I do not have an efficient solver. This method using e.g. GMRES with a textbook preconditioner, is unfeasibly inefficient and alternative approaches of this sort will need to wait.

I am very grateful for any insight as to how I should proceed in this situation.

Best regards,

Simon

Answer 1

[Christine Tobler]

Ideally this shouldn't crash, but produce an out-of-memory error. However, for example on Linux there is the "out-of-memory killer", a process that will detect when too much system memory is used and just kill the process that uses the most memory. Since this happens on the OS side and not as an error when trying to allocate, MATLAB can't do much about it.

Now about the actual problem: I would have thought that the memory need for computing a factorization of F - sigma*M is what causes the out-of-memory error, but then that should also affect asking for just one eigenvalue. You could try calling

dA = decomposition(F - sigma*M);

and seeing if this finishes, and if yes, how much memory is left over on your OS. You can reuse that factorization by passing a function handle @(x) dA \ x into eigs as its first input (with the second input specifying the matrix size).

You could then try to use the SubspaceDimension NVP to limit the additional search space to be allocated in eigs; however, this usually really isn't that much, so I'm a bit astonished that it would be a problem.

[Simon Kern]

Thank you for your comment!

I also think that it's the OS killing matlab and not matlab itself crashing due to some bug. I was monitoring the memory consumption and it was indeed filling up before the crash. I have not been able to compute the decomposition directly for the large matrix as you suggested (I tried it just now), the GUI got killed with the same memory issue. Is matlab computing a decomposition under the hood when eigs is called? I guess the solution of the linear system required for the shift and invert is what's blowing up the memory cost and there really isn't a way around that unless one implements a specialised iterative solver.

I have played around with the arnoldi subspace dimension a bit but did not see any effect. As you say, the overhead of a few complex vectors of length 100000 is negligible compared to F itself. Maybe the problem is just too big for the machine.

[Christine Tobler]

Yes, MATLAB is computing a decomposition when eigs is called with a specified shift or with the "smallestabs" option, it then iteratively solves linear systems with this decomposition. (To be very precise it's using some of the same internal helper functions as the decomposition object, but the cost of computing the decomposition will be the same here).

With sparse matrices the memory usage for decomposition depends a lot on the sparsity structure of the matrix that's being passed in (which you can look at calling spy(F - sigma*M)). Some things to try that might help:

• Different types of decomposition (that is, if F is symmetric, both "ldl" and "lu" are possible to use; which is better in terms of memory usage is best to just try out)
• Different reordering algorithms (see Sparse Matrices under Reordering Algorithms, interesting are amd, colamd, dissect, for symmetric also symamd and symrcm) can impact the memory usage - decomposition makes a quick choice which may not always be optimal. You can try these reordering algorithms by getting the permutation of rows and/or columns they suggest and then calling decomposition on this reordered matrix.

Answer 2

[Steven Lord]

Please send the crash log file along with the code and data with which you can reproduce the crash to Technical Support directly using the Contact Support link under the Get Support heading at the end of this page. Since those data files are likely to be quite large, if you contact Support and let them know you have a large data file to upload they should be able to set up a secure location to which you can upload the file.