Multiplication of gigantic matrices

1 回表示 (過去 30 日間)
carlos g
carlos g 2023 年 3 月 6 日
コメント済み: Steven Lord 2023 年 3 月 7 日
I am trying to multiply two gigantic matrices: [1585152 90]*[90 1585152]. MATLAB runs out of memory. Is there any workaround for this?
  5 件のコメント
John D'Errico
John D'Errico 2023 年 3 月 6 日
Do you have the memory? The size of your result will be:
elements. Multiply that by 8, assuming doubles. Converting to rough gigabytes:
So only 18700 gigabytes, or 18.7 terabytes of RAM.
Again, do you have that much RAM? I'm not talking about disk space, where even that would be seriously big. Even in these days of large chunks of cheap RAM, 18 terabytes is flat out significantly big.
Unless your arrays are seriously sparse matrices, even converting them to sparse will not help much.
Far better is to do as has been suggested, to NOT do this. A little linear algebra will show you how to do much with the original arrays. And, worse, when you do that multiplication, you actually make things numerically pooly behaved. There are many good reasons to not do what you want to do. And one good reason why you CANNOT do it, because of the memory requirenemt.
Just because you have a formula that shows that product of matrices, does not mean it is a good idea to write code following that formula.
Steven Lord
Steven Lord 2023 年 3 月 7 日
What were you hoping to do with this extremely large matrix? There may be alternate approaches that don't require nearly so much memory. For example, if you were hoping to solve a system using the normal equations don't do that. Use mldivide, \ or one of the iterative methods for solving a system.


回答 (1 件)

Allen 2023 年 3 月 7 日
You can use datastores and tall arrays to work with large amounts of data. See references to both.
  1 件のコメント
Bruno Luong
Bruno Luong 2023 年 3 月 7 日
See John's comment, it need 18.7 Tb to store it, without looking to time to work with huge matrix.
Explicit multiplication is a bad idea that will lead pratically to nowhere.



Help Center および File ExchangeChristmas / Winter についてさらに検索

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by