フィルターのクリア

How to load a mat. file containing one big matrix

32 ビュー (過去 30 日間)
Klara Stribrna
Klara Stribrna 2023 年 3 月 9 日
コメント済み: Walter Roberson 2023 年 3 月 10 日
Hello, I am working with 23.8 GB mat. file so I can't load it with the load function. I've tried to load it with matfile function but I have found out that in only contains one variable, which is a 1x1 struct and is again too big to be loaded. I've tried to read one element of the matrix (it's 4-dimensional) and it didn't work, still the same error ("Requested 994x961x3346x1 (23.8GB) array exceeds maximum array size preference (15.8GB). This might cause MATLAB to become unresponsive.") The mat. file was generated from an external program and I think the problem might be that it was not saved with the -v7.3 flag so it doesn't support partial loading. Is there a way to resave this file with the -v7.3 flag without having to load it all first? Or do you have any other suggestions how to acces this data?
  6 件のコメント
Bruno Luong
Bruno Luong 2023 年 3 月 10 日
Thanks. Is there any difference (performance)
  • The array size limit is unchecked?
  • Checked with slider at 100% (default)?
Stephen23
Stephen23 2023 年 3 月 10 日
"Is there any difference (performance)"
As far as I can recall, if the size limit is unchecked then performance is unaffected (at least, nothing humanly noticeable) until an array larger than the installed memory is requested, then the virtual memory is used and it is slower.
But still quite useable if using an SSD.

サインインしてコメントする。

回答 (1 件)

Jan
Jan 2023 年 3 月 9 日
編集済み: Jan 2023 年 3 月 10 日
If the variable in the MAT file has more than 2GB, it must have the v7.3 format.
You try to extract a 994x961x3346 subarray, which needs 25.57 GB for the type double (The difference to 23.8 is ([EDITED], "most likely" removed) a mixing of GB and GiB). But you explain, that the complete file has 23.8 GB also?
If the wanted submatrix does not match into your RAM, this is not a problem of the MAT file, but of an insufficient amount of installed RAM. So install more RAM or import parts of the submatrix only.
  11 件のコメント
Jan
Jan 2023 年 3 月 10 日
Fine. Thanks for the explanation.
Walter Roberson
Walter Roberson 2023 年 3 月 10 日
If you are talking about RAM, for historical reasons it is very common to use 1024 as the multiplier. For example 8 GB RAM is typically 8 * 1024^3 -- at least when you are talking in terms of how much RAM is installed or what the limits are for linking or program size. But if you are looking at how many kilobytes a program is using in the process monitor, there is a tendancy to slip into 1000 as the prefix.
But when you are talking about hard disks especially, 1000 has used as the multiplier for quite a number of years now; possibly not originally but the marketing department won the battle with hard disks a long time ago.

サインインしてコメントする。

カテゴリ

Help Center および File ExchangePerformance and Memory についてさらに検索

タグ

製品


リリース

R2021b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by