Import large txt partially

Hello,

we use matlab to process data stored in large txt files (several GB) and import the data successfully using

readmatrix(dataPath,'OutputType','single');

of the size [many million, 20]

However, often we would like to import only two first few million rows for a faster test-processing and most importantly for a faster import, which often takes several minutes.

Which is the most efficient way to import onkly the first x rows of txt file containing a matrix?

We would like to know this both for the newest and for the R2020b version.

Best regards

Felix

Answer 1

[Fangjun Jiang]

Utilize Tall Arrays ?

Deferred Evaluation of Tall Arrays

Answer 2

[Ayush Modi]

Hi Felix,

I understand you want to load only a portion of the data at once from large txt files.

You can achieve it using "readmatrix" function only by adding additional parameter "Range". Here is an example to demonstrate how you can achieve this:

limitedData = readmatrix('filename.txt',"Range",[1,1,3,3])

Please refer to the following MathWorks documentation for more information on the “readmatrix” function:

https://www.mathworks.com/help/matlab/ref/readmatrix.html

Hope this helps!