MATLAB Answers

DK
0

Generate a Datastore for binary data?

DK
さんによって質問されました 2017 年 10 月 13 日
最新アクティビティ Rick Amos
さんによって 回答されました 2017 年 10 月 17 日
I went to the Matlab expo in Silverstone last week and quite a few of the demonstrations made use of the datastore function for processing large data files.
From what I could tell however, these examples all seemed to use CSV files.
Is there anyway to do this with binary data? We almost exclusively store data using float32 binary data format. I've typically read blocks in using fread but I'd like to streamline this if possible.
I tried making using fileDatastore to use my own read function, but this hung the program and I'm assuming this was due to the function reading the whole file in.
fds = fileDatastore(filename,'ReadFcn',@testfcn);
function output = testfcn(filename)
fid = fopen(filename,'r');
fread(fid,'float32');
fclose(fid);
end
So is there anything I can do? I really want to take advantage of things like tall arrays and the plots which increase in resolution as the data is read.
Thank you

  0 件のコメント

サインイン to comment.

2 件の回答

Rick Amos
回答者: Rick Amos
2017 年 10 月 17 日

As of R2017b, MATLAB now allows you to write custom datastores. This will allow you to create datastores that handle large binary files, without requiring to read entire files in one go.
See the documentation page for Develop Custom Datastore for more details.

  0 件のコメント

サインイン to comment.


dpb
回答者: dpb
2017 年 10 月 13 日

I've not used it much but have read some of the doc...see <What is a datastore> for an overview of what can use; there's a table at the end. It appears you would have two choices -- either a .mat file with key values that KeyValueDatastore will operate with or a custom file format for which you supply the read function with FileDatastore .
Perhaps with stream data, what you're really looking for is memmapfile

  0 件のコメント

サインイン to comment.



Translated by