I went to the Matlab expo in Silverstone last week and quite a few of the demonstrations made use of the datastore function for processing large data files.
From what I could tell however, these examples all seemed to use CSV files.
Is there anyway to do this with binary data? We almost exclusively store data using float32 binary data format. I've typically read blocks in using fread but I'd like to streamline this if possible.
I tried making using fileDatastore to use my own read function, but this hung the program and I'm assuming this was due to the function reading the whole file in.
fds = fileDatastore(filename,'ReadFcn',@testfcn);
function output = testfcn(filename)
fid = fopen(filename,'r');
So is there anything I can do? I really want to take advantage of things like tall arrays and the plots which increase in resolution as the data is read.