Read files from multiple folders

Hi,

Lets say I have some 100 .dat files in each subfolder A, B , C , D that is placed inside the folder named Data. Here, I would like to perform operation in a one go, rather than running sperately for each subfolder. Is there a way to do this, does it requires parallel processing for this task ??

For e.g.

read data from subfolder A

for i = 1:1:100

some operation

end

read data from subfolder B

for i = 1:1:100

some operation

end

Answer 1

[Image Analyst]

Use a fileDatastore:

topLevelFolder = pwd;       %  folder in which your images exists
filePattern = fullfile(topLevelFolder, '*.dat');
ds = fileDatastore(filePattern, 'ReadFcn', @readmatrix)         %  Creates a datastore for all images in your folder
allFileNames = {ds.Files{:}}'
for k = 1 : numel(allFileNames)
    thisFileName = allFileNames{k};
    fprintf('Processing %s.\n', thisFileName);
    % Do something with it.
end

If you have the Parallel Processing Toolbox you can use parfor instead of for.

[Turbulence Analysis]

Thanks.. Got the below error!!

Error using fileDatastore (line 102)
Function readmatrix does not exist.

[Image Analyst]

You must have a version older than r2019a. Just use csvread instead of readmatrix and it should work, as long as you have a version later than 2016a.

ds = fileDatastore(filePattern, 'ReadFcn', @csvread);

Or you can also try it with dir using two stars.

topLevelFolder = pwd;       %  folder in which your data files exist.
filePattern = fullfile(topLevelFolder, '**\*.dat');
fileList = dir(filePattern);
allFileNames = fullfile({fileList.folder}, {fileList.name})
for k = 1 : numel(allFileNames)
    thisFileName = allFileNames{k};
    fprintf('Processing %s.\n', thisFileName);
    % Do stuff with the file, like read the data and process it somehow.
end

[Turbulence Analysis]

It's working perfectly with two stars !!