Process data from sequentially named folders each having a file with the same name

I have a set of sequentially named folders each containing a single file with the same name called event.txt that I wish to process (plot and save graphs as names of folders). Any ideas?

Here's what I have so far…

F = 'Event.txt';
J = ('C:\Users\myname\SM_Results\AnalysisCase.1');
S = dir(fullfile(J,'C1R*'));
X = [S.isdir] & ~ismember({S.name},{'.','..','AnalysisCase.1.C1R*'});
files = {S(X).name};
numfiles = length(X);
mydata = cell(1, numfiles);
 
for k = 1:numel(files)
  existfilename = fullfile(J,files{k},F)
  if exist(existfilename)
  mydata{k} = importdata(existfilename);
  %now process the data ...
  data = readtable(existfilename);
  time=data{:,'Time'};
  Ty=data{:,'Expression_Point_E_Ty_value_mm_'};
  [~,fileName,~] = fileparts(sprintf('Event_%d', k));% <-need help here to rename properly  
  figure
  plot(time,Ty,'-')
  xlabel('time, s')
  ylabel('Ty (mm)')
  title(fileName)
 %  savefig(fileName)
  print(fileName,'-dpng')

[Stephen23]

@david ocallaghan: what exactly is your question? Do you have a specific problem that you wish for us to help you on?

You should revise and clarify your code, otherwise some of it does not make much sense:

• what are the names of folders?
• why parentheses around the J character vector?
• what is 'AnalysisCase.1.C1R*' supposed to match? (the function ismember interprets all characters literally, there are no wildcards).
• files is a very misleading name.
• why do you import the data twice?
• what is fileparts supposed to do?
• please align your code consistently and format it properly on this page using the code formatting buttons.

I have a feeling that your folder names are actually something like "AnalysisCase.1.C1R" in which case your use of fullfile will not work. But this is just a guess based on your ismember usage. Please clarify the folder name pattern.

[Stephen23]

david ocallaghan's "Answer" moved here:

• what are the names of folders?

C:\Users\DOCALLAGHAN\SM_Results\AnalysisCase.1\C1R1\Event.txt

C:\Users\DOCALLAGHAN\SM_Results\AnalysisCase.1\C1R2\Event.txt

up to

C:\Users\DOCALLAGHAN\SM_Results\AnalysisCase.1\C1R32\Event.txt

• why parentheses around the J character vector?

ok not needed, thanks

• what is 'AnalysisCase.1.C1R*' supposed to match?

where Event.txt does not exist theres three files AnalysisCase.1.C1R1.LMSMotionInfo, AnalysisCase.1.C1R1.LMSMotionResults andAnalysisCase.1.C1R1.LMSMotionSolverInput

• files is a very misleading name.

ok wil take another look. Printing existfilename does show the correct file path to .../Event.txt

• why do you import the data twice?

hmm, I assume importdata and readable are doing the same thing?

• what is fileparts supposed to do?

wanted to rename Event.txt to Event_1.txt and name plots Event_1.png etc for each.

• please align your code consistently and format it properly on this page using the code formatting buttons.

ok I asssume this for the if statement.

Thanks for your first glance. The code in its current form processes ../C1R20/Event.txt and renames it and the plot Event_13.txt and Event_13.png respectively and I know k is not linking to the sequentially numbered folders (pain)

[Stephen23]

Thank you for giving the subfolder names.

"where Event.txt does not exist theres three files AnalysisCase.1.C1R1.LMSMotionInfo, AnalysisCase.1.C1R1.LMSMotionResults andAnalysisCase.1.C1R1.LMSMotionSolverInput"

Sure. But your dir call does not look inside the subfolders where those files might be, it only looks in the main folder itself. And ismember does not have any "wildcard" characters.

"ok wil take another look."

files is misleading because they are names of folders, not files.

"wanted to rename Event.txt to Event_1.txt and name plots Event_1.png etc for each."

Just use sprintf to generate the required name directly.

Question: do you want the subfolders processed in numeric order, i.e. 1, 2, ...32, or is character order 1,10,...,2,20,,, 32, acceptable?

[Stephen23]

david ocallaghan's "Answer" moved here:

"your dir call does not look inside the subfolders " yes of course thanks!

files as folders, yes I see, thanks.

ok will use sprintf, tried earlier but I didnt name it correctly (maybe because k wasnt matching numerical folder order)

I'd like the subfolder processed in numerical order - this will be a big fix allowing k to match C1R(k)

Thanks for your help, very insightful

Answer 1

[Stephen23]

To get the subfolders in numeric order download my FEX submission natsortfiles:

https://www.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort

And use it something like this:

F = 'Event.txt';
P = 'C:\Users\myname\SM_Results\AnalysisCase.1';
S = dir(fullfile(P,'C1R*'));
S = natsortfiles(S([S.isdir]));
for k = 1:numel(S)
    Z = fullfile(P,S(k).name,F);
    if 2==exist(Z,'file')
        A = importdata(Z);
        S(k).data = A;
        ... your code
        G = sprintf('Event_%d.png',k);
        ... your code
        print(G,...)
    end
end

I doubt that you should make a new figure on every loop iteration.

Note that an alternative to natsortfiles would be to process the files in the order provided by your OS, extract the second number from the subfolder and use it as the index (instead of k).

Note that the behavior of dir changes if there is only one matching subfolder.

Answer 2

[david ocallaghan]

Thanks you so much Stephen awesome!