Reading and Processing Many Text Files

Hi,

I am working on a project that uses Fortran to perform high intensity calculations and writes the results as text files, at certain time step intervals. To get a sense of how the properties of my simualted domain change over time, the Fortran code creates results (.txt) files every X amount of time steps. The files are labelled RESULTS_ONE_XX and RESULTS_TWO_XX, where XX is a number between 15 and 99.

I am using MATLAB to postprocess and visualise the data, particularly the contour function. I have sucessfully made MATLAB read and process a single file, using the file directory, fopen and textscan (working code below).

What I would like to do is to make MATLAB read, for example, RESULTS_ONE_15, then 16, 17 ... 99. Is there a way to use a For loop to automate the reading of the files? I also want to generate contour plots at every results time step, which can be overwritten (creating somewhat of an animation). I can figure that part out, but I am having difficulty attempting to automate the reading process.

My main issue currently is trying to figure out how to make the file directory dynamic but still valid - I've tried something along the lines of

for i=1:X
fid1 = fopen('C:*****\RESULTS_ONE_(i).txt')
end 

But obviously that doesn't work - no surprises there! Would I need to make dynamic variables too? I've read that that is never a great idea but if it is the only option I would definitely consider it. Either way, I still don't know what to put in the file directory!

If anyone has any advice then that would be greatly appreciated.

Many Thanks,

JP

*** WORKING CODE BELOW ***

fid1 = fopen('C:***FILE LOCATION***\RESULTS_ONE_15.txt');  % OPEN FILE
A = textscan(fid1,'%f %*f %*f %*f %*f');  % Read First Column
Col_1 = cell2mat(A);                      % Convert to List
List_1 = Col_1(1:N_Nodes);                % Select the First Column
Q_T = reshape(List_1,R_Nodes,D_Nodes);    % Convert List to Array (Requires Transposing)
Q = Q_T';                                 % Transpose

Answer 1

[Chunru]

foldername = 'C:***FILE LOCATION***';         % True folder name needed
fileprefix = 'RESULTS_ONE_';
for i=1:N                                     % N is number index to file name
    % Combine foldername, fileprefix, and file number to form the full file
    % name
    fn = fullfile(foldername, sprintf('%s%d.txt', fileprefix, i));
    
    fid1 = fopen(fn);                         % OPEN FILE
    A = textscan(fid1,'%f %*f %*f %*f %*f');  % Read First Column
    Col_1 = cell2mat(A);                      % Convert to List
    List_1 = Col_1(1:N_Nodes);                % Select the First Column
    Q_T = reshape(List_1,R_Nodes,D_Nodes);    % Convert List to Array (Requires Transposing)
    Q = Q_T';                                 % Transpose
end

[KSSV]

You must close the opened file.......

[Joseff Parke]

Brilliant thank you! That was a lot easier than I was imagining! Works perfect :)

[Stephen23]

@Joseff Parke: this answer is missing FCLOSE.

Having many open files will slow down your OS, slow down MATLAB, and can cause MATLAB to crash.

[Joseff Parke]

Hi Both,

Ok Brilliant thanks for your help, I'll include that too!

Answer 2

[KSSV]

txtFiles = dir('*.txt') ; 
N = length(txtFiles) ; 
for i = 1:N
    thisFile = txtFiles(i).name ; 
    fid1 = fopen(thisFile);  % OPEN FILE
    A = textscan(fid1,'%f %*f %*f %*f %*f');  % Read First Column
    Col_1 = cell2mat(A);                      % Convert to List
    List_1 = Col_1(1:N_Nodes);                % Select the First Column
    Q_T = reshape(List_1,R_Nodes,D_Nodes);    % Convert List to Array (Requires Transposing)
    Q = Q_T';                                 % Transpose
    fclose(fid1) ; 
end

[Joseff Parke]

Amazing thanks for your help, I accepted the other answer purely as there were more comments but yours was equally as helpful!