How can I import multiple .out files in a single folder as separate tables or arrays?

I want to import these multi-column files into MATLAB. I tried the code attached, however the following error prompts.

Error using matlab.io.ImportOptions/readtable (line 503)

Unable to find or open 'xw2-rfile.out'. Check the path and filename or file permissions.

Error in mat6mm (line 46)

vxw{i} = readtable(filename,opts);

xw = [2,5,8,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,40,60,80,100];
vxw = zeros(size(xw));
%vxw2-rfile%
for i = 1:1:size(xw)
    xwstr = string(xw(i));
    filename = strcat('xw',xwstr,'-rfile.out');

    opts = delimitedTextImportOptions("NumVariables", 3);
    opts.DataLines = [4, Inf];
    opts.Delimiter = " ";
    opts.VariableNames = ["Time Step", "Velocity (m/s)", "Flow Time (s)"];
    opts.VariableTypes = ["double", "double", "double"];
    opts.ExtraColumnsRule = "ignore";
    opts.EmptyLineRule = "read";
    opts.ConsecutiveDelimitersRule = "join";
    opts.LeadingDelimitersRule = "ignore";

    vxw{i} = readtable(filename,opts);
    clear opts
end

[Mathieu NOE]

hello

can you share one file to let us test the code ?

[Andre Amare]

Hi, here are some sample files.

Answer 1

[Stephen23]

unzip reports
P = '.'; % absolute or relative path to where the files are saved
S = dir(fullfile(P,'vxw*.out'));
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    T = readtable(F, 'FileType','delimitedtext', 'NumHeaderLines',3);
    T.Properties.VariableNames = ["Time Step", "Velocity (m/s)", "Flow Time (s)"];
    S(k).data = T;
end

All imported file data are stored in the structure S, together with the filenames:

S.data

ans = 903x3 table
    Time Step    Velocity (m/s)    Flow Time (s)
    _________    ______________    _____________

        0                 0                 0   
        1            38.783           0.00125   
        2            32.244            0.0025   
        3            29.822           0.00375   
        4            22.325             0.005   
        5            15.228           0.00625   
        6            10.821            0.0075   
        7            7.4653           0.00875   
        8            4.8391              0.01   
        9            3.1796           0.01125   
       10             2.375            0.0125   
       11            2.0878           0.01375   
       12            2.0821             0.015   
       13            2.2287           0.01625   
       14            2.4675            0.0175   
       15            2.7645           0.01875   

ans = 903x3 table
    Time Step    Velocity (m/s)    Flow Time (s)
    _________    ______________    _____________

        0                  0                0   
        1              42.42          0.00125   
        2             43.135           0.0025   
        3              28.47          0.00375   
        4             13.876            0.005   
        5             3.8513          0.00625   
        6             -1.947           0.0075   
        7            -5.0266          0.00875   
        8            -6.5324             0.01   
        9            -7.0503          0.01125   
       10            -6.8511           0.0125   
       11            -6.0375          0.01375   
       12            -4.6509            0.015   
       13            -2.7659          0.01625   
       14           -0.50731           0.0175   
       15             2.2569          0.01875   

ans = 903x3 table
    Time Step    Velocity (m/s)    Flow Time (s)
    _________    ______________    _____________

        0                  0                0   
        1             59.177          0.00125   
        2             37.241           0.0025   
        3             27.487          0.00375   
        4             14.306            0.005   
        5             5.1833          0.00625   
        6             1.4645           0.0075   
        7            0.21347          0.00875   
        8            -0.2766             0.01   
        9           -0.56677          0.01125   
       10           -0.72735           0.0125   
       11           -0.66859          0.01375   
       12           -0.32446            0.015   
       13            0.28777          0.01625   
       14             1.1088           0.0175   
       15             2.0586          0.01875   

ans = 903x3 table
    Time Step    Velocity (m/s)    Flow Time (s)
    _________    ______________    _____________

        0                 0                 0   
        1             56.49           0.00125   
        2            43.074            0.0025   
        3            28.096           0.00375   
        4             17.62             0.005   
        5              11.8           0.00625   
        6            7.0652            0.0075   
        7            3.8357           0.00875   
        8            2.1461              0.01   
        9            1.3706           0.01125   
       10            1.0638            0.0125   
       11            1.0081           0.01375   
       12              1.14             0.015   
       13            1.4259           0.01625   
       14            1.8361            0.0175   
       15             2.329           0.01875   

[Mathieu NOE]

@Andre Amare

now you have plenty of solutions to play with...

a few minor things :

1/ Could be useful (generally) : sort file names into natural order ( guaranted , not like the regular dir output) - it can save you some headaches in the future....

then use this @Stephen23 another beautiful creation : Natural-Order Filename Sort - File Exchange - MATLAB Central

2/ if you have tons of files and you're looking for faster processing then a lower level import tool can save a bit of time (and no, I have no problem with readtable...) , so here I tried the older readclm (see attachment) vs the modern readtable. You could save a factor of 3 in your processing... up to you to test it if that matters .

and the variable names are here retrieved from the header in the file, no need to put them manually in the code .

P = 'C:\Users\A0H36019\Downloads\vxw'; % absolute or relative path to where the files are saved
S = dir(fullfile(P,'vxw*.out'));
S = natsortfiles(S); % sort file names into natural order , see : 
           %(https://fr.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort)

%% option 1 with readtable
P = 'C:\Users\A0H36019\Downloads\vxw'; % absolute or relative path to where the files are saved
S = dir(fullfile(P,'vxw*.out'));
S = natsortfiles(S); % sort file names into natural order , see : 
           %(https://fr.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort)

tic
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    T = readtable(F, 'FileType','delimitedtext', 'NumHeaderLines',3);
    T.Properties.VariableNames = ["Time Step", "Velocity (m/s)", "Flow Time (s)"];
    S(k).data = T;
end
toc

%% option 2 with readclm
clearvars
P = 'C:\Users\A0H36019\Downloads\vxw'; % absolute or relative path to where the files are saved
S = dir(fullfile(P,'vxw*.out'));
S = natsortfiles(S); % sort file names into natural order , see : 
           %(https://fr.mathworks.com/matlabcentral/fileexchange/47434-natural-order-filename-sort)

tic
for k = 1:numel(S)
    F = fullfile(P,S(k).name);
    [outdata,head] = readclm(F,3);
    VariableNames = split(head(end,3:end-2),'" "')';
    S(k).Data = outdata;
    S(k).VariableNames = VariableNames;
    
end
toc

[Andre Amare]

Thank you so much! This helps a lot.

[Mathieu NOE]

as always, my pleasure !

Answer 2

[dpb]

unzip reports.zip
d=dir('vx*.*');
for i = 1:numel(d)
  filename=fullfile(d(i).folder,d(i).name);
  vxw{i} = readtable(filename,'FileType','text');
  vxw{i}.Properties.VariableNames={'Time', 'Velocity','FlowTime'};
  vxw{i}.Properties.VariableUnits={'s', 'm/s','s'};
end
head(vxw{1})

    Time    Velocity    FlowTime
    ____    ________    ________

     0            0           0 
     1       38.783     0.00125 
     2       32.244      0.0025 
     3       29.822     0.00375 
     4       22.325       0.005 
     5       15.228     0.00625 
     6       10.821      0.0075 
     7       7.4653     0.00875 

summary(vxw{1})

903x3 table

Variables:

    Time: double (s)
    Velocity: double (m/s)
    FlowTime: double (s)

Statistics for applicable variables:

                NumMissing     Min       Median          Max           Mean            Std     

    Time            0           0           451            902            451        260.8179  
    Velocity        0           0        5.1484        48.0584        12.9954         14.4680  
    FlowTime        0           0        0.5637         1.1275         0.5637          0.3260  

Using a suitable wildcard for desired files makes the code more generic and removes the need to bury the "magic" constants in the code and to manually create filenames--only the matching root filename wildcard string is then needed besides the variable identification data.

I shortened variable names to not have embedded blanks so can use "dot" referencing of the table names without quoting for coding convenience and place the units in the variables metadata reserved for them...

The ".out" filename extension is not one of those recognized by MATLAB automagically as a text file, you have to tell the functions that specifically.

These files don't need an import object, but If you really, really wanted to use one, it could go outside the loop since all the files are the same format...and all can set is the variable names; the units metadata field is not supported..

opt=detectImportOptions(fullfile(d(1).folder,d(1).name),'FileType','Text');
opt.VariableNames={'Time', 'Velocity','FlowTime'}

opt = 
  DelimitedTextImportOptions with properties:

   Format Properties:
                    Delimiter: {' '}
                   Whitespace: '\b\t'
                   LineEnding: {'\n'  '\r'  '\r\n'}
                 CommentStyle: {}
    ConsecutiveDelimitersRule: 'split'
        LeadingDelimitersRule: 'keep'
       TrailingDelimitersRule: 'ignore'
                EmptyLineRule: 'skip'
                     Encoding: 'UTF-8'

   Replacement Properties:
                  MissingRule: 'fill'
              ImportErrorRule: 'fill'
             ExtraColumnsRule: 'addvars'

   Variable Import Properties: Set types by name using setvartype
                VariableNames: {'Time', 'Velocity', 'FlowTime'}
                VariableTypes: {'double', 'double', 'double'}
        SelectedVariableNames: {'Time', 'Velocity', 'FlowTime'}
              VariableOptions: [1-by-3 matlab.io.VariableImportOptions] 
	Access VariableOptions sub-properties using setvaropts/getvaropts
           VariableNamingRule: 'modify'

   Location Properties:
                    DataLines: [4 Inf]
            VariableNamesLine: 0
               RowNamesColumn: 0
            VariableUnitsLine: 0
     VariableDescriptionsLine: 0 
	To display a preview of the table, use preview

for i = 1:numel(d)
  filename=fullfile(d(i).folder,d(i).name);
  vxw{i} = readtable(filename,opt);
  vxw{i}.Properties.VariableUnits={'s', 'm/s','s'};
end
summary(vxw{i})

903x3 table

Variables:

    Time: double (s)
    Velocity: double (m/s)
    FlowTime: double (s)

Statistics for applicable variables:

                NumMissing     Min       Median          Max           Mean            Std     

    Time            0           0           451            902            451        260.8179  
    Velocity        0           0        4.4901        56.4899        13.8717         16.1153  
    FlowTime        0           0        0.5637         1.1275         0.5637          0.3260  

[Andre Amare]

Thank you for your answer!