How to read data of files which are in the same folder

Hi all,

I've been searching on the internet without any succeed...

1. I want to read files which are in a directory and then read data contaiedn in these files (there are columns).
2. Below is my script :

cd 'C:\Users\'
folder=('C:\Users\')
txt=dir(fullfile('dry','*.txt'))

% creation of a matrice that contains paths of files
for i=1:length(txt)
  fileName(i,:)=fullfile(folder,txt(i).name)
end

% reading of data which is contained in these files
for i=1:length(fileName)
  M(i,:)=importdata(fileName(i,:))
end

Very easy but it doesn't work ... It says : "Subscripted assignment between dissimilar structures."

Thanks for your help ! Florian

Answer 1

[Florian]

Many thanks it works !

But so sorry, there's a new error I'm trying to fix from an hour. Because of a header, I've coded as below : delimiterIn=' '; headerLines=11; M=[] ;

    for i=1:length(txt)
      A=importdata(txt(i).name,delimiterIn,headerLines);
      M=[M A];
    end

Then I'm trying to view data by writing M.data(1,1) for instance and then I get this message : "Field reference for multiple structure elements that is followed by more reference blocks is an error."

Thank you so much for your help ! Florian

[Florian]

Partly resolved, but not completly :

      for i=1:length(txt)
         A=importdata(txt(i).name,delimiterIn,headerLines);
          M=[M A.data ];
      end

Now the message is :

The following error occurred converting from double to struct: Error using struct Conversion to struct from double is not possible.

Any idea ?

[dpb]

Well, you're trying to concatenate whatever M was/is with a structure reference that is of unknown characteristics from what's been shown. Not terribly surprising there's a mismatch of some sort.

If it were me, I'd look at the actual data format of the file and read it directly--I've not used importdata to know it otomh, sorry.

[Florian]

Nice, thank you very much for your kind help, I'll try with textscan ...

Bye Florian

[dpb]

Well, more info would help...do you know the size of the data in the file and are they all the same size?

I just looked at the doc for importdata -- you might try sotoo:

M=[];
for i=1:length(txt)
  M=[M;importdata(txt(i).name,delimiterIn,headerLines)];
end

That concatenates vertically so will need same number of columns in each and it creates the empty target array before the loop.

NB: if there are a large number of files and/or the files are large themselves, this automagic reallocation will be a real performance bottleneck as size grows as you're causing a new array and copy every pass thru the loop.

[Florian]

Hi,

I copy the code that I've written, it was finally easier with textscan, because my goal was to build a matrix of matrix and with importdata, because it's a structure, it seems being impossible.

txt=dir(fullfile(folder,'*.txt'));

for i=1:length(txt)
  block = 1

    fid=fopen(txt(i).name)
    inputText=textscan(fid,'%s',11,'delimiter','\n') % number of lines of header
    text=inputText{1} %stocke le header

    while (~feof(fid))   
        inputText=textscan(fid,'%f %f %f %f %f %f %f %f %f %f %f %f\n') % each '%f' corresponds to a column
        data{i,block,1}=cell2mat(inputText)
        block = block + 1
    end
    fclose(fid)
end

That way I've succeeded in building a cell like data{nb_of_files,1}(row,column) I can access more easily to my data !

Thanks Florian

[Image Analyst]

Please mark dpb's answer ( not yours) as "Accepted" .

Answer 2

[dpb]

Firstly, have you checked size() txt after the dir call? You don't want fullfile here --

>> fullfile('dry','*.txt')
ans =
dry\*.txt
>>

fullfile presumes the first argument is a directory and adds the delimiter. I've always thought it needed some more flexibility, but it is what it is...

Try sotoo--

folder=('C:\Users')
d=dir(fullfile(folder,'dry*.txt'));
for i=1:length(d)
  M(i,:)=importdata(d.name(i));
end

No need for the intermediary; just use the name field directly. It's one real advantage of cellstrings over character variables--you don't need the ':' indexing to get the full string.

Now your at least down to whether the data is of the right form and all...

[Florian]

Hi ! Thank you very much for you answer ! But there's still something wrong with the fullfile function. It doesn't "create" the entire path to the file, it just take into account the name and the extension of the file. Though when I use it into the loop, necessarily I have the answer "Unable to open file."

Do you have an idea ?

And in the loop I think it's rather :

for i=1:length(d)
   M(i,:)=importdata(d(i).name);
end

[dpb]

fullfile concatenates the pieces you give it--

>> folder=('C:\Users');
(fullfile(folder,'dry*.txt'))
ans =
C:\Users\dry*.txt
>>

That's a fully-qualified path and wildcard filename to pass to dir Presuming that's where your files are, that's what you need for it.

But, indeed the .name field returned by dir doesn't include the path so you do need to prepend it again--my bad for forgetting/overlooking the point.

And, you're also correct in that it's d that's the structure array to reference into inside the loop.

So

M(i,:)=importdata(fullfile(folder,d(i).name));

looks like it should be correct (if I've not blown something else... :) )