textscan problem (seperate data and sring)

hi: I have a large file, the first two lines are strings, and the third to end (such as 10000th line) are data.

I use textscan to read these data to a cell. but I do not know how to seperate them fastly.

my code is:

     fid=fopen('test.txt','rt');
     g = textscan(fid,'%s','delimiter','\n');
     A=g{1,1};

here I have a cell matrix of A with about 20 million lines. the first two lines are easy to operate. the problem is how to convert the rest lines into a data matrix.

if I use the for loop:

     for i=3:1:end
        result(i-2,:)=str2num(A);
     end

it may cost a lot of time. I also tried str2double, but still a lot of time.

so my question is that is there a better way to read the file at the beginning:

1. read the first two lines of strings to A.
2. read the rest data to a data matrix instead of a cell matrix to B.

thanks! Li

Answer 1

[KSSV]

Read the documentation of textscan. There is an option of specifying header Lines which skips those lines and read the data you want. You can also specify your format of output...For your problem give head lines 2 and format as %f..you will get all numbers. Read the documentation.

[Yu Li]

hi: thanks for your reply.

my problem is that there are more than 100 columns in the data matrix. so do I have to type 100 %f to read them?

meanwhile, the data matrix size differs from one file to another, so typing the actual number of %f looks not a good way.

do you have more suggestions?

thanks! Li

[Stephen23]

@Yu Li: please do not put leading space in front of every line of your comment: doing so formats the text as code, which is not really very easy to read.

"so do I have to type 100 %f to read them?"

No, just use repmat:

fmt = repmat('%f',1,100);

"the data matrix size differs from one file to another, so typing the actual number of %f looks not a good way"

You can easily determine the number of columns automatically: read the first few lines using fgetl, determine the number of columns by parsing that string, rewind back to the start using fseek, and then use textscan.

Or as an alternative use dlmread.