Problem keeping input file in the same order using textscan

I have a csv file with about 2000 rows. The format of the file is below.

I use this code to read in the file and parse out the input vectors

fid = fopen('temp.csv');
data = textscan(fid,'%n%n%f', 'headerlines', '1', 'delimiter', ',');
tradeDate=data{:,1};
stockId = data{:,2};
price = data{:,3};

I then calculate the returns using this code returns = tick2ret(price);

As a sanity check I list the first four prices and they are not the same as the first four items in the text file. These records are somewhere in the middle of the file.

so price(1) is not equal to the first record in the input file price (2) is not equal to the second record and so on.

So return(1) does not match what the first return should be.

================INPUT FILE ================= t_date stock_id price_low 37623 16622 38.035 37624 16622 38.665 37627 16622 38.885 37628 16622 38.82 37629 16622 38.6 37630 16622 38.99 37631 16622 38.855 37634 16622 39.09 37635 16622 39.26 37636 16622 39.075 37637 16622 39.19 37638 16622 38.45 37642 16622 38.04

Answer 1

[Matt Tearle]

Ahahahahaha! I'm gonna take a bet that price(1) is actually the price from the 49th row, right?

You'll love this. Instead of

data = textscan(fid,'%n%n%f', 'headerlines', '1', 'delimiter', ',');

try

data = textscan(fid,'%n%n%f', 'headerlines', 1, 'delimiter', ',');

See the difference? It's subtle... 1 ~= '1'!

[Walter Roberson]

Good catch, Matt.

[luke]

Ok that works perfectly! but can you tell me why it was from the 49th row?

[Matt Tearle]

Heehee. I was leaving that as a brainteaser. Enter >> double('1') at the command prompt and you'll see.

[Fuller explanation: textscan expects a numeric value there; it got a string; so it converted...]

[luke]

That was a tricky bug. I am surprised I didn't get a warning message letting me know that a string was being converted to a double

[Elsa Birch]

Hey Tearle! I stumbled upon this while trying to solve a problem about importing .csv files, and remember you emphasizing awareness of this in class. - Elsa

[Matt Tearle]

Wow, someone actually listened to me babbling on in class? I've seen many cases of weird char2double issues, but this specific one was new to me. For the record, I've submitted an enhancement request that this behavior be changed, and it's under consideration.

Answer 2

[Walter Roberson]

It works when I try it.

I note though that the text file you show as input does not match your format description: you set 'delimiter' to ',' as would be common for csv files, but the input file you presented has no commas.

Also, you should specify 'rt' as an additional option on fopen() as you are dealing with a text file.