import text file using importdata

Hello everyone,

When I try to import text files with experiment infomation using importdata, it seems to only import a couple of lines from the .text file.

I use the following code that looks for the (attached) readme file

data_folder=dir(data_path); %data_path opens a participants folder with mulitple files including the readme file   
for log=1:length(data_folder)
    if endsWith( data_folder(log).name , 'README.txt' )
    logfile_1 = importdata([data_path data_folder(log).name]);
    end
end

Instead of loading the whole file I get the attached logfile_1 file. Where if you open the logfile_1.textdata we only have 3 rows and those are not complete. When googling solutions to loading .txt files this is I believe the way to go. I have tried loading it as a table, but that messes too much with the structure.

Does anyone have any idea why this happens and how I can force it to import the whole file?

Thank you for your time,

Douwe

Answer 1

[Chunru]

You data format is a special format. You can consider to read them as text and then extract individual information. Otherwise you have to use commands like fscanf to read and parse data line by line. importdata can not tackle this kind of data format.

s = fileread("1111 F.A.S.T. Response Task 0-00-0000 README.txt");
s

s = 
    '~~~~~~~~~Run in Presentation V20.1~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     Participant ID:	11111
     Control
     Date:0-00-0000				
     Gender: M			Age: 9999		Handedness:R 		Temperature:
     Glasses/contacts:NO		Medication:NO		Height:		
     Weight:				Exp: xx			booth:
     Externals: (normal 1,2,3,4)
     External location: 1 Right eye, 2 left eye, 3 right mastoid, 4 left mastoid
     Light:on			(normal on)
     Screen:97 		(!!!! normal at 97cm !!!!)
     Cap:yellow and red, 64 			(color + channels)
     Hearingtest:	  L	  R	(write N/A if hearing test is not done on EEG day)
     	500hz 	 25dB	 0 dB	
     	1000hz 	 10dB	 10dB
     	2000hz 	 5 dB	 10dB
     	4000hz 	 5 dB	 10dB
     Vision Test:	 L  	R   Both  (write N/A if vision test is not done on EEG day)
     		20/16 20/20 20/20 
     notes:some alpha. poz, cms, o1, iz, cz were regelled before the task begun 
     Save as: ID#_fast_1
     ~~~~~~~~~~Triggers~~~~~~~~~~
     120 - Repeat faces,   20 - New Face
     130 - Repeat Objects, 30 - New Object 
     1 - button press
     Sequence: 
     20 or 30 should not be followed by a 1
     120 or 130 should always be followed by a 1
     ~~~~~~~~~~Triggers~~~~~~~~~~
     Practice:good
     Block 1:faces - data good - response good - et good
     Block 2:faces - data good - response good - et good 
     Block 3:objects - data good - response good - et good 			 
     Block 4:faces - data good - response good - et good 			
     Block 5: objects - data good - response good - et not good		
     Block 6: objects- data good - response good - et good 
     Block 7: faces - data good - response good - et good 
     Block 8: objects- data good - response good - et good 
     Block 9: faces- data good - response good - et good 
     Block 10: faces - data good - response good - et good 
     Block 11: objects- data good - response good - et good 
     Block 12: objects- data good - response good - et good 
     			 '

[Douwe Horsthuis]

Thank you, this works like a dream!

Answer 2

[Image Analyst]

This is a custom file format that simple generic readers can't handle. You'll have to write your own reader function for this type of file.

allLines = readlines('1111 F.A.S.T. Response Task 0-00-0000 README.txt')

allLines = 43×1 string array
    "~~~~~~~~~Run in Presentation V20.1~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
    "Participant ID:→11111"
    "Control"
    "Date:0-00-0000→→→→"
    "Gender: M→→→Age: 9999→→Handedness:R →→Temperature:"
    "Glasses/contacts:NO→→Medication:NO→→Height:→→"
    "Weight:→→→→Exp: xx→→→booth:"
    "Externals: (normal 1,2,3,4)"
    "External location: 1 Right eye, 2 left eye, 3 right mastoid, 4 left mastoid"
    "Light:on→→→(normal on)"
    "Screen:97 →→(!!!! normal at 97cm !!!!)"
    "Cap:yellow and red, 64 →→→(color + channels)"
    "Hearingtest:→  L→  R→(write N/A if hearing test is not done on EEG day)"
    "→500hz → 25dB→ 0 dB→"
    "→1000hz → 10dB→ 10dB"
    "→2000hz → 5 dB→ 10dB"
    "→4000hz → 5 dB→ 10dB"
    "Vision Test:→ L  →R   Both  (write N/A if vision test is not done on EEG day)"
    "→→20/16 20/20 20/20 "
    "notes:some alpha. poz, cms, o1, iz, cz were regelled before the task begun "
    "Save as: ID#_fast_1"
    "~~~~~~~~~~Triggers~~~~~~~~~~"
    "120 - Repeat faces,   20 - New Face"
    "130 - Repeat Objects, 30 - New Object "
    "1 - button press"
    "Sequence: "
    "20 or 30 should not be followed by a 1"
    "120 or 130 should always be followed by a 1"
    "~~~~~~~~~~Triggers~~~~~~~~~~"
    "Practice:good"

Then have a loop where you check for certain words in each line (might use contains for that) , and then parse the found line to extract the particular words or numbers you want.

[Douwe Horsthuis]

Thank you for that last suggetion. I ended up up adding loops like this to do extract the info:

if contains(logfile_1,'Age:')
    Age=extractBetween(logfile_1, 'Age:', 'Hand');
    Age = strtrim(Age);% deleting tabs, then deleting spaces
    Age=strcat('Age: ',Age);
end

It still sometimes leaves in a tab of something, but for most cases it works perfect.

[Image Analyst]

OK, glad it works for you. But how did the answer you accepted instead actually parse out all the stuff for you? You said that "works like a dream" but, to me, it looks like it merely just sucks up the whole file into a single character array and not giving you things like Age.

[Douwe Horsthuis]

Right I agree with you, that it just turns the txt file into a long line of characters.

But that allowed me,to find pieces based on a start and ending point that would always be in the data.

for example :

Age=extractBetween(logfile_1, 'Age:', 'Hand');

Will give me the Age if it is filled out.

Which is what I was looking for and prefer better than creating a whole function. In my case I need to find specifics within the text, so as long as I was able to import it all, I could do.

Which is not something I was able to do with my original importdata([data_path data_folder(log).name]);.

So it solved my problem, of importing the .txt file as a whole, like a dream