How to extract data with multiple headerlines locations, with a differing number of headerlines per file?

All,

Been looking for a solution for a while, but have caved for some help.

Basically i need to be able to extract the latter half of the data from the file i have included below. It seems like the grand majority of Matlab commands seem to only extract the first set of numeric data, which is not what i am interested in extracting.

Does anyone have any suggestions as to how to tackle this? I need to collect all data below the 'ZCURVE' text.

Here is where this gets challenging:

• The number of headerlines above the data of interest varies.
• Most MATLAB 2017 commands only extract the first set of data, which is not data of interest.

I have trimmed the file, it is normally a lot longer and is normally in .dta format.

I greatly appreciate any and all help on the matter.

----simplified sample file-----

EXPLAIN

TAG EISPOT

TITLE LABEL Potentiostatic EIS Test &Identifier

DATE LABEL 8/3/2021 Date

TIME LABEL 16:52:26 Time

NOTES NOTES 1 &Notes...

PSTAT PSTAT REF600-11020 Potentiostat

VDC POTEN 0.00000E+000 T DC &Voltage (V)

FREQINIT QUANT 1.00000E+006 Initial Fre&q. (Hz)

FREQFINAL QUANT 1.00000E+000 Final Fre&q. (Hz)

PTSPERDEC QUANT 1.00000E+001 Points/&decade

VAC QUANT 2.50000E+001 AC &Voltage (mV rms)

AREA QUANT 4.00000E-005 &Area (cm^2)

CONDIT TWOPARAM F 5.00000E+000 0.00000E+000 Conditionin&g Time(s) E(V)

DELAY TWOPARAM F 5.00000E+000 0.00000E+000 Init. De&lay Time(s) Stab.(mV/s)

SPEED SELECTOR 0 &Optimize for:

ZGUESS QUANT 1.00000E+004 E&stimated Z (ohms)

OCVCURVE TABLE 39

Pt T Vf Vm Ach Over

# s V vs. Ref. V V bits

0 0.258333 6.59852E-002 6.59852E-002 4.84743E-004 ...........

1 0.516667 6.59860E-002 6.60275E-002 4.85476E-004 ...........

2 0.775 6.59900E-002 6.60169E-002 4.80143E-004 ...........

EOC QUANT 0.0657299 Open Circuit (V)

PSTATMODEL IQUANT 4 Pstat Model

PSTATSECTION LABEL REF600-11020 Pstat Section

PSTATSERIALNO LABEL 11020 Pstat Serial Number

CTRLMODE IQUANT 1 Control Mode

IESTAB IQUANT 0 I/E Stability

[more of this stuff]

INSTRUMENTVERSION LABEL 3.50 Instrument Version

ZCURVE TABLE

Pt Time Freq Zreal Zimag Zsig Zmod Zphz Idc Vdc IERange

# s Hz ohm ohm V ohm ° A V #

0 1 1000078 756.6094 -1569.005 1 1741.905 -64.25559 -3.171526E-007 0.0683075 8

1 2 794390.6 907.2675 -1893.579 1 2099.708 -64.39959 1.011809E-006 0.067796 8

[Star Strider]

It would help to have examples of at least two different files to experiment with. If they have the desired headers and data in common, that would make this considerably easier. If not, it might not be possible.

I generally use textscan for these problems, because I’m used to using it for them.

.

[Suzette Mastrangelo]

I have tried that, but unfortunately it stops reading after the first data set.

Answer 1

[Star Strider]

My apologies, however this has to be the most inelegant code I ever wrote.

It has the virtue of being able to read the posted file, however unless the others are exactly like it, it’s unlikely to work with any others.

It works first by reading through the file and getting the beginning line of the numeric arrays, and then uses a second loop to read them.

What complicates this is that the two numeric arrrays have different numbers of columns, so different textscan calls are required for each one, explaining the presence of the if block.

fidi = fopen('sample.txt','rt');
poundsign = [];
k2 = 1;
startline = [];
while ~feof(fidi)
    s = fgetl(fidi)
    poundsign = strfind(s,'#')
    if ~isempty(poundsign)
        startline = [startline k2]
    end
    poundsign = [];
    k2 = k2 + 1;
end
startline

for k1 = 1:numel(startline)
    fseek(fidi, 0, 'bof')
    for k = 1:startline(k1)
        s = fgetl(fidi);
    end
    if k1 == 1
        C{k1} = textscan(fidi, '%f%f%f%f%f%*s', 'HeaderLines',0, 'CollectOutput',true);
    else
        C{k1} = textscan(fidi, '%f%f%f%f%f%f%f%f%*s', 'HeaderLines',0, 'CollectOutput',true);
    end
end
M1 = cell2mat(C{1});                                    % First Numeric Array
M2 = cell2mat(C{2});                                    % Second Numeric Array

Have fun with the other files! This code may need to be changed to work with them if they are significantly different from this one.

.

[Image Analyst]

Looks fine to me. Custom readers for quirky input files often are somewhat complicated.

(Adding attachments for completeness.)

Answer 2

[Image Analyst]

Generally for something this complicated, you need to write a custom reader for the file.

Attach a sample data file and say what, out of all that tons of stuff, you want to extract. Perhaps you could use fgetl() to pull out one line at a time and parse it to figure out what that line means and assign it to the right field or variable.

[Suzette Mastrangelo]

Sure!

Yes, i have tried using fgetl() but i think i have been doing something incorrectly.

I have attached one of the files.