Data not indexing properly?

Hi there,

I am compiling group data and plotting it with a best fit line, but am having issues with the way some of the output is indexing. For instance, I pull in the group data in a loop and concatenate it so I have all my data in one place. Then, I create an index for the values greater than zero and try to index into that later to make a new variable for best fit line plotting. Everything works until I do the last step, where it just seems to index into the 1st set of data I imported.

for isubj = 1:length(SUBJECTS)

    if isubj == 1

        qthres = xlsread(xlsfile, 1,'J3:J16');
        peakamp = xlsread(xlsfile, 1, 'L3:L16');
        scalae = xlsread(xlsfile, 1, 'C3:C16'); % can only use elecs 2-15 here because of qthres!

        [STi, STj, ST] = (find(scalae == 1 & peakamp > 0));
        [SMi, SMj,SM] = (find(scalae == 2 & peakamp > 0));
        [SVi, SVj,SV] = (find(scalae == 3 & peakamp > 0));

        [peakampi, peakampj] = find(peakamp > 0);

all the way to 8...

else isubj = 8

        qthres = xlsread(xlsfile, 8,'J3:J16');
        peakamp = xlsread(xlsfile, 8, 'L3:L16');
        scalae = xlsread(xlsfile, 8, 'C3:C16');

        [STi, STj, ST] = (find(scalae == 1 & peakamp > 0));
        [SMi, SMj,SM] = (find(scalae == 2 & peakamp > 0));
        [SVi, SVj,SV] = (find(scalae == 3 & peakamp > 0));

        [peakampi, peakampj] = find(peakamp > 0);

    end

....plot data which works fine...

    if ~isrow(peakamp(peakampi))
        peakamp(peakampi) = (peakamp(peakampi))';
    end

    allpeakamp = cat(2, allpeakamp, peakamp);
    allthres = cat(2, allthres, qthres);
    allscalae = cat(2, allscalae, scalae);

    [findampi, findampj] = find(allpeakamp > 0);
    [findthresi, findthresj] = find(allthres > 0);

...and end loop.

Both inside and outside the loop the above variables allpeakamp and findampi (for instance) look correct. However, when I try to do allpeakamp(findampi) so I can get values only above zero, it only indexes into subject 1's amplitudes, when it should be looking as a whole. I know something is wonky with how I"m doing it, but I can't put my finger on it. Essentially I need to compile the group amplitudes, find what's above zero, and use that to plot a best fit line against threshold...and the same issue is occurring with threshold as well.

Sorry this is so long. Thanks!

Answer 1

[Geoff Hayes]

Lindsay - your code after the plotting starts with the following if statement

 if ~isrow(peakamp(peakampi))
     peakamp(peakampi) = (peakamp(peakampi))';
 end

The condition checks to see if the data in peakamp indexed at peakampi is not a row, and then tries to replace the data with the transpose of that data? There will be no change to the vector peakamp - it will still be a column. Is the intent to modify the column vector to a row vector, or is it something else? Why only consider some of the elements in the vector (i.e. those referenced by peakampi) when the next line

 allpeakamp = cat(2, allpeakamp, peakamp);

uses all of the peakamp data?

-------------------

Given that

 peakamp = xlsread(xlsfile, 1, 'L3:L16');

reads a number of rows from one column of the Excel spreadsheet, it is going to be a column vector of negative and/or positive values. Then

 allpeakamp = cat(2, allpeakamp, peakamp);

is going to concatenate along the column dimension of allpeakamp the peakamp column vector. Over time, this becomes a 14x8 matrix.

Now when you do

 [findampi, findampj] = find(allpeakamp > 0);

look at the results for findampi and findampj. Both are column vectors, the first corresponding to the row indices (so will be values from 1-14, repeated for each subject that has data greater than zero) and the second corresponding to the column indices (so will be values from 1-8, repeated for each subject that has data greater than zero). Both vectors will be of the same length.

Now when you evaluate

 allpeakamp(findampi)

the code is only using the row indices and not using the column indices from findampj and so the result will only ever be repeated values from the first subject. Here is a small example

 A = [1 2 ; 3 4; 5 6];

The first column has odd numbers, the second even. Now we will do something similar

 [rowIdcs, colIdcs] = find(A>0)

 rowIdcs =

     1
     2
     3
     1
     2
     3

 colIdcs =

     1
     1
     1
     2
     2
     2

Clearly, all elements of A are positive, yet when we try

 A(rowIdcs)

 ans =

     1
     3
     5
     1
     3
     5

we only see the odd numbers from the first column. That is because when accessing a matrix with a single vector, there is an assumption that we are trying to access these elements using the linear index equivalents to the row and column subscripts. So the repetitive nature of peakampi means that we will only ever access elements from the first column of allpeakamp i.e. the first subject.

To get around this, convert the row and column subscripts of peakampi and peakampj respectively to their linear index equivalents using sub2ind as

 linIdcs = sub2ind(size(allpeakamp),peakampi,peakampj);

 allpeakamp(linIdcs) % to return all positive values across all subjects

Try replacing with the above two lines, and see what happens!

[Lindsay]

Thanks! This is wonderful.

So I tried what you said and it seems to sort of work, except that my amplitude index (ampind) and thres index (thresind) seem to only return the indices from the last subject run, whereas I need all those indices for when I plot a group best fit line. So now I have:

ampind = sub2ind(size(allpeakamp),peakampi,peakampj);

thresind = sub2ind(size(allthres),thresi,thresj);
groupthres = allthres(thresind);
groupamps = allpeakamp(ampind);

And eventually need to plot fit = polyfit(groupthres(ampind), groupamps,1) if I'm thinking about this correctly.

I tried putting these new variables both inside and outside the loop. Any ideas? Thanks!

[Lindsay]

And looking closer it still seems to be taking the indices from the first subject. My guess is I'm doing something wrong here :)

[Geoff Hayes]

Lindsay, can you attach your Excel file and full code (not just a snippet of it)?

[Lindsay]

Sure can! Far as the m-file goes, I'm only really working on the first figure as everything that comes after will be modeled on this first bit of code. I'm looking at a lot of different relationships between a number of variables.

Thanks! This is a great learning experience for a new coder.

[Geoff Hayes]

I should have made clear that you do the

 [findampi, findampj] = find(allpeakamp > 0);

outside of the for loop, once you have collected all of the data. There are two problems with doing this inside the loop as

    ampind      = sub2ind(size(allpeakamp),peakampi,peakampj);
    thresind    = sub2ind(size(allthres),thresi,thresj);
    allampind   = cat(2, allampind, ampind);
    allthresind = cat(2,allthresind,thresind);

The cat will fail if ampind has a different number of rows than the previous iteration. This is possible (and happened on the fourth subject!) and you will observe a

 Error using cat
 Dimensions of matrices being concatenated are not consistent.

error message. That is a problem with trying to concatenate matrices of unknown sizes together. The second problem, is how the code gets all grouped data outside of the for loop

 groupthres = allthres(thresind);
 groupamps  = allpeakamp(ampind);

Note how threshind and ampind are being used. And these are the indices computed for the eighth subject. But their linear indices correspond to their peakamp column vector (and so the linear indices will be in the range of 1 through 14), and allpeakamp(ampind) will return only those elements from the first subject (as they correspond to the linear indices of 1 through to 14 in the larger allpeakamp matrix).

I recommend that we move this calculation, of determining which threshold and peak amp values are greater than zero, outside of the for loop.

I also think that we can simplify the code a little by doing the following- consolidate all conditions (for subjects 1,2,3,...,8) into one block of code since the only difference appears to be which Excel spreadsheet to read from. (Note that the final condition on the if/elseif is else isubj = 8 which works but probably not the way it was intended to). So your for loop should become

 %%cycle through all subjects, importing data from Excel
 for iSubj = 1:length(SUBJECTS) 

    %%import relevant data from Excel
    qthres  = xlsread(xlsfile, iSubj, 'J3:J16'); 
    peakamp = xlsread(xlsfile, iSubj, 'L3:L16');
    scalae  = xlsread(xlsfile, iSubj, 'C3:C16'); 

    %isolates peak amplitudes above zero relative to scalae
    [STi, STj, ST] = (find(scalae == 1 & peakamp > 0)); 
    [SMi, SMj,SM]  = (find(scalae == 2 & peakamp > 0));
    [SVi, SVj,SV]  = (find(scalae == 3 & peakamp > 0));

    %%Plot data relative to scalae %%

    plot(qthres(STi),peakamp(STi), 'ko', 'LineWidth', 1.5);
    hold on;
    plot(qthres(SMi),peakamp(SMi), 'gd', 'LineWidth', 1.5);
    plot(qthres(SVi),peakamp(SVi), 'b^', 'LineWidth', 1.5);

    %store peak amplitude values for each subject 
    allpeakamp = cat(2, allpeakamp, peakamp); 

    %store threshold values for each subject 
    allthres = cat(2, allthres, qthres);

    %store scalae info for each subject 
    allscalae = cat(2, allscalae, scalae);

 end

In the above, I removed all commented out code, and have used iSubj to access the specific sheet from the spreadsheet. (I also remove two of the three hold on statements because once you call this once, there is no need for another call to it.)

Now, outside of the for loop, we get all data that such that

 [peakampRowIdcs,peakampColIdcs] = find(allpeakamp>0);
 [threshRowIdcs,threshColIdcs]   = find(allthres>0);

 ampind   = sub2ind(size(allpeakamp),peakampRowIdcs,peakampColIdcs);
 thresind = sub2ind(size(allthres),  threshRowIdcs, threshColIdcs);

 groupthres = allthres(thresind);
 groupamps  = allpeakamp(ampind);

Note that there is no guarantee that groupthres and groupamps have the same number of elements. So this will affect how you handle the polyfit and corr. I modified them slightly to

 fit   = polyfit(allthres(ampind),groupamps, 1);
 [r,p] = corr(allthres(ampind),groupamps)

Note how this is different from your code of using

 fit   = polyfit(groupthres(findampi),groupamps, 1);
 [r,p] = corr(groupthres(findampi),groupamps);

since findampi is no longer defined, and for at least polyfit, both input vectors need to have the same size.

So try applying some of the above changes into your code, Lindsay. You should be able to retrieve all positive data outside of the for loop for both the amp and thresh data sets.

[Lindsay]

Geoff,

Wow, thank you so much for this thoughtful help. I am heading into the lab today and will try this and let you know. This Matlab question center has been a gift as I'm learning this! I'll let you know if I have any follow up questions as I'm meeting with my advisor today and she'll probably help me finish this if I can't.

[Lindsay]

It worked! And whoa is this code much more concise. T Hanks again -- I appreciate receiving an explanation so I can learn for the future rather than just someone throwing code at me. This has made my final analysis so much easier, Geoff!

Annnnnd my p-value is still significant ;)

[Geoff Hayes]

Glad to have been able to help, Lindsay!