Extracting a block from a big array

Hi I have a data set say 30000 x 12.

With in this data set, I have a number of sub blocks of data of my interest.

Each sub block starts with a -9999 value in first column (:,1).

Each sub block comprises of the data from first 4 columns. (rest of the 8 (12-4) columns are unnecessary).

Each of the sub block ends when the value in 3rd column turns from +ve to -ve.

I am using for loops and if statements to the job. however struggling at the logic to extract my required block. Can you plz suggest proper set of MATLAB commands to do this.

Best regards

[Andrei Bobrov]

your code

[Muhammad]

Thank you andrei for such an exact answer :)

[James Tursa]

Please post your code and then we can comment on how to improve it. Do you need each block individually for processing within the loop? Do you need all the blocks extracted and concatenated to one large matrix? What are you doing with these blocks downstream? Can those operations be vectorized? Etc etc.

Answer 1

[Geoff]

I like using find for these kinds of problems. To find all the starting indices of your blocks, you can go:

idxStart = find( data(:,1) == -9999 );

As for the end of the blocks, the code depends on known conditions. For instance, is it guaranteed that the third column flips +ve to -ve before the next block begins? If so, that simplifies things.

Regardless of assumptions, you can detect these flips in different ways, but the simplest (I think) is to put two copies of the array side by side and offset by one. Um, sounds complicated... It's easy when you see it in code:

col3 = data(:,3);
idxEnd = find( [0;col3] > 0 & [col3;0] < 0 );

That'll give you the index of the -ve row. If you want the positive row that precedes it, then of course you do:

idxEnd = idxEnd - 1;

Let's just say that your data is clean and well-defined and that you CAN assume every block start has a block end before the next block starts.... Then you know that idxStart and idxEnd will have the same number of elements and they will also match up.

blocks = cell(1, length(idxStart));
for b = 1:length(idxStart)
    blocks{b} = data( 1:4, idxStart(b):idxEnd(b) );
end

This would slurp all your blocks into a cell array called 'blocks'.

Of course, if your start and end indices aren't guaranteed to match up you're going to have to be more clever. You may prefer to filter them so that they DO match up, and then use the code above to extract the blocks.

I'll leave that filtering step as an exercise for you. =)

-g-