Row combination for repeated values

Hello,

I have different variables (date, time, lat, lon) and X for different depths, where each column represents n times (number of different depths) the same variable at a different depth, let say 0 m, 100 m, 500 m. Here a more explicit example :

A = [20200101, NaN, NaN, 1200, NaN, NaN, 90, NaN, NaN, 45, NaN, NaN, 2, NaN, NaN; % 0 m
NaN, 20200101, NaN, NaN, 1200, NaN, NaN, 90, NaN, NaN, 45, NaN, NaN, 3, NaN; % 100 m
NaN, NaN, 20200101, NaN, NaN, 1200, NaN, NaN, 90, NaN, NaN, 45, NaN, NaN, 4 ]; % 500 m

How can I keep the repeated band combinations for date, time, latitude and longitude, so my final output would be, for each unique Date / Time / Lat / Lon :

B = [20200101, 1200, 90, 45, 2, 3, 4]; % Date / Time / Lat / Lon / Val 0m / Val100m / Val500m

I was thinking of using a for loop with find

find(A(i,1)==A(:,2) & A(i,1)==A(:,3) etc.)

But I if I have 50 different depths, it will be a lot of combinations. Does anyone has a simpler way in mind ?

Cheers

[Jon]

I'm not understanding what the columns in your A matrix represent. In particular how come the date appears in a different column in each row. Similarly why does the latitude appear in a different column. It looks like they shift over by 1 column in each succesive row, but I don't know if this is always the case, and what it means.

Answer 1

[Spencer Chen]

Now, you were not very clear on your column structure. I understand it as in:

A = [date.depth1, date.depth2 .. date.depthNtime, lat.depth1, lat.depth2 ..lat.depthN, etc.]
% A = row X (depth and Var)

If that's the case, then something like this code would solve you problem:

nrows  = size(A,1);
ndepth = 3;
A2 = reshape(A,nrows,ndepth,[]);   % A2 = row X depth X Var
B  = squeeze(nansum(A2,2));

The crux of it is to use nansum() to combine and ignore values from other depths. So we restructure the matrix so we can use nansum().

Blessings,

Spencer

[Maha]

Hello Jon and Spencer, thanks for the comment, maybe I wasn't clear at all :)

Yes my structure is what you understand in your A, with A(:,1) being date.depth1 etc

I can have a structure like B from my row data, that is not an issue. But you can start from your B to reach my B it is more clear for you :

B = [ 20200101, 1200, 90, 45, 2;
    20200101, 1200, 90, 45, 3;
    20200101, 1200, 90 ,45,4];

My problem is that I want to create, for each combination of date / time and position, all the data available at different depths, like I show in my B example.

B = [20200101, 1200, 90, 45, 2, 3, 4]; % Date/Time/Lat/Lon/Val0m/Val100m/Val500m

I hope that is it clearer

[Maha]

I accepted your answer because it helped me to resolve my problem with an easier way of visualisation.

Not sure if it is the most efficient way to do it (for huge matrices) :

A = [20200101, 1200, 90, 45, 2;
    20200101, 1200, 90, 45, 3;
    20200101, 1200, 90 ,45,4];
nbDepths=3;
C=nan(nbDepths,length(A(1,:))+nbDepths-1);
for i = 1 : length(A)
    [B, ~, ib] = unique(A(:,1:4), 'rows');
    ind = accumarray(ib, find(ib), [], @(rows){rows});
    C(i,:)=[B(1,1:4),A(ind{1,1},5)']; 
end
[D, ~, id] = unique(C, 'rows');

Edit : Not working for non consistent data

[Maha]

I don't know if I should ask an other question. It turns out that it only works for a consistent matrix A (1 line = 1 depth). Any idea on how to transform A into a consistent matrix ?

The A1 I have :

A1 = [20200101, 1200, 90, 45, 2; % depth 1
    20200101, 1200, 90, 45, 3; % depth 2
    20200101, 1200, 90 , 45, 4; % depth 3
    20200102, 1200, 90 , 45, 5; % depth 1
    20200102, 1200, 90 , 45, 6]; % depth 3

A2 is A1 in a consistent version that I don't have yet :

A2 = [20200101, 1200, 90, 45, 2; % depth 1
    20200101, 1200, 90, 45, 3; % depth 2
    20200101, 1200, 90 , 45, 4; % depth 3
    20200102, 1200, 90 , 45, 5; % depth 1
    20200102, 1200, 90 , 45, NaN; % depth 2
    20200102, 1200, 90 ,45, 6]; % depth 3

I do have the index of each depth available, i.e

ind_depth_1 = [1; 4];
ind_depth_2 = [2];
ind_depth_3 = [3; 5];

The Result that I should have if my solution from the previous message is applyied :

B = [20200101, 1200, 90, 45, 2, 3, 4;
    20200102, 1200, 90, 45, 5, NaN, 6];

[Jon]

I'm sorry, but I can not understand the problem description from the above. If you could please provide just one complete example of the input matrix or matrices that you actually would begin with and the final resulting matrix that you are looking for perhaps this would help. I see lots of matrices in the above discussion, but I can't follow what they are trying to illustrate. I think, one, simple, complete example would help.

[Maha]

My actual real matrix looks like A1, from my previous message, with 500k lines, and 10 different depths.

I need the B matrix as a result (every depth information for unique spatial and temporal information).

The solution I've shown just work for a consistent matrix A2, where each line would represent a depth. My A1 / True matrix isn't consistent.

I would like to either

1) switch from A1 to A2, then I could apply my previous solution

or

2) Find a direct solution to transform A1 into B