how i can delete useless data from matrix

hi guys,

i have a matrin m lines 11 columns.

i wish a new matrix where are deleted the lines when:

there are three lines row the same of the 8th columns and how the second condition the same but with 6 column. Like the the example in the insert.

from the 715 lines there is the problem that i would deley in the new matrix.

thanks

[Adam Danz]

Sorry, I'm having difficulty understanding. Maybe try again or provide an example?

[Giuseppe Antonacci]

if you look the image,in the 8th column from the 715 lines there are the same number, for more of two lines, the same conditions there is in the 6th column.

i wish a new matrix where there are this two condition whitout this lines.

[madhan ravi]

Illustration with an example will avoid the delay in answer https://www.mathworks.com/matlabcentral/answers/443329-how-i-can-delete-useless-data-from-matrix#comment_667952

[Giuseppe Antonacci]

i add a new images whith the example:

from matrix A i would matrix B where are removed the dates when there are the two conditions in the imagess.

[Bob Thompson]

So you want to remove the duplicates from the 8th and 6th columns. That's relatively easy to do by identifying the unique values using unique() and some logic indexing.

How are you deciding which row of the duplicates to keep. It looks like there are some differences in columns 1 and 5, so each row cannot be regarded as completely the same.

[Guillaume]

Note that posting actual files instead of screenshots makes it much easier for us to give an answer that works. We can't copy/paste screenshots into matlab to test our answers. We can use actual files (I assume the input is an excel file or similar, here).

[Giuseppe Antonacci]

sorry but the matrix is too big. i would remove the duplicates with the idea. when i have the same number for three lines. i think that i use a cycle for/if.

[Guillaume]

It's very unlikely that you need loops and if statement. If the matrix is too big, attached a smaller one which still has enough data to show what you want.

I'm with Bob, I'm really unclear on what you want. So attach a simple example of input and the corresponding desired output.

Answer 1

[Bob Thompson]

Here is something to try. Probably not the most efficient, but it should do what you're asking.

vals = unique(data(:,8));
data2 = [];

for i = 1:length(vals);
    if size(data(data(:,8)==vals(i),:),1)<=3
        data2 = [data2; data(data(:,8)==vals(i),:)];
    else
        tmp = data(data(:,8)==vals(i),:);
        data2 = [data2; tmp(1:3,:)];
    end
end

I would agree with Guillaume that extra explanation would be helpful. If this loop combo does what you're looking for we can probably come up with some way of doing this without the loops, just let us know.

Answer 2

[Stephen23]

Do NOT use a loop for this.

MATLAB code should be neat and simple:

[~,idx] = unique(data(:,[6,8]),'stable','rows');
out = data(idx,:)

Because you have floating point numbers, I recommend either using uniquetol, or scale the values and round them before unique.

[Giuseppe Antonacci]

I'm gonna try to explain my problem in an easier way. So i have an m*n matrix like this:

21 42 38 29

29 39 41 29

23 3 2 18 <---- TO REMOVE

15 3 2 31 <---- TO REMOVE

24 3 2 48 <---- TO REMOVE

26 3 2 19 <---- TO REMOVE

51 35 24 22

I need an algorithm that: If i have on two distinct column the same value for three row (or more), i have to remove those rows.

------------------------------------------------------------------------------------------

I give you an other example where removing is not needed:

21 42 38 29

29 39 41 29

23 3 2 18 <---- OK

15 3 2 31 <---- OK

51 35 24 22

------------------------------------------------------------------------------------------

I also give you a pseudo code example:

matrix = [values here]

new_matrix = []

for row in range (0, len(matrix)):

if (matrix [row] [2] != matrix [row + 1] [2] && matrix [row+1] [2] != matrix [row + 2] [2] && matrix [row+2] [2] != matrix [row + 3] [2]):

new_matrix.append(row)

In simple words: if i don't find the same value on a given column for the next three rows, that row is ok, but if i find the same value (on a given column) for the next three rows, i have to remove that row.

[Stephen23]

Download Jan Simon's excellent RunLength here:

https://www.mathworks.com/matlabcentral/fileexchange/41813-runlength

and use it like this:

>> M = [21,42,38,29;29,39,41,29;23,3,2,18;15,3,2,31;24,3,2,48;51,35,24,22;21,42,38,29;29,39,41,29;23,3,2,18;15,3,2,31;51,35,24,22]
M =
    21    42    38    29
    29    39    41    29
    23     3     2    18
    15     3     2    31
    24     3     2    48
    51    35    24    22
    21    42    38    29
    29    39    41    29
    23     3     2    18
    15     3     2    31
    51    35    24    22
>> R = 3; % >=R identical rows will be deleted.
>> X = all(diff(M(:,[2,3]),1,1)==0,2); % check columns 2 & 3.
>> [B,N] = RunLength([false;X]|[X;false]);
>> B(N(:)<R & B) = false;
>> Y = repelem(B,N)
>> M(Y,:) = [] % remove rows
M =
    21    42    38    29
    29    39    41    29
    51    35    24    22
    21    42    38    29
    29    39    41    29
    23     3     2    18
    15     3     2    31
    51    35    24    22

[Giuseppe Antonacci]

(undefined function or variable "RunLength"). why?