How to find out standard deviation of a matrix when there is a lot of NaN values?

Hi altruists,

I have a 71*294 matrix that contains a lot of NaN values.

In fact, the first 7 rows, the last 7 rows, and row 30 to 41 are just NaNs! But the rest of the matrix has values. I want to calculate the row-wise std (standard deviation). How can I write down the code that automatically neglects the NaNs and calculate the std of the non NaN values?

Answer 1

[Kevin Holly]

You can ignore NaN values with the std function by adding 'omitnan' as an input.

A = rand(9,9);
A(3,4) = NaN;
A(7,8) = NaN;
A(8:9,:) = NaN

A = 9×9

0.8525 0.7498 0.3202 0.4618 0.2857 0.0152 0.0025 0.7856 0.8587 0.9196 0.9043 0.7895 0.8567 0.1709 0.5375 0.7785 0.3022 0.3562 0.5906 0.8111 0.0398 NaN 0.4670 0.0575 0.9813 0.1220 0.8259 0.1217 0.8268 0.8372 0.2509 0.8764 0.6745 0.6064 0.4309 0.4537 0.0781 0.7542 0.5271 0.0940 0.1519 0.8002 0.9233 0.3918 0.2708 0.5682 0.1108 0.3855 0.1143 0.4872 0.0658 0.5214 0.7553 0.8456 0.2884 0.0712 0.7995 0.0562 0.5466 0.5103 0.9874 NaN 0.8934 NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

B = std(A,0,2,'omitnan')

B = 9×1

0.3457 0.2871 0.3766 0.2690 0.3229 0.2840 0.3600 NaN NaN

[Ashfaq Ahmed]

Hi Kevin, thank you soo much!

But what does (A,0,2) mean?

[Steven Lord]

Looking at the Syntax section of the documentation page those three inputs match the (A, w, dim) syntax. They could also match the (A, w, vecdim) syntax but since 2 is a scalar those syntaxes would do the same thing.

Both the Description section and the Input Argument section on that page go into more detail about the meaning of the A, w, and dim inputs.

[Kevin Holly]

From the documentation on the function provided:

S = std(A,w,dim) returns the standard deviation along dimension dim for any of the previous syntaxes. To maintain the default normalization while specifying the dimension of operation, set w = 0 in the second argument.

The first input to the function, A, is your matrix. The second input, w, is the weight (default is 0 which I had chosen). The third is the dimension to operate along. In this case, 2 was chosen so that the standard deviation of the elements would be calculated in each row. A value of 1 (default value) would calculate the standard deviation for each column.

Answer 2

[Voss]

std(x,[],2,'omitnan')

where x is your matrix, 2 tells std to go by row, and 'omitnan' tells std to ignore NaNs in x.

[Ashfaq Ahmed]

Thank you so much :)