How to simplify this code without having multilevel indexing error???

Hi,

I've a matData consists of (1x96 struct) and each structure, ji has (1x35 cell) and for each cell has (20x2 double). I want to arrange the data into (1400x96 double).

As for now, I only managed to get (1400x1 double) separately for each matData by using the code below,

 a1 = matData(1).ji;
 aa1 = cell2mat (a1);
 p1 = reshape(aa1, 1400, 1);

 a2 = matData(2).ji;
 aa2 = cell2mat (a2);
 p2 = reshape(aa2, 1400, 1);

 a3 = matData(3).ji;
 aa3 = cell2mat (a3);
 p3 = reshape(aa3, 1400, 1);

What should I do to simplify this code without having manually typing for all 96 matData?

I can't straightly write as,

 aa = matData(1:35).ji{1:35};

because I keep getting this error 'Field reference for multiple structure elements that is followed by more reference blocks is an error'.

Thank you

Answer 1

[Cedric]

Here is one way. I build a simple test data set with 4 structs, each ji field contains 3 cells, and each cell contains a 2x2 numeric array:

 S(1).ji = {[10 12;11 13],[14 16;15 17],[18 20;19 21]} ;
 for k = 2:4, S(k).ji = cellfun( @(x)k*x, S(1).ji, 'UniformOutput', false ) ; end

To aggregate:

 >> bigArray = cell2mat( cellfun( @(x)x(:), vertcat( S.ji ).', 'UniformOutput', false ))
 bigArray =
    10    20    30    40
    11    22    33    44
    12    24    36    48
    13    26    39    52
    14    28    42    56
    15    30    45    60
    16    32    48    64
    17    34    51    68
    18    36    54    72
    19    38    57    76
    20    40    60    80
    21    42    63    84

Evaluate the internal expressions to understand, and ask if anything is unclear. Looking at your code, the only thing that you may not be familiar with is S.ji (with no struct index), which is a comma separated list (CSL), that we concatenate with VERTCAT. Then all the rest is about manipulating cell and numeric arrays.

[hana razak]

I got it..

Thank you so much. It was really very helpful!

[Cedric]

My pleasure! Don't forget to [Accept] or up-vote either answer if they have been helpful.

[hana razak]

Thank you for letting me know about the [Accept] or up-vote.

This is my first time being here.

Thanks once again.

[Cedric]

No problem and welcome!

[hana razak]

Hi,

I just noticed that the S(1).ji coding was for you to build the sample data set. That was why all my data had been altered. I just found out while analyze the results.

This is how I write to access matData.ji,

a = [matData.ji]; % a 1x3600 cell with each cell has (20x2 double)

bigArray = cell2mat( cellfun( @(x)x(:), vertcat(a).', ... 'UniformOutput', false )); % a 135760x1 double

The final result is a bit weird when it only gives 135760X1 double instead of 144000x1 (3600x20x2).

Please help me once again.

Thank you

[Cedric]

Ah, sorry for that and yes my solution was the one-liner after "To aggregate:"; what precedes is just for building a sample/training data set. We usually give these so people can reproduce the full example/test.

Without having access to the data, I can only guess what is happening. You tried my solution

 bigArray = cell2mat( cellfun( @(x)x(:), vertcat( matData.ji ).', 'UniformOutput', false ))

and you got a size mismatch error, then you tried using

 a = [matData.ji] ;

which is a HORZCAT actually (the VERTCAT in the the call is then useless) and it somehow works because sizes are matching, but you don't understand what is happening.

My guess is that if you run the following

 sz = cellfun( @size, [matData.ji], 'UniformOutput', false ) ;
 unique( vertcat( sz{:} ), 'rows' )

which returns unique size(s) of the internal arrays, you will realize that you won't get only 20 2 but some other numbers as well. Try it and let me know, if it is the case, I can explain what is happening.

[hana razak]

Firstly, I'd mistakenly pressed the calculator. I should get 134400x1 not 144000x1.

Your latest code seem like it goes further. It only gives the sizes of the cells. I tried to change vertcat to horzcat but still it gives 135760x1 instead of 134400x1.

I've faced a problem with multilevel indexing error to begin with and you were right about the missing of structure index.

Apparently, the problem to access into a 1x96 struct with each ji, 1x35 cell is solved by using,

a = [matData.ji];   which gives 1x3360 cell (96x35).

Now, I failed to access into each cell of 20x2 double of each 1x3360 cell.

I only get half way right by using your code;

bigArray = cell2mat( cellfun( @(x)x(:), vertcat( a ).', 'UniformOutput', false ));

I keep playing with cellfun, cell2mat, reshape, vertcat but I failed to get desired results. It seem so close yet so far.

If I can get 134000x1 double, I can easily reshape it to 1400x96 double.

Thanks so much

[Cedric]

But what does this code return?

 sz = cellfun( @size, [matData.ji], 'UniformOutput', false ) ;
 unique( vertcat( sz{:} ), 'rows' )

[hana razak]

sz => 1x3660 cell and each cell shows only [20, 2] which is the data size not numeric data

I got error for unique code.

[Cedric]

What version of MATLAB are you using? Is the error about VERTCAT or UNIQUE, and what is the error message? Also, what error message are you getting when you run strictly:

 bigArray = cell2mat( cellfun( @(x)x(:), vertcat( matData.ji ).', 'UniformOutput', false ))

[hana razak]

I'm using MATLAB R2015a. My mistake - the error was about vertcat not UNIQUE.

Error using vertcat
Dimensions of matrices being concatenated are not consistent.

Error in test3 (line 12)
unique( vertcat( sz{:} ), 'rows' )

There is no error when stricly run the 'bigArray' code.

That was the best code for now but it doesn't give right answer which is 134400x1 double instead of 135760x1 double with 1360 extra.

[hana razak]

Hi,

Thank you very much for your attention & assistance.

I'd figured it out.

Both codes are always right from the beginning.

bigArray = cell2mat( cellfun( @(x)x(:), vertcat( matData.ji ).', 'UniformOutput', false ))

 sz = cellfun( @size, [matData.ji], 'UniformOutput', false ) ;
 unique( vertcat( sz{:} ), 'rows' )

The problem is, both only relevant up to 49 data set (1 data set is 1400x1 double). It gets weird when handles more than 50 data set (mine is 96).

Is there any other way for MATLAB to handle bulk data in 1 folder?

Thank you

[Cedric]

If your data is not too confidential, the simplest way to go would be to drop me an email using my personal email (you get it when I send you notices that I posted a comment to your thread), attaching a MAT-File with matData.

If you cannot do this, the whole point of the test that I suggested was to identify a situation where there is a size mismatch.

See, if you have

 >> A = {randi(10,2,3),randi(10,2,3);randi(10,2,3),randi(10,2,2)}
 A =
  2×2 cell array
    {2×3 double}    {2×3 double}
    {2×3 double}    {2×2 double}

where there is a size mismatch (A{2,2} is 2x2 and not 2x3 like the others), you will be able to concatenate all contents horizontally but not vertically:

 >> horzcat( A{:} )
 ans =
     7     7    10     3     3     6     4     5     8     5     7
    10     7     6     1     9     5    10     7     7     8     7

 >> vertcat( A{:} )
 Error using vertcat
 Dimensions of matrices being concatenated are not consistent.

The horizontal concatenation works technically, but it doesn't mean that it is really the operation that you intended to do. It certainly let the size mismatch undetected.

My guess was that you ran my code (that uses a vertical concatenation), got this type of error, and then tried using a horizontal concatenation. And while it went through, the size doesn't match your expectation.

Your comment seems to indicate that you found the size discrepancy. Now all you have to do is to understand whether it should be concatenated with the rest (maybe it is just invalid), and in what direction.

[hana razak]

Yes, my data is not too confidential. You can send the notice in order for me to attach the MAT-File. Thank you very much. This would be a great help for me.

Answer 2

[KL]

s2c = struct2cell(matData);
c2c = cellfun(@(x) x',reshape(s2c,[],1,1),'uni',0);
c2a = cell2mat(cellfun(@cell2mat,c2c,'uni',0));

[hana razak]

It only got half way through. I've got an error for this code;

c2a = cell2mat(cellfun(@cell2mat,c2c,'uni',0));

It just stated as below,

'Error in test2 (line 19) c2a = cell2mat(cellfun(@cell2mat,c2c,'uni',0));'

I don't have any idea how to correct it. So, is there any other way to get it right? Thanks a lot

[hana razak]

I got it..

The codes are work very well. The error was due to bulk data.

Thank you so much. It was really very helpful!