Help preallocating variables for a table

Hi everyone,

so i have written some code which goes like this:

function caTable = makeCaTable(caCM)

% creates a table with information about the frames(coordinates of the
% centroids and the total number of cells in the frame)

Frame = {};

PosX = {};

PosY = {};

NumberOfCells = {};

for i = 1:size(caCM,2)

      C = chop(caCM{1,i},0);

      for j = 1:size(C,1)

          Frame{end+1,1} = i;

          PosX{end+1,1} = C(j,1);

          PosY{end+1,1} = C(j,2);

          if(j == 1)

              NumberOfCells{end+1,1} = size(C,1);

          else
              NumberOfCells{end+1,1} = '-';
          end
      end
  end
  % write the table
  caTable = table(Frame, PosX, PosY, NumberOfCells);

it works and gives me the results i want. But now i want to preallocate the variables. I thought I do it like this:

[nrows,ncols] = cellfun(@size,caCM);

rows = sum(nrows);

Frames = cell(rows,1);

PosX = cell(rows,1);

PosY = cell(rows,1);

NumberOfObjects = cell(rows,1);

But if i do so i cant use my for loop the way i wrote it, since it would always put the values at the end of the preallocated matrices. But if i simply use:

Frames(j,1) = i

my table will not have all the fields. I really don't know how i can fix this. Maybe some of you could help me with that.

Thank you very much!

[Guillaume]

Please get rid of all the extra blank lines you've added to the code and in the future use the {}Code button to format code as such, as I've done for you now.

What is C (returned by chop)? A cell array? A matrix?

Why are PosX, PosY and particularly Frame cell arrays. Frame only stores scalar numerics. Not sure about PosX and PosY (depends on what C is).

[Julian]

Thank you Guillaume. C is a matrix with doubles in it. They don't necessarly have to be cell arrays. I just built it that way because it was the first thing that came to my mind. They actually all hold scalar numerics.

Answer 1

[Guillaume]

C is a matrix with doubles in it

In that case,

function caTable = makeCaTable(caCM)
   %doc goes here
   %  caCM: a 2D(?) cell array
   %  caTable:a table with 4 variables: Frame, PosX, PosY, NumberOfCells
 
   chopped = cellfun(@(x) chop(x, 0), caCM(1, :), 'UniformOutput', false)';  %transpose for easier repelem and cell2mat
   NumberOfCells = cellfun(@(m) size(m, 1), chopped);
   NumberOfCells = repelem(NumberOfCells , NumberOfCells );
   Frame = repelem((1:size(chopped, 1))', NumberOfCells);
   chopped = cell2mat(chopped);
   PosX = chopped(:, 1);
   PosY = chopped(:, 2);
   caTable = table(Frame, PosX, posY, NumberOfCells);
end

As a rule you shouldn't be using cell arrays unless you need to, they're slower to process and takes a lot more memory than a matrix (15 times more if each cell is just a scalar double).

Note that I'm replicating the number of cells for all rows of a frame instead of having '-' for all rows but the first of the frame. It's rarely a good idea to mix numbers and characters in a variable. If you want a different value for all but the first row, I'd recommend using NaN instead. It's trivially done:

     NumberOfCells = repelem(NumberOfCells, NumberOfCells);  %continuing on from there
     NumberOfCells([false; diff(NumberOfCells) == 0]) = NaN;

If you really want a '-', then it's slightly more complicated:

     ncells = repelem(NumberOfCells, NumberOfCells);  
     NumberOfCells = num2cell(ncells);
     NumberOfCells([false; diff(NumberOfCells) == 0]) = {'-'};

[Julian]

Hi Guillaume,

thanks for your answer. This looks pretty sophisticated and i'm not quite sure if i get everything. But trying to implement it, i have a few issues. caCM is a cell with 1xn elements. And everyone of these elements is a mx2 matrice of doubles. n depends on the number of frames in the video and m on the number of cells that are currently in that frame.

I get C from the for loop. So this is where i should insert your code right?

[Guillaume]

Sorry, C should have been caCM. I've fixed the code. My code completely eleminate the need for loops and what I've written is the whole function.

If caCM is a row cell vector, then you can replace that first line by:

chopped = cellfun(@(x) chop(x, 0), caCM, 'UniformOutput', false)';  %transpose for easier repelem and cell2mat

Since all matrices, tables, etc. are created in one go already filled, there's no need to preallocate anything (which would actually be counterproductive)

[Julian]

Great, with this change i can run it now until 'Frame' and then i get this error:

Error using repelem

In repelem(v, N), N must be a scalar or a vector of length equal to v.

I try to explain it with an example:

If i have 2 frames with 72 cells in the first frame and 73 in the second frame, then v would be 2 and N would be 145. And what i want in the table to be displayed is 1 for the first 72 cells and a 2 for the following 73 cells (73-145). This was probably not clear from the description of my code, i'm sorry.

[Guillaume]

No, I understood the intent the first time round. It's my mistake, I meant to use the original NumberOfCell before it got repelem'd on that Frame line. Swapping the order of the two lines will fix the error:

   NumberOfCells = cellfun(@(m) size(m, 1), chopped);
   Frame = repelem((1:size(chopped, 1))', NumberOfCells);    %to be done first
   NumberOfCells = repelem(NumberOfCells , NumberOfCells);

Better would be not to reuse NumberOfCells for different purpose so replace the above three by:

ncells = cellfun(@(m) size(m, 1), chopped);
NumberOfCells = repelem(ncells, ncells);
Frame = repelem((1:size(chopped, 1))', ncells);

and then the order doesn't matter.

[Julian]

Thank you so much Guillaume. I really learnt a lot! It works just fine now

Answer 2

[Steven Lord]

I think you want to use the T = table('Size',sz,'VariableTypes',varTypes) syntax shown on the documentation page for the table function.

[Julian]

Yes i did that:

    [nrows,ncols] = cellfun(@size,caCM);
rows = sum(nrows);
varTypes = {'double', 'double', 'double', 'double'};
varNames = {'Frame', 'PosX', 'PosY', 'NumberOfCells'};
sz = [rows 4];
caTable = table('Size', sz, 'VariableTypes', varTypes, 'VariableNames',varNames);

but I still don't know how to use the for loop to make it fill out the empty fields the right way.

[Peter Perkins]

The preallocation syntax that Steve is referring to was introduced in R2018a.

But Guillaume's answer avoids needing to preallocate the table, because nothing is growing dynamically, and everything is a vectorized assignment. +1.