cell to mat-file

Hi

As far as i know partial loading with MAT-file does not support a variable of cell class. My question is if there exists a some way to partial load a cell of size 1x n

Answer 1

[Walter Roberson]

Mathworks specifically documents that you cannot do a partial load of a cell array.

You would have to break up your cell array into variables when you stored it in the matfile. For example,

chunksize = 10;
varbase = 'MyVar';
chunkname = cat(varbase, '_chunk_');

m = matfile('myFile.mat','Writable',true);

varlen = size(MyVar, 1);   %I am going to presume chunking by groups of rows

chunklist = [1 : chunksize : varlen, varlen+1];
numchunks = length(chunklist) - 1;

chunknames = cell(numchunks,1);
for chunknum = 1 : numchunks
   thisvarstr = sprintf('%s%d', chunkname, chunknum);
   m.(thisvarstr) = MyVar(chunklist(chunknum):chunklist(chunknum+1)-1,:);
   chunknames{chunknum} = thisvarstr;
end

chunkinfo = struct('chunklist', chunklist, 'chunksize', chunksize, 'size', size(MyVar), 'chunknames', chunknames, 'chunkname', chunkname);
chunkinfostr = cat(chunkname, 'info');
m.(chunkinfostr) = chunkinfo;

This produces file variables MyVar_chunk_info with some descriptive information, and MyVar_chunk_1 MyVar_chunk_2 and so on to hold the information.

Given any particular column index, the chunk number can be computed based upon chunksize. Or you could use the chunk list, which is structured as a list of boundaries such as [1 11 21 22] with the N'th chunk starting at the cell index given in the N'th entry and ending before the cell index given in the next entry -- so the above list would describe chunks with indices {1:10, 11:20, 21}.

[counts, chunknumber] = histc(rowindexlist, chunklist);

Adapt for whatever flexibility you need.

Answer 2

[Image Analyst]

Why not just load the one variable that you're interested in:

storedStructure = load(matFullFileName, 'yourCell');
yourCell = storedStructure.yourCell;

[Maciek]

if so, can i use indexing for this variable and loading it (for example 1:100) along with other variables of double class.

[Walter Roberson]

Image Analyst is proposing to load the entire variable and then work with it. That could cause problems if the variable was very large.

[Maciek]

the size is 1x100000

[Maciek]

that's why i'm stuggling to find a way of partial loading

[Image Analyst]

I was proposing to store manageable size variables into your mat file, like Walter did. Then you can pull out just one manageable chunk at a time. A cell array of 100,000 cells is not necessarily that big - it depends on what is in each cell. What is in a typical cell? Just a few numbers or like an image or something? If it was so big, then how did you have it in your program in the first place to even be able to save it out with save()???

You might also take a look at memmapfile().

Answer 3

[Jan]

c = cell(1, 100);
d = pi;
save('a.mat', 'c', 'd');
clear
cc = load('a.mat', 'c');   % ==> cc.c : {1x100 cell}

So why do you mean that the partial loading does not work with cells?