problem of memmapfile function

m=memmapfile(fileStr,'Offset',3600,'Format',{sampleType,[470 141],'ss'},'Repeat',1);

Program can run normally.

Modify the procedure as follows:

numSamp=470;
numTrace=141;
m=memmapfile(fileStr,'Offset',3600,'Format',{sampleType,[numSamp numTrace],'ss'},'Repeat',1);

Error information：

Error using memmapfile/set.format (line 186)The Format field specified is invalid.
m=memmapfile(fileStr,'Offset',3600,'Format',{sampleType,[numSamp numTrace],'ss'},'Repeat',1);

In the program design process, often encounter [] matrix of rows and columns is variable, how to achieve the objective, thank you very much. email:[e-mail]

[Walter Roberson]

In my test, I had no problem using variables for the size.

Answer 1

[Walter Roberson]

To use variable length data using memmapfile, you need to store the length of the data in a place before the variable section. Then you need to memmapfile() from the starting Offset for the variable length data, filling in the now-known length. For example,

uint16_len = 2;
double_len = 8;

base = 3600;

m = memmapfile(fileStr, 'Offset', base, 'Format', {'uint16', [1 1], 'numrows'}, 'Repeat', 1);
numrows = m.Data.numrows;
base = base + uint16_len;   %bytes

   m = memmapfile(fileStr, 'Offset', base, 'Format', {'uint16', [1 numrows], 'rowlengths'}, 'Repeat', 1);
   rowlens = m.Data.rowlengths;
   base = base + uint16_len * numrows;

   rowcontents = cell(1, numrows);
   for rownum = 1 : numrows
     thiscount = rowlens(rownum);
     m = memmapfile(fileStr, 'Offset', base, 'Format', {'double', [1 thiscount], 'rowdata'}, 'Repeat', 1);
     rowcontents{rownum} = m.Data.rowdata;
     base = base + double_len * thiscount;
   end

This would be for in input file that at position 3600 (bytes) contained a single uint16 that was the number of rows in the data. Immediately following the uint16 would be that many more uint16, each of which was the number of columns in the corresponding row. Immediately following those uint16 would be a vector of doubles of length given by the first row count, then immediately after that, a vector of doubles of length given by the second row count, and so on, until all of the counted information had been read in.

At each point in the reading sequence, you need to advance the base position by the amount of data you have read so far.

It is possible to get more complicated than this, including having structures -- as long as the sizes have already been read in (or calculated) before you do the corresponding memmapfile(), and as long as you advance the base by the appropriate number of bytes.

If you are going to use this kind of binary structure, but you do not always need all of the variables, then it can be useful to store a total length for what follows just before any of the variable length content, along with an indication of what it is that follows. For example if you have 3 different "variables" stored in the file, you might store

<2 byte indicator that this is the first variable> <8 byte indicator of the number of bytes of the file used by the first variable> <bytes that represent the first variable>
<2 byte indicator that this is the second variable> <8 byte indicator of the number of bytes of the file used by the second variable> <bytes that represent the second variable>

and so on.

If you use this design, where there is a "tag" and an associated length, then if you want to find the second variable, you can look at the first tag in the file, see that it does not match, and advance forward in the file by the indicated number of bytes, which positions you after all of the first variable without having to read it at all. And then you can look at the tag you find there, see that it matches what you are looking for, ignore the size information, start processing the stream of bytes.

If you do use a design like I describe, it becomes allowed to put the individual variables in any order. It even becomes allowed to "erase" a variable in a file without touching the ones after it, by setting the tag to a tag that means "unused memory" and the size after it; that would allow you to "grow" a variable by writing the information for it at the end of the file and setting its former space to be unused. (You can then also reclaim space in the file if you find you want to write in something that fits within an unused space.)