How can I store a complex number on a binary file?

Hello. Lets say I create the following complex number:

x = complex(rand(1,1), rand(1,1));

How could I store this 16 byte number on a binary file and be able to load it back into Matlab at a later time?

I know that if I have a regular double type number in Matlab I can write it to and read it from a binary file using the following code:

a = rand(1,1);

%write
fid = fopen('test.bin', 'w');
fwrite(fid, a, 'double');
fclose(fid);

%read
fid = fopen('test.bin', 'r');
b = fread(fid, 1, 'double');
fclose(fid);

Thanks!

Answer 1

[James Tursa]

OK, here are the mex files. There are two separate files, one for writing a single variable (numeric or char or logical) to disk, and a separate file for reading the variable from the disk. The real and imag parts are written to two separate files to keep the file sizes down. There is a fair amount of error checking in them, but not a lot of testing so I won't claim they are foolproof yet. The advantage of these mex files is that they don't make copies of your data to do the file writing and they put the real and imag parts in separate files. So they shouldn't blow your memory and hopefully the file sizes will be small enough for your purposes. These are intended to be used on the same machine and same MATLAB version for writing and reading (otherwise there may be variable size or endian issues to deal with). To use them:

- Save writing file as writevar.c somewhere on the MATLAB path

- Save reading file as readvar.c in the same directory as writevar.c

- Change your default directory to this directory

- Issue the following commands at the MATLAB prompt:

mex writevar.c
mex readvar.c

If you get an interactive question popping up asking you about compilers, just enter Y (let MATLAB search for them) and then enter the number of a C/C++ compiler that shows up in the list (e.g., lcc).

32-bit MATLAB ships with the lcc compiler, but if you are running a 64-bit version of MATLAB then you would have to get a 64-bit C compiler installed before you can compile the mex files.

writevar.c
----------
// writevar.c writes a variable to file(s), one each for real and imag
// Companion file to readvar.c
// Syntax: writevar(filename,var)
//         filename = char string containing base filename
//         var = full numeric or logical or char variable
// Note: writes real part to filename_r.bin, and imag part to file filename_i.bin
// Programmer: James Tursa
// Date: Feb 7, 2013
// Building:  mex writevar.c

#include "mex.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#ifndef MWSIZE_MAX
#define  mwSize    int
#endif

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
  FILE *fp;
  char *filename;
  short *sp;
  mwSize i, nf, nelements, ndim;
  mwSize *dims;
  size_t elements_written, element_size;
  void *data_ptr;
  mxClassID classid;

  if( nrhs != 2 ) {
    mexErrMsgTxt("Need exactly two inputs ... (filename,variable)");
  }
  if( nlhs ) {
    mexErrMsgTxt("Too many outputs.");
  }
  if( !mxIsChar(prhs[0]) || mxIsEmpty(prhs[0]) ) {
    mexErrMsgTxt("1st argument must be base filename string.");
  }
  if( !(mxIsNumeric(prhs[1]) || mxIsLogical(prhs[1]) || mxIsChar(prhs[1])) || mxIsSparse(prhs[1]) ) {
    mexErrMsgTxt("2nd argument must be full numeric or logical or char variable.");
  }
  nf = mxGetNumberOfElements(prhs[0]);
  filename = mxMalloc(nf+7);
  sp = mxGetData(prhs[0]);
  for( i=0; i<nf; i++ ) {
    filename[i] = *sp++;
    if( filename[i] == 0 ) {
      mexErrMsgTxt("Null (0) character detected in filename");
    }
  }
  strcpy(filename+nf,"_r.bin");
  mexPrintf("Opening Real File: %s\n",filename);
  fp = fopen(filename, "wb");
  if( !fp ) {
    mexErrMsgTxt("File for real part will not open.");
  }
  classid = mxGetClassID(prhs[1]);
  element_size = mxGetElementSize(prhs[1]);
  nelements = mxGetNumberOfElements(prhs[1]);
  data_ptr = mxGetData(prhs[1]);
  ndim = mxGetNumberOfDimensions(prhs[1]);
  dims = mxGetDimensions(prhs[1]);
  mexPrintf("Writing real part to file ...\n");
  fwrite( &classid, sizeof(classid),  1, fp );
  fwrite( &element_size, sizeof(element_size),  1, fp );
  fwrite( &nelements, sizeof(nelements),  1, fp );
  fwrite( &ndim, sizeof(ndim),  1, fp );
  fwrite( dims, sizeof(*dims), ndim, fp );
    elements_written = fwrite( data_ptr, element_size, nelements, fp );
    fclose(fp);
  if( elements_written != nelements ) {
    mxFree(filename);
    mexErrMsgTxt("ERROR: Not all elements were written to file.");
  }
  if( mxIsComplex(prhs[1]) ) {
    filename[nf+1] = 'i';
      mexPrintf("Opening Imag File: %s\n",filename);
      fp = fopen(filename, "wb");
      if( !fp ) {
      mxFree(filename);
        mexErrMsgTxt("File for imag part will not open.");
      }
      data_ptr = mxGetImagData(prhs[1]);
      mexPrintf("Writing imag part to file ...\n");
      fwrite( &classid, sizeof(classid),  1, fp );
      fwrite( &element_size, sizeof(element_size),  1, fp );
      fwrite( &nelements, sizeof(nelements),  1, fp );
      fwrite( &ndim, sizeof(ndim),  1, fp );
      fwrite( dims, sizeof(*dims), ndim, fp );
        elements_written = fwrite( data_ptr, element_size, nelements, fp );
        fclose(fp);
      if( elements_written != nelements ) {
        mexErrMsgTxt("ERROR: Not all elements were written to file.");
      }
  }
  mxFree(filename);
}

readvar.c
---------
// readvar.c reads a variable from file(s), one each for real and imag
// Companion file to writevar.c
// Syntax: var = readvar(filename)
//         filename = char string with base filename
// Note: reads real part from filename_r.bin, and imag part from file filename_i.bin
//       gives an error if the files do not contain exactly same number and type of data
// Programmer: James Tursa
// Date: Feb 7, 2013
// Building: mex readvar.c

#include "mex.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#ifndef MWSIZE_MAX
#define  mwSize    int
#endif

void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
  FILE *fp;
  char *filename;
  short *sp;
  mwSize i, nf, nelements_r, nelements_i, ndim_r, ndim_i;
  mwSize element_size_r, element_size_i;
  mwSize *dims_r, *dims_i;
  size_t elements_read;
  void *data_ptr;
  mxClassID classid_r, classid_i;

  if( nrhs != 1 ) {
    mexErrMsgTxt("Need exactly one input ... (filename)");
  }
  if( nlhs > 1 ) {
    mexErrMsgTxt("Too many outputs.");
  }
  if( !mxIsChar(prhs[0]) ) {
    mexErrMsgTxt("Argument must be base filename string.");
  }
  nf = mxGetNumberOfElements(prhs[0]);
  filename = mxMalloc(nf+7);
  sp = mxGetData(prhs[0]);
  for( i=0; i<nf; i++ ) {
    filename[i] = *sp++;
    if( filename[i] == 0 ) {
      mexErrMsgTxt("Null (0) character detected in filename");
    }
  }
  strcpy(filename+nf,"_r.bin");
  mexPrintf("Opening Real File: %s\n",filename);
  fp = fopen(filename, "rb");
  if( !fp ) {
    mexErrMsgTxt("File for real part will not open.");
  }
  mexPrintf("Reading real part from file ...\n");
  fread( &classid_r, sizeof(classid_r),  1, fp );
  fread( &element_size_r, sizeof(element_size_r),  1, fp );
  fread( &nelements_r, sizeof(nelements_r),  1, fp );
  fread( &ndim_r, sizeof(ndim_r),  1, fp );
  dims_r = mxMalloc(ndim_r*sizeof(*dims_r));
  fread( dims_r, sizeof(*dims_r), ndim_r, fp );
  plhs[0] = mxCreateNumericArray( ndim_r, dims_r, classid_r, mxREAL );
  data_ptr = mxGetData(plhs[0]);
    elements_read = fread( data_ptr, element_size_r, nelements_r, fp );
    fclose(fp);
  if( elements_read != nelements_r ) {
    mxFree(filename);
    mexErrMsgTxt("Not enough values were read from the real file");
  }
  filename[nf+1] = 'i';
  mexPrintf("Opening Imag File: %s\n",filename);
  fp = fopen(filename, "rb");
  mxFree(filename);
  if( !fp ) {
    mexPrintf("No Imag File ... returning real variable.\n");
    mxFree(dims_r);
    return;
  }
  mexPrintf("Reading imag part from file ...\n");
  fread( &classid_i, sizeof(classid_i),  1, fp );
  fread( &element_size_i, sizeof(element_size_i),  1, fp );
  fread( &nelements_i, sizeof(nelements_i),  1, fp );
  fread( &ndim_i, sizeof(ndim_i),  1, fp );
  dims_i = mxMalloc(ndim_i*sizeof(*dims_i));
  fread( dims_i, sizeof(*dims_i), ndim_i, fp );
  if( classid_i != classid_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag classid");
  }
  if( element_size_i != element_size_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag element size");
  }
  if( nelements_i != nelements_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag number of elements");
  }
  if( ndim_i != ndim_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag number of dimensions");
  }
  for( i=0; i<ndim_i; i++ ) {
      if( dims_i[i] != dims_r[i] ) {
      mxFree(dims_i);
      mxFree(dims_r);
        mexErrMsgTxt("Mismatch Real vs Imag dimensions");
      }
  }
  mxFree(dims_i);
  mxFree(dims_r);
  data_ptr = mxMalloc(nelements_i*element_size_i);
    elements_read = fread( data_ptr, element_size_i, nelements_i, fp );
    fclose(fp);
  if( elements_read != nelements_i ) {
    mxFree(filename);
    mexErrMsgTxt("Not enough values were read from the imag file");
  }
  mxSetImagData(plhs[0],data_ptr);
}

[Reynaldo]

Wow these are awesome! Thank you so much for your time! I am very grateful!

[James Tursa]

If the file sizes are still too big, the next step would be to have the mex routines split up each of the real & imag parts into multiple files. Let me know if you need that.

[James Tursa]

For R2018a or later, see this FEX submission for reading and writing interleaved complex variables:

https://www.mathworks.com/matlabcentral/fileexchange/77530-freadcomplex-and-fwritecomplex

Answer 2

[Azzi Abdelmalek]

x = complex(rand(1,1), rand(1,1));
a=[real(x) imag(x)]
fid = fopen('test.bin', 'w');
fwrite(fid, a, 'double');
fclose(fid);
clear
fid = fopen('test.bin', 'r');
b = fread(fid, 2, 'double');
b=b(1)+j*b(2)
fclose(fid);

[Reynaldo]

how can I accomplish this using a vector containing more than one complex number? For example:

x = exp(i*rand(4,1))

Thanks!

[Azzi Abdelmalek]

x = exp(i*rand(4,1))
a=[real(x) imag(x)]
fid = fopen('test.bin', 'w');
fwrite(fid, a, 'double');
fclose(fid);

% to read

clear
fid = fopen('test.bin', 'r');
b = fread(fid, 8, 'double');
b=b(1:4)+j*b(5:8)
fclose(fid);

[Reynaldo]

Thank you!

Answer 3

[Image Analyst]

Why not just use save() to store it in a .mat file (which is a binary file)?

[Reynaldo]

Because for some reason MatLab would not let me save files larger than 2GB in size... I am working with 4GB > files.

[James Tursa]

What is the size of the variable you are trying to write / retrieve? You might consider a mex routine, which could avoid temporary copies of large amounts of data during the writing and retrieving process.

[Reynaldo]

I am working with vectors containing 2^28 elements (complex numbers). Each complex number takes 16 bytes. What do you mean by a mex routine? Thanks!

[James Tursa]

A mex routine is a C/C++ (or Fortran) compiled routine that interfaces with MATLAB just like a regular function. I will write one up tonight and post it for you.

[Reynaldo]

Allright! Thank you so much!

[James Tursa]

Worked on this some last night but didn't have time to finish it. I will try again tonight ...

[Reynaldo]

Dont worry! I will wait until you have time to finish it. Thank you so much James!