Aggregating selected variables from multiple mat files

I have 276 mat files (each corresponds to a month for a specific year starting 1993 till 2015). In these files I have numeric vectors with the same length. Some of these vectors cease to exist after a certain point in time. Other new vectors appear at some point in time. I want to aggregate all these vectors in 1 mat file where all vectors have the same length (which means when a vector ceases to exist it will be filled with nans for the remaining periods and when a vector appears the preceding period will be filled with nans). Any advice about how to do that is greatly appreciated.What facilitates the issue is that one vector 'rssd9999' exists in all 276 mat files. I started a code that led nowhere using the eval function (didn't know how to proceed). I think there are easier methods . Basically what i wanted to say in code format:

clear all
files = dir('bhcf*.mat');
for i = 1:length(files)
load(files(i).name);
a=who('BHC*','rssd9999','RSSD9999','rssd9001','RSSD9001');

      for j=1:length(a)
      v1 = genvarname(['v' a{j}]);
      index= isvarname(a{j});
          if index==1 && i==1
          % here I want to say v1=v2; for example vBHCK0010=BHCK0010; 
          %I tried eval([v1 '=a{j};']);   but it didn't work
          elseif index==0 && i==1 
              v1=nan(size(rssd9999);
           elseif v2==1 && i>1 && the variable existed previously
          % here I want to say that v1=[v1;the variable a{j}]
          elseif v2==0 &&  i>1 && the variable existed previously
           v1=[v1;nan(size(rssd9999)] ;
          elseif v2==1 && i>1 && the variable didn't exist previously
          % here I want to say that v1=[nan(size(vrssd999));the variable a{j}]
          elseif v2==0 &&  i>1 && the variable didn't exist previously
           v1=[nan(size(vrssd999));nan(size(rssd9999)] ;
          end
      end
  end

[José-Luis]

How do they "cease to exist" after a certain time?

[wesso Dadoyan]

because of a change in regulations. these vectors will not be adopted when the regulations decide to create a different measure.

[José-Luis]

1. Store your variables in a cell array. Dynamically generated variable names are a terrible idea.
2. If you are sure they are the same length (not obvious to me because of the "cease to exist" then preallocate an array of the adequate size outside the loop and then store your variable in each row (or column).

[wesso Dadoyan]

but the variables are changing from one month to another.I mean that the length of vectors of each month are the same. which means when a variables cease to exist I need to keep on filling it with nans to keep the size of the output of equal lengths)

[José-Luis]

I don't understand the issue: change your variable as needed beforehand and place in the array.

Answer 1

[Guillaume]

As I've just said in another question: it is better to load your variable into a structure, simply by giving an output to load, then you can use dynamic field names to access your variables.

The way, I'd do it is load your variables into a 2D cell array (with rows: file, columns: variable name), growing the columns as new variables appear, and at the end fill empty cells with nan arrays of the right size:

Warning, untested code, there may be bugs / typos:

files = dir('bhcf*.mat');
monthlydata = cell(numel(files), 0);  %starting empty
varnames = {};  %correspond to columns of monthly data, starting empty
for fileidx = 1 : numel(files)  %numel is safer than length
   matcontent = load(files(i).name);  %get content of file as structure
   matvars = fieldnames(matcontent)';  %list of variable in file (transposed into a row vector)
   matcontent = struct2cell(matcontent);  %convert structure into cell array to make it easier to index
   varstokeep = ~cellfun(@isempty, regexpi(matvar, '^BHC.*$|^rsdd(9999|9001)$')); %only keep variables 'BHC*','rssd9999','RSSD9999','rssd9001','RSSD9001'
   matcontent = matcontent(varstokeep);
   matvars = matvars(varstokeep);
   [varexists, varcols] = ismember(matvars, varnames);  %identify which variable already have a column in the cell array (and which column)
   monthlydata(fileidx, varcols) = matcontent(varexists); copy existing variable into corresponding row and columns
   numnewvars = sum(~varexists);
   monthlydata = [monthlydata, cell(numel(files), numnewvars)]; %grow by the number of non-existing variables
   monthlydata(fileidx, end-numnewvars+1 : end) = matcontent(~varexists); and copy those new variables at the end
   varnames = [varnames, matvars(~varexists)];  %append new variable names into list of variables
end
%everything is loaded, now fill up empty cells with nans. 
%Assuming that the size can vary by variable iterate over the variables
for varidx = 1 : numel(varnames)
   emptyrows = cellfun(@isempty, monthlydata(:, varidx));
   nonemptydata = monthlydata(~emptyrows, varidx); %cell array containing the non-empty rows for that variable, 
   montlydata{emptyrows, varidx} = nan(size(nonemptydata{1})); %fill up with nan of the size of the first non-empty matrix for that column
end

I would certainly not create an endless list of new variables or you'll be back to similar problems when you want to process the lot later on.