Concatenate Structures: select structures only if not empty.

I'd appreciate help on merging data from different sources. After importing the data, I store them in structures. One structure for each data source, each structure has the same fields. So, concatenating the structures into one is straight forward:

DataMerged = [Data1; Data2; Data3; Data4];

Now, the tricky part: For some data sets (experiments), one or several of these structures can be empty, i.e. contain not even any fields. How would I need to modify the code above to concatenate only those structures that are not empty? Currently, I use this work-around:

if ~isempty(test) && ~isempty(test2) && ~isempty(test3) && ~isempty(test4)
    listFiles = [test; test2; test3; test4];
    disp('Concatenate 1-4')
end
if ~isempty(test) && ~isempty(test2) && ~isempty(test3) && isempty(test4)
    listFiles = [test; test2; test3];
    disp('Concatenate 1-3')
end

and so on, for all combinations. Would you know a more elegant way of doing this?Thanks, for any help and suggestions,

ThorBarra

Answer 1

[Jos (10584)]

This clearly shows the drawback of naming your variables dynamically, like A1, A2, A3, A4. If you change, for instance, the way you import variables into your workspace, this would be a trivial, non-exisiting, problem.

Let's assume that you load the data like this or something similar:

Data1 = import('Exp1.dat')
Data2 = import('Exp2.dat')

Change this to a loop, and life becomes much measier:

Files = {'Exp1.dat', 'Exp2.dat'}
for k=1:numel(Files)
    Data{k} = import(Files{k}) ;
end
tf = ~cellfun(@isempty, Data)
AllData = [Data{tf}] ;

[ThorBarra]

Thanks, Jos. Works like a charm. Here is the final code:

%% List specific files
% in directory and sub-directory

% IGOR format
TmpFileList{1} = dir([dataPath_XPS filesep '**' filesep 'N*.ibw']);
% h5 format
TmpFileList{2} = dir([dataPath_NEXAFS filesep '**' filesep 'N*.h5']);
% Photos
TmpFileList{3} = dir([dataPath_Photo filesep '**' filesep 'A*.bmp']);
% Photos
TmpFileList{4} = dir([dataPath_Photo filesep '**' filesep '201*.bmp']);

Do things to each list.....

%% Merge lists into one
%

% get index of non-empty structures
tf  =~cellfun(@isempty,TmpFileList);
% merge the lists into one structure (cat(1, A, B) is the same as [A; B].)
ListFiles = cat(1,TmpFileList{tf});
% delete all other variables
clearvars -except ListFiles

[Jos (10584)]

You're welcome :-)

Another suggestion: use fullfile to create filenames:

fullfile(dataPath_XPS, '**', 'N*.ibw')

Answer 2

[Walter Roberson]

Create a template that has all of the fields but no contents, such as

template_struct = struct('filename', [], 'is_loaded', [], 'SNR', []);

If you check with size() and fieldnames() you will see that this will be a struct of size 0 but which has the right fields.

Then

if isempty(test); test = template_struct; end
if isempty(test2); test2 = template_struct; end
if isempty(test3); test3 = template_struct; end
if isempty(test4); test4 = template_struct; end
DataMerged = [test1; test2; test3; test4];

[Walter Roberson]

Or just use:

DataMerged = [];
if ~isempty(test); DataMerged = [DataMerged; test]; end
if ~isempty(test2); DataMerged = [DataMerged; test2]; end
etc