How do I fix this uigetfile multiselect error?

I have developed a script to process raw data from wind turbine noise. I am attempting to adapt it to automatically process a number of files as opposed to running the script for each file individually. The script is shown below:

function ruk_am_v1_OPTI
%%Initiatilise Variables and Setup Environment
% RenewableUK AM Assessment Program - V1 December 2013
% ########################################
% Clear all variables and workspace data from system memory   
clear all;
% Clear command window
clc;
% Clear all old figures
close all;

% Configure run and initialise all variables
repository          = 'C:\';
blockTime           = 10;   % Main analysis interval (here 10 sec)
sampPerSec          = 10;   % Number of Leq periods per second (here 10 Hz, or 100 msec)
maxBlocks           = 1440; % Maxiumum number of blocks program can process at once (4 hours)

%%Ask user to specify blade passing frequency
      bladePassingFrequency = 1.25;
%%Select data file using UI control
[filename,pathname] = uigetfile('*.txt','Select the TXT file to process',repository,'Multiselect','on');

if isequal(filename, 0)
 disp('User selected Cancel')
 return;
end
for n = 1:length(filename)
  % Create name for output file and open for output
  filename   = lower([pathname,filename(n)]);
  filenameOut= strrep(filename(n), 'txt', 'out');
  filenameOut= char(filenameOut);
  fid        = fopen(filenameOut, 'w');

    %%Read input data
    blockLength = blockTime * sampPerSec;

    % Open input TXT file and read first line, which contains number of items
    filename=char(filename);
    fidtxt = fopen(filename(n), 'r');
    dummy = fgetl(fidtxt);
    totalSamples = str2double(dummy);
    LeqWholePeriod = zeros(1,totalSamples);
end
% Create progress bar for reading input data
h = waitbar(0,'Reading LAeq,100msec levels from TXT file...');
for i = 1:totalSamples
    dummy = fgetl(fidtxt);
    LeqWholePeriod(1,i) = str2double(dummy);
    waitbar(i/totalSamples);
    end

close(h);
% Close input TXT data file
fclose(fidtxt);

% Determine how many 10 sec periods the input TXT file contains
numBlocks = max(1,floor(totalSamples / blockLength));

% Check that this is not more than maxBlocks, which is the maximum
if numBlocks > maxBlocks
  msgBox('Cannot process more than ''maxblocks'' blocks','Program Termination Warning','warn');
  return;
end

%%Put information on screen and in 'Out' file
for fileHandle = [1,fid]
  fprintf(fileHandle,'Input file name           %s\r\n',filename);
  fprintf(fileHandle,'Output file name          %s\r\n',filenameOut);
  fprintf(fileHandle,'Number of samples in file %9i\r\n',totalSamples);
  fprintf(fileHandle,'No of samples per sec.    %9i\r\n',sampPerSec);
  fprintf(fileHandle,'Duration of file          %9.2f s\r\n',totalSamples/sampPerSec);
  fprintf(fileHandle,'Length of data blocks     %9i s\r\n',blockTime);
  fprintf(fileHandle,'No of data blocks         %9i\r\n',numBlocks);

    % Detrend using 5th order polynomial
    fprintf(fileHandle,'Detrending Method.        Polynomial(');
    fprintf(fileHandle,'Order  5)\r\n');
    fprintf(fileHandle,'Blade passing frequency   %9.2f Hz (User chosen)\r\n',bladePassingFrequency);
end

% Print file headers for output 'Out' file
fprintf(fid,'\r\n');
fprintf(fid,'   Block No.|   Spec Line|      Frequency/Hz| Raw Spec Level/dB| Int Spec Level/dB\r\n');

%%Now loop over the whole input file in 'blockTime' second blocks
n2 = 0;
hold off

% Start processing each block in turn
for iblock = 1:numBlocks
  % Block counter
  disp(['Processing Block Number ',num2str(iblock),' of ',num2str(numBlocks)]);
  t = 0:1/sampPerSec:blockTime-(1/sampPerSec);
  n1 = 1 + n2;
  n2 = min(blockLength,totalSamples) + n2;
  Leq   = LeqWholePeriod(1,n1:n2);
  nsamp = length(Leq);
  time  = nsamp / sampPerSec;
  if blockLength ~= nsamp(1,1)
      msgbox('blockLength differs from nsamp!','Program Termination Warning','error');
  end
  if blockTime ~= time(1,1)
      msgbox('blockTime differs from time!','Program Termination Warning','error');
  end       

    %%Check for bad data

    % Calculate Leq for 'blockTime' seconds
    LeqTotal = 10*log10(1/sampPerSec*(sum(10.^(Leq/10)))/blockTime);

    % Test whether max value of Leq is > 10 dB more than mean
    % (implies non-stationarity rather than error) or data which
    % has spikes, i.e. overal level >= 60 dB(A)
    if max(Leq) - mean(Leq) > 10
        iBad = 1; % set to bad data
    elseif LeqTotal > 60
        iBad = 1; % set to bad data
    else
        iBad = 0;
    end

If block data NOT bad then proceed, otherwise discard it

    if iBad == 0
        %%Detrend and plot the Leq Data
        p = polyfit(t,Leq,5);
        deT = polyval(p,t);
        Leq = Leq - deT; % detrend data
        %%Now perform the spectral analysis of the detrended data
        % Create spectrum object and call its PSD method.
        h = spectrum.periodogram('Rectangular');
        hopts = psdopts(h,Leq);
        set(hopts,'Fs',sampPerSec,'SpectrumType','onesided');
        amspec = psd(h,Leq,hopts);

        % Extract results from object
        Fw = amspec.Frequencies;
        Pw = amspec.Data;
        FreqRes = Fw(2) - Fw(1);

        maxFreq = sampPerSec/2;

        Pw_max = 10.0;
        Pw_min = 0;

        % Convert spectrum results to dB levels
        ampMod = 2*sqrt(2*FreqRes*Pw);

        %%Integrate the spectrum to determine the 'true' level
        % Integrate the PSD to determine the energy in a critical band. The
        % critical band is defined as the +/-10% of the BPF
        critBand = 2 * 0.1 * bladePassingFrequency;
        % determine the number of frequency intervals this includes.
        windowSize = floor(critBand /FreqRes) + 1;
        if mod(windowSize,2) == 0
            halfWindow = windowSize/2;
        else
            halfWindow = (windowSize - 1)/2;
        end

        intAmpMod = zeros(1,length(Fw));
        % Calculate rolling sum over entire spectrum
        for j = 1:length(Fw)
            if j <= halfWindow
                % Portion of spectrum < halfWindow from beginning
                intAmpMod(j) = sum(Pw(1:j + halfWindow));
            elseif j >= length(Fw)- halfWindow + 1
                % Portion of spectrum < halfWindow from end
                intAmpMod(j) = sum(Pw(j - halfWindow:length(Fw)));
            else
                % rest of spectrum
                intAmpMod(j) = sum(Pw(j - halfWindow:j + halfWindow));
            end
        end

        % Convert to dB units, as before
        intAmpMod = 2 * sqrt(2 * FreqRes * intAmpMod);
        % Print out results to text file
        for j = 1:length(Fw)
            fprintf(fid,'%12i|%12i|%18.6f|%18.3f|%18.3f\r\n',iblock,j,Fw(j),ampMod(j),intAmpMod(j));
        end
    else %iBad ~= 0
        fprintf(fid,'%12i|  Bad data - affected by ',iblock);
        fprintf(fid,'high energy peaks - not analysed\r\n');
    end
end
%%Close all files and exit
% Turn warnings back on
warning('on','all');
% Close 'Out' text file
fprintf(fid,'\r\nRun Date: %s',datestr(now));
fclose(fid);

% Alert user that data processing is complete
msgbox('Data processing complete!','Warning Message!','warn');

The error I am receiving is as follows:

Error using fgets
Invalid file identifier. Use fopen to generate a valid file identifier.

Error in fgetl (line 33)
[tline,lt] = fgets(fid);

Error in ruk_am_v1_OPTI (line 52)
dummy = fgetl(fidtxt);

Does anyone know how to correct this?

Thanks.

Answer 1

[W. Owen Brimijoin]

Don't know if this is the cause of your problem, but I can say that uigetfile in 'MultiSelect' mode has the bothersome behavior where it returns a variable of a different class depending on whether the user picks one or more than one file. Very frustrating! In the first instance your variable 'filename' will be a char, so

fopen(filename{n}, 'r');

will fail, saying 'Cell contents reference from a non-cell array object.' Which is true, because it's not a cell array! Similarly if you use brackets:

fopen(filename(n), 'r');

will fail, but this time it's because you have specified the filename as the first letter in the name of the file. The first method works if the user has picked more than one file... so I would advise you to first check the number of files selected as follows:

n_files = numel(cellstr(filename));

because this will accurately report the number of files selected, instead of reporting the number of files as being the number of characters in the name of the one file you picked, then loop through n_files using an if statement for the special case of one file selected:

for file = 1:n_files,
    if n_files==1, 
        filenameOut = filename;
    else filenameOut = filename{file};
    end

      load(filenameOut)
      %load your shiny new data and do something here
  end

Does this make any sense?

[Gavin Brown]

I think this has got me on the right track because the script is now actually running however it only returns one output file from the multiple .txt files collected. This is what i've changed it to:

n_files = numel(cellstr(filename));
for file = 1:n_files,
  if n_files==1, 
    filenameOut = filename;
  else 
    filenameOut = filename{file};
  end
  load(filenameOut)
end    
% Create name for output file and open for output
filename    =  lower([pathname,filenameOut]);
filenameOut = strrep(filename, 'txt', 'out');
fid         =  fopen(filenameOut, 'w');
%%Read input data
blockLength = blockTime * sampPerSec;
fidtxt = fopen(filename, 'r');
dummy = fgetl(fidtxt);
totalSamples = str2double(dummy);
LeqWholePeriod = zeros(1,totalSamples);

Can you see where i'm still going wrong?

Thanks for all your help so far!

[dpb]

doc debug

I recommend looking at the solution to the single string/multiple cellstr conundrum I posted above as being simpler; then you won't need the additional case processing.

I don't see an obvious error above, but there's no data here to test against so looks like at this point it's probably simplest for you to step thru the loop that is trying to open the files and see where it is failing. Looking at the results you get as you do that will likely show you the error in your ways... :) and more quickly than we can debug from afar.

[Gavin Brown]

I've tried but I'm not having much success. Do you think the problem with the outputs could be something to do with this section?

%%Put information on screen and in 'Out' file
for fileHandle = [1,fid]
  fprintf(fileHandle,'Input file name           %s\r\n',filename);
  fprintf(fileHandle,'Output file name          %s\r\n',filenameOut);
  fprintf(fileHandle,'Number of samples in file %9i\r\n',totalSamples);
  fprintf(fileHandle,'No of samples per sec.    %9i\r\n',sampPerSec);
  fprintf(fileHandle,'Duration of file          %9.2f s\r\n',totalSamples/sampPerSec);
  fprintf(fileHandle,'Length of data blocks     %9i s\r\n',blockTime);
  fprintf(fileHandle,'No of data blocks         %9i\r\n',numBlocks);

    % Detrend using 5th order polynomial
    fprintf(fileHandle,'Detrending Method.        Polynomial(');
    fprintf(fileHandle,'Order  5)\r\n');
    fprintf(fileHandle,'Blade passing frequency   %9.2f Hz (User chosen)\r\n',bladePassingFrequency);
end

[dpb]

No klew, you don't give any hints as to what "the problem" is, even...

But, the loop looks highly suspicious, as [1, fid] is a vector beginning with the number '1' followed by what may (or may not be) a valid file handle based on the variable name.

Is the purpose to echo file output to the screen followed by to a file?

Have you verified fid is, in fact a valid open handle?

What is the problem, precisely???

[Gavin Brown]

Apologies if im being a bit vague here, I'm not very experienced with matlab (or programming at all actually) and I'm trying to adapt an original script which processes a single .txt file (successfully) to process multiple files. So I don't exactly know what everything does unfortunately.

I mentioned in the 1st comment on this answer that the script is now actually running however it only returns one output file (the 9th of 10 files in this particular case) from the multiple .txt files collected.

fid = fopen(filenameOut, 'w');

[dpb]

OK, I reread your code more carefully...there was so much posted to wade thru I kinda' glossed over it.

I think the problem comes from the way you tried to mung on the single-file case when you went to multiple cases...

I thought it seemed like you were using the variable filename awfully loosely but I presumed (always get in trouble doing that, doesn't one? :) ) this wasn't the problem only the reference of char vis a vis cell when moving over to 'multiselect' mode.

But, au contraire, you did something more subtle but quite damaging--you had the statement of

filename= lower([pathname,filename{n}]);

ASIDE The above will cause issues on any UNIX-like OS on which file names are case-sensitive. It's also a little "user-belligerent" in that you're changing a filename perhaps that the user has carefully CamelCased for legibility or even uniqueness in which case it may also actually do harm...

Returning to regularly scheduled programming after editorial comment...

where we just fixed up the reference to the cell. What happens, though is that you reassigned the whole array on the LHS to the one element of the n th file only. Fix this by turning the LHS into a new "one-at-a-time" variable to contain only the one file you want each iteration.

Here is a bare-bones routine that should work through the number of file you have selected and open each and an associated output file for each or tell you what happened if it doesn't.

repository = 'C:\';
[filename,pathname] = uigetfile('*.txt','Select the TXT file to process',repository,'Multiselect','on');
if isequal(filename, 0)
  disp('User selected Cancel')
  return;
end
% Fixup the one-file case using multi-select
if isstr(filename), filename={filename}; end
for n = 1:length(filename)
  % Create name for output file and open for output
  fname    = lower([pathname,filename{n}]);
  filenameOut = strrep(fname, 'txt', 'out');
  [fid,msg]   = fopen(filenameOut, 'w');
  if fid==-1, error([msg ':  ' filenameOut]),end % error w/ msg
  fidtxt = fopen(fname, 'r');
  if fidtxt==-1, error([msg ':  ' fname]),end % error w/ msg
  % close open handles...
  fidtxt=fclose(fidtxt)
  fid=fclose(fid);
end

This doesn't do any processing but it should get you through the opening and you can study it to see what you need to clean up in the mod's you made in the original.

[Gavin Brown]

Yeah that works for the 1st part cheers! Should I be able to sort out the processing with a similar loop?

[Gavin Brown]

Having real difficulty in doing so, I'm either getting 1 processed file and the rest blank or 1 file processed the number of times equal to the total number of files and the rest blank.

[dpb]

On the first, it is your loop with the one change of adding the new variable fname as the qualified name for each iteration for the input file to be processed and then the substitution of 'out' for the extension for the output file.

On the latter, I can't see what you've done with the processing but it doesn't seem like should be more than what I outlined above wrapped around the processing after where the files area actually opened--

a) return the array selected names from uigetfile as filename

b) fixup for the one-only case

c) begin the loop over the number in the array

d) store the nth filename(n) in the temporary fname

e) use that in place of where the filename array variable is in the original loop.

As outlined, the problem before was (or well, included besides the issue of cell versus character strings) that it was ok to use/modify the returned name variable when was only one but "you can't do that!" when it's an array and you have to process all of them. That is what the original code did on the line where the lower casing is being done.

I personally, still don't like the idea of doing that (the case change,that is) to the user, but if you're the only user and you're ok with it, then fine; just be aware that there are case-sensitive file systems and remember the comment simply on legibility.

ADDENDUM

OK, I went back to your original posting yet again and reformatted the early portion so it's more legible. I found the following...

for n = 1:length(filename)
  % Create name for output file and open for output
  filename   = lower([pathname,filename(n)]);
  filenameOut= strrep(filename(n), 'txt', 'out');
  filenameOut= char(filenameOut);
  fid        = fopen(filenameOut, 'w');
  %%Read input data
  blockLength = blockTime * sampPerSec;
  % Open input TXT file and read first line, which is number of items
  filename=char(filename);
  fidtxt = fopen(filename(n), 'r');
  dummy = fgetl(fidtxt);
  totalSamples = str2double(dummy);
  LeqWholePeriod = zeros(1,totalSamples);
end
% Create progress bar for reading input data
h = waitbar(0,'Reading LAeq,100msec levels from TXT file...');
for i = 1:totalSamples
...

Now it's possible to see you closed off the for...end loop over the number files after simply opening them and creating/opening the output files sequentially, but you don't begin any processing until after that loop is done. Hence all the first N-1 file handles are orphaned as you've reused the handle variable but the files are still open and you have no way to access them to close them other than the 'all' club.

This loop must enclose the entire processing engine from open to close of the single-file process. Hence this end statement goes somewhere way, way down at or very near the absolute end of the function.

I cannot emphasize strongly enough that you need to format the function clearly with proper nesting for logical structure to be able to see at a glance what is/isn't in a given construct. While the spacing doesn't matter to the interpreter, it's a major boon to the programmer.

[dpb]

OK, per the last comment I pasted the original function into the code editor and think I've made the above corrections...you'll have to test.

I attached the copy from here to the original answer, good luck...

BTW, how long does it take to read a typical one of those text files? The line-by-line via fgetl and then a str2double on the single value retrieved is as slow a way as could possibly have chosen...if this taking some time as one would presume is so given they coded a waitbar, ought to seriously recode this part; there's no sense int wasting time unnecessarily....

[Gavin Brown]

Thanks a lot for all your help it works now! Sorry for the delay I've been working in a different department the last few weeks and this was completely void from my mind. Yeah it probably could be quicker, it only takes like 2-3 seconds to process the files but the waitbar you mentioned takes about 30 seconds. However, for some reason, if I remove it the final 2 columns of the table it produces are all just zeros. Little bit annoying but its quick enough for its purpose really and as I mentioned I'm not the most experienced with MATLab to put it lightly.

Cheers again!

[Gavin Brown]

In fact I've removed the waitbar and each files only takes a second or 2 now.

[Gavin Brown]

I'm now trying to save all the files into a folder. I've created the folder where I want it with:

prompt1 = 'Name the destination folder:';
dlg_title1 = 'Create a destination folder';
num_lines = 1;
folder = inputdlg(prompt1,dlg_title1,num_lines);
m=cell2mat(folder);
str=sprintf('/%s',m);
mkdir('\\Biggar\noise\out. files',str);

The problem is I cant seem to get the files to save into the folder...

Any ideas?

[dpb]

Well, you show nor give no hint as to either the error(s) received nor even the code with which you tried to save the files, so, "no"...

Answer 2

[dpb]

In future, please trim amount of code to include only significant sections...this is a lot to wade thru to find the pertinent lines only...

Here's your problem...

...
filename=char(filename);
fidtxt = fopen(filename(n), 'r');
...

Don't convert the full array to character; simply the one cellstring needed when need to...

fidtxt = fopen(filename{n}, 'r');

"Use the curlies, Luke..."

AMPLIFICATION

As mentioned, have to cleanup TMW's mess when use the multiselect option...the portion for the input file open would then look something like--

% Select data file using UI control
[filename,pathname] = uigetfile('*.txt', ...
           'Select the TXT file to process', ...
           repository, ...
           'Multiselect','on');
if isequal(filename, 0)
  disp('User selected Cancel')
  return;
end
% If user only selected one, it'll be a character string instead
% of cell string; simplest is to convert to cellstr for later
if isstr(filename), filename={filename}; end
% now length() is always over the cell array size as needed
for n = 1:length(filename)
  ...
  fidtxt=fopen(filename{n}, 'r');
  ...

ADDENDUM

Tutorial info--

Once you converted the whole array, then the subscript n refers only to a single character, not a cell. But, fopen and friends haven't been "smartened up" to know how to deal with a cellstring input so you have to ensure you return the character string stored in the cell, not the cellstring. Somewhat subtle, yes, but a critical difference to learn.

[Gavin Brown]

Okay I've removed that line and replaced the brackets around n with the curlies but I'm still getting the same error? Any more ideas?

Thanks for your help

[dpb]

OK, W.Owen reminded me of the issue with uigetfile and the MultiSelect option being as I tend to say, "user belligerent" in that it returns a different class result depending on what the user does. This is a pit(proverbial)a(ppendage) and TMW should be seriously castigated for having done it to us...anyway, as he shows, you need to test the result when using the multiple selection option...

I have run into the same problem in the past but it's been long-enough ago that it slipped my mind earlier...

See the updated response for my solution that I think is a little simpler than his altho his does work...