File manipulation

Dear Matlabians

I have done a script that takes csv and txt files manipulates them and produces a final xls with the data in those files. Each time i take on specific file and then I continue to the next one.

But sometimes there is some files that the data in there are wrong. In my script I have found a way to check this and I know which files will be wrong. And I have made the script to stop at those points

The question is: Is there a way during the script is stop to open the file which is wrong to maniputale it by hand (delete the data that are wrong) then save the file again and make the script to restart for this file and continue as it suppose to continue

Thank you

Answer 1

[Fangjun Jiang]

Yes. Use try-catch and keyboard() to control the program flow.

try
  ProcessFileForTheFirstTime();
catch
  keyboard;
  % Outside MATLAB, OpenTheProblematicFile();
  % Outside MATLAB, ManualEditAndSaveFile();
  ProcessFileForTheSecondTime();
end

The keyboard() function can be used to pause your program till you type return.

[Alexandros]

Than you very much to both of you. The second answer is the one that worked for me. I am sure that the first one is also good but the second is a little more clean. :)))

My script understands where to put the data in the final xls. Thus when I stop the program at keyboard is it possible to continue ProcessFileForTheSecondTime() (which is a function that i call) from the file it found the error. Like this i don't have to re-do all the manipulation of data from the beginning.

I was thinking maybe to remake the same fuction i use for the ProcessFileForTheFirstTime() but with a little diferent inputs

[Fangjun Jiang]

That will be hard because you don't know where the error occurred. It could be different every time, right. The easy way is just re-do the processing. Hope the processing won't take too long. ProcessFileForTheFirstTime() and ProcessFileForTheSecondTime() could be the same function. If no error, only the statement in "try" section is executed.

Answer 2

[Walter Roberson]

fileinfo = dir();

fileinfo([fileinfo.isdir]) = [];  %discard directories

wanthisfile = cellfun(@(Name) ismember(Name(end-2:end),{'txt','csv'}), {fileinfo.name});
fileinfo(~wantthisfile) = [];  %isolate to .txt and .csv

for K = 1 : numfiles
  thisfile = fileinfo(K).name;
  while true
    try
      ProcessFile(thisfile);
      break;  %this file worked so leave while
    catch
      fprintf(1, 'problem with file %s', thisfile);
      keyboard;
      % Outside MATLAB, OpenTheProblematicFile(thisfile);
      % Outside MATLAB, ManualEditAndSaveFile(thisfile);
      %and then allow ourselves to loop back to try run again
    end  %end try/catch
  end    %end while true
end      %end file loop

[Fangjun Jiang]

try-catch is enough. I don't see the need to add the while-loop.

[Sean de Wolski]

it allows the user to continue through the while loop automagically after fixing the file.

[Sean de Wolski]

Not sure what it gets you voer a for-loop though since in theory we knwo the number of files. Though that could be contained in ProcessFile

[Fangjun Jiang]

I guess the benefit of adding the while-loop is to allow the user to fail multiple times in fixing the file.

Answer 3

[Sean de Wolski]

Hmm. One way would be to have a variable that is the index to the files you wish to open. Run a for-loop that traverses that variable. When you hit a bad file, dump the for-loop, edit the iteration variable to start at that file, and provide you with a friendly (or angry) message to do some editting: pseudoishcode

%%initial file list
file_idx = 1:10;

%%Cell to run manually until complete

      [dump_idx] = go_through_files(file_idx);
      errordlg(['Failed file: ' num2str(file_idx(dump_idx))]);
      file_idx = file_idx(dump_idx:end);

%fix stuff
%manually restart this cell now!

Go through files function

function dump = go_through_files(fidx)

for ii = 1:length(fidx)
    if file(fidx(ii)) == fail
        dump = ii;
        return
    else
        process_file(fidx(ii))
    end
end
dump = 'Complete!';