How to optimize my code ?

I am writing a code to read multiple images from a folder which contains multiple sub folders and different type of files. I get a list of all sub folders and look for *.tif images and then analyze them.

Here's the part of code that walks through the folders and reads the image (I found this method under one of the posts in this community) :

start_path = fullfile( 'C:\Project\');
% Get list of all subfolders.
allSubFolders = genpath(start_path);
% Parse into a cell array.
remain = allSubFolders;
listOfFolderNames = {};
while true
  [singleSubFolder, remain] = strtok(remain, ';');
  if isempty(singleSubFolder)
    break;
  end
  listOfFolderNames = [listOfFolderNames singleSubFolder];
end
numberOfFolders = length(listOfFolderNames);

% Process all image files in those folders.
for k = 1 : numberOfFolders
    thisFolder = listOfFolderNames{k};
    filePattern = sprintf('%s/*.tif', thisFolder);
  baseFileNames = dir(filePattern);
    numberOfImageFiles = length(baseFileNames);
  % Now we have a list of all files in this folder.
  
  if numberOfImageFiles >= 1

    % Go through all those image files.
    for f = 1 : numberOfImageFiles
      fullFileName = fullfile(thisFolder, baseFileNames(f).name);
      % fprintf('     Processing image file %s\n', fullFileName);
                      I = imread(fullFileName);

%%And Some Analysis..

So the problem is that imread takes too much time. I ran it on sample of about 8000 images and it took 1600 sec(~25min) for the whole code but just imread took about 1100sec. that's a lot of time just to read.

Is this normal with imread or can i try and optimize this. i am only gonna look for *.tif files and if there's a way that i could speed up the reading process, it would be great.

This particular function takes up about 80% of time in imread : " imagesci\private\readtif " and it's child function is : " imagesci\private\rtifc " which takes up 99% of its time.

Any help is appreciated, ThankYou!

[Muthu Annamalai]

Can you only load the files you need instead immediately, instead of building a vast library in memory ?

If not I would try to consolidate all the images under one sub-folder as a single image and load it.

It is well known that CPU speed is faster than I/O and it seems it manifests itself in your code.

[dp sb]

I can't really consolidate all the images into one sub-folder. I am gonna be dealing with more than a million images which are under one directory in multiple sub folders.

[shefna manaf]

@Muthu Annamalai-can u plz tell how to consolidate as a single image?

[Walter Roberson]

shefna manaf: do you mean something like montage() ?

Answer 1

[Walter Roberson]

If you are going to end up reading the same image multiple times (perhaps by running the same program with multiple parameters) then pre-process the images by reading them in and writing them out in binary form.

Depending on how you process the files, you might also find it effective to memory map the binary files into memory when you go to use them. If not then fread() them.

Also, if you have the Parallel Processing Toolbox, you can imread() inside a parfor()