Why does adding a specific directory to Matlab path cause the save function to slowdown?

I'm having the following problem: in my application I'm storing a few hundred files. Lately, I notice that the save function has a delay of 35 seconds. Over a few hundred files, this adds up to a few hours of extra runtime.

I think I have narrowed it down, but I've reached the end of my debugging skills (if any are present at all ;).

I'm expecting the following:

>> hop = 7
hop = 7
>> tic;save(fullfile(pwd, 'hop', 'hopjesvla'), 'hop');toc

Elapsed time is 0.034323 seconds.

Now, when I add a specific directory of my code to the Matlab path, I get the following:

>> tic;save(fullfile(pwd, 'hop', 'hopjesvla'), 'hop');toc

Elapsed time is 35.140016 seconds.

When the directory is removed from the Matlab path, save is fast again, so it seems the presence of this directory in the path is somehow influencing some behaviour of the save function. I've used the profiler, but that does not show anything inside the save function.

A number of remarks:

• I'm using Matlab under Windows, both 2011b or 2013b
• The save location is on a network drive
• The directory I'm adding to the Matlab path contains a fair number (about 60) of @class directories
• The directory is added to the end of the Matlab path.
• Providing a full path to the save function seems necessary. When I use save('hopjesvla', 'hop'), the delay is not triggered.
• The delay is consistently 35 seconds, causing me to think of a timeout or something like that.

Can anyone point to further items I can investigate? Is there a limit to the number of @class directories in the path? Can this be (network) filesystem related?

Any help is appreciated.

Update (and thanks for thinking with me):

@Matt J

A different folder still gives the delay. However, in experimenting I found that the current folder also plays a role. In some cases (I haven't narrowed it down yet) the delay does not happen if I change the current folder.

@Adam

It does not seem to matter either way. I used pwd to indicate that a full path was given to the save function, and TIC TOC to indicate the time delay. When I look at the time stamps of the created files, each file is separated by 35 seconds. I also found that the file is created immediately, but with a file size of 0. Only after 35 seconds, the file is filled with something. Can this be hinting at a buffer being flushed?

[Matt J]

What happens when you write to a different (but empty) path folder on the same network drive?

[Adam]

To properly isolate 'save' as the cause of the problem, pull out

filename = fullfile(pwd, 'hop', 'hopjesvla')

so you just have

tic; save( filename, 'hop' ); toc

in both cases. tic-toc is not an ideal way to time things, especially if you are just running on the command line, but for a difference between 0.03s and 35s it is certainly adequate!

Answer 1

[Sean de Wolski]

• The save location is on a network drive

^ This

What happens if you save locally? Network directories on the path, being the current directory, etc. are often the cause of lag like this.

If you're saving large files to a network eventually, I usually just do it locally and then use a tool designed specifically for moving files (Windows Explorer, FileZilla etc.) to actually move them over.

[Jan]

Using the network drive is indeed the point where it hurts. There seems to be some interaction between the network drive and Matlabs use of the save function that causes the behaviour I'm seeing. Saving to the local drive is very fast. However, the IT environment requires of me to use network drives.

The issue that I'm seeing is that there is a difference between saving with the standard Matlab path and the changed Matlab path. In both cases I'm saving to the same network drive and location, but in the first case it's fast, while in the second case it's slow.

As an aside, the files themselves are a few kb, so are usually saved very quickly.

Also, thanks for hinting at changing the current folder. That also seems to have an influence. I might find a workaround using that.

[Sean de Wolski]

Is that additional directory on the network as well? If so, it needs to check that directory for changes and commands before running. If there are a lot of files in there, this could be a non-trivial amount of time.

[Jan]

Yes, it is. Are you saying that the save function triggers a check for changes and commands?

[Sean de Wolski]

Well really anything does. Every time the command window is given control, a rehash is implicitly done to check for file changes.