Matlab loop (with load/save) is getting slower and slower
14 ビュー (過去 30 日間)
古いコメントを表示
Hi
I'm running a large code that is basically:
for i = 1:50
load(i.mat, input);
output = myfun(input);
save(i-out.mat, output);
end
I've run "myfun" hundreds of times and it's long-winded but it works fine and is entirely deterministic. The problem which I'm getting is new, basically since I've added the load/save part, which replaces me just saving the output as one element in a pre-allocated vector. The time taken to execute the loop is getting longer and longer as i increases. The first 4 are completed within hours, the but by the time I reach 20 it's taking a full 24 hours before 3 iterations are completed.
There is no accumulation, each iteration should be totally independent, there is no correlation between the size of i and the input file i.mat. I would test using "for i = 25:1" to make sure order doesn't matter, but I don't have another week.
Does anyone know a simple explanation of why saving/loading (these are large files) might slow this down?
Thanks for any help Mike
5 件のコメント
Image Analyst
2013 年 1 月 24 日
編集済み: Image Analyst
2013 年 1 月 24 日
Let's see the actual line where you call save. The paraphrased/pseudocode line you gave is not helpful enough.
回答 (3 件)
Image Analyst
2013 年 1 月 24 日
Might not make any difference but I'd delete any existing mat file before saving to it to make sure it's not adding/appending to it. See if that helps. Use tic and toc to find out which is taking longer and longer - the load() or the save().
2 件のコメント
Image Analyst
2013 年 1 月 24 日
Are you running with any breakpoints set? That can really slow things down.
Shashank Prasanna
2013 年 1 月 24 日
編集済み: Shashank Prasanna
2013 年 1 月 24 日
Michael, try the following: put a memory command in the the loop:
for i = 1:50
load(i.mat, input);
output = myfun(input);
save(i-out.mat, output);
memory
end
Either paste the output or let us know if the memory is increasing at each iteration, this may be a leak. Or I can think of JIT issues.
0 件のコメント
vincent caillet
2017 年 8 月 21 日
編集済み: vincent caillet
2017 年 8 月 21 日
Hi,
I may have a solution for your problem. In your loop, try to use close all force.
My work is essentially to read a large number of images, do some filtering, and save them back. When I loop through a large number of images, the computer memory linearly increases. I've tried to clear as much memory as possible as the end of the loop, but was not very successful.
I notice that clearing the variables AND closing worked however. Not sure if that applies to other people, but for me it's works perfectly. See the example below:
for i = 1:50
load(i.mat, input);
output = myfun(input);
save(i-out.mat, output);
clearvars -expect i OtherVariableOfInterest
close all force
end
While the code is running, inspect your computer performances (Alt+Esc) to see whether the memory keeps increases, or also drops at the end of your loop.
0 件のコメント
参考
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!