Memory issues using parfeval

12 ビュー (過去 30 日間)
Andre Zeug
Andre Zeug 2020 年 4 月 22 日
コメント済み: Andre Zeug 2020 年 5 月 1 日
Hi everyone!
Since years I am using the nice function export_fig to save figures, which I create with my scripts and functions. To speed this up I tried to do this in background using 'parfeval'. Unfortunately, for some reason workers pile up memory consumption which can be monitored e.g. with the ProcessExplorer.
function export_figP(fname, varargin)
% same input as export_fig, but first input must be the filename, last
% input must be the figure handle (could be improved)
tParFeval = tic;
tt = []; fig_1 = [];
set(varargin{end},'Visible','off')
myEvalStr = 'tt = parfeval(@export_fig, 0, fname';
for n=1:nargin-2
myEvalStr = sprintf('%s, ''%s''',myEvalStr, varargin{n});
end
myEvalStr = sprintf('%s, fig_1);',myEvalStr);
n = nargin-1;
eval(sprintf('fig_1 = varargin{%d};',n))
eval(myEvalStr)
afterEach(tt, @(~)evalAfterAll(tt,fig_1,fname,tParFeval), 0);
function evalAfterAll(tt,hh,saveStr,tParFeval)
if ~isempty(tt.Error)
fprintf(2,'!!! Error saving Figure %s !!! \n',saveStr)
else
fprintf(2,'Figure ''%s'' saved within %.3fs (total time %.3fs)\n',saveStr,(datenum(tt.FinishDateTime)-datenum(tt.CreateDateTime))*24*60*60, toc(tParFeval))
close(hh)
tt = [];
clear all
end
end
end
The function can be tested with this lines (requires export_fig)
%% Test export_figP
if ~isfolder(fullfile(pwd, 'figs')); mkdir('figs');end
tic
for n = 1:1000
% create simple figure
fig1 = figure('Color','w','Visible','on');
imagesc(rand(1000, 1000))
colorbar
% save figure in Background
saveStr = sprintf('%04d_TestFig.png',n);
export_figP(sprintf('figs/%s',saveStr),'-q100','-nocrop','-r300',fig1);
fprintf('saving of %s started (%.3f)\n', saveStr, toc)
end
What do I do wrong?
How to clear the workers memory after the function call (except shuting down parallel pool)?
Probably my function 'export_figP' does something substantially wrong.
  1 件のコメント
Andre Zeug
Andre Zeug 2020 年 5 月 1 日
Is there an option to clear workers memory when no further job is processed?
p = gcp('nocreate')
p =
ProcessPool with properties:
Connected: true
NumWorkers: 16
Cluster: local
AttachedFiles: {}
AutoAddClientPath: true
IdleTimeout: Inf (no automatic shut down)
SpmdEnabled: true
q = p.FevalQueue
q =
FevalQueue with properties:
Number Queued: 0
Number Running: 0
But memory consumtion is still big after several calls of parfeval:

サインインしてコメントする。

回答 (0 件)

カテゴリ

Find more on Parallel Computing Fundamentals in Help Center and File Exchange

製品


リリース

R2020a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by