Correct fetchNext index after clearing FevalFuture object
5 ビュー (過去 30 日間)
古いコメントを表示
I am using parfeval to run code asynchronously on workers. The running time varies a lot between workers depending on some parameters that in the following code I represent with 'm'.
When I collect the results with fetchNext, I then process them with idx, in particular retrieving m(idx). At the same time, I need to clear the last retrieved FevalFuture object to save memory.
In Parfeval - Memory consumption piling up - Clear output data? it is suggest to simply assign [ ] to the FevalFuture object. However, it seems to me that doing this mix up the indices idx of the objects, which I need to further process the data. See for instance this mve
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
total = 0;
for idx = 1:N
[idx, data] = fetchNext(f);
idx
f(idx) = []; % This line here
% process data with idx, m(idx) ..
end
If you comment the line indicated the code will work fine, but if I clear f(idx) the just-used idx could be reused and I would process the data with the wrong m(idx).
How is it possible to get the correct index idx with which f(idx) was called in the fist loop?
Thanks in advantage
0 件のコメント
回答 (1 件)
Edric Ellis
2019 年 10 月 15 日
One way you can deal with this is by making the same modification to m as you proceed. Something like this perhaps:
N = 10;
m = 1:10;
for idx = N:-1:1
f(idx) = parfeval(@(m,N) m*rand(N), 1, m(idx), N);
end
for idx = 1:N
[fidx, data] = fetchNext(f);
% Get the value out of "m" corresponding to the completed future
mval = m(fidx);
% Delete corresponding elements of both "f" and "m"
f(fidx) = [];
m(fidx) = [];
% Work with the particular value of "m" and the result
disp([mval, sum(data(:))]);
end
0 件のコメント
参考
カテゴリ
Help Center および File Exchange で Asynchronous Parallel Programming についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!