Slice called broadcast variable in parfor

I have a code I want to parallelize, something like:
parfor i = 1:length(all_idx(:,1))
combis = nchoosek(1:k,2);
results = zeros(length(combis(:,1)),1);
for j = 1:length(combis(:,1))
results(j) = fun(X(:,all_idx(i,combis(j,1))), X(:,all_idx(i,combis(j,2))));
end
...
end
X and all_idx are broadcast variables, especially X is a huge matrix and produces a significant overhead. I read something about slicing the variables, the computation results are strored in (variables in which I write), but is there a way to somehow slice the variables which are read within the loop? As far as I understand, the whole matrices X and all_idx have to be sent to every worker in order to extract the columns needed, or did I miss something?

2 件のコメント

Edric Ellis
Edric Ellis 2019 年 12 月 18 日
It looks from your code very much like each iteration of your outer parfor loop needs the whole value of X available. In this case, it is unavoidable that X must be sent separately to each worker.
"Slicing" variables is good - when it works - i.e. when your parallelism can be expressed as operations over independent "slices" of your data
Mariella Dreißig
Mariella Dreißig 2019 年 12 月 18 日
Okay thanks, then I got it right and I can't do anything against this broadcast variable.

サインインしてコメントする。

回答 (0 件)

カテゴリ

ヘルプ センター および File ExchangeLoops and Conditional Statements についてさらに検索

製品

リリース

R2016b

質問済み:

2019 年 12 月 17 日

コメント済み:

2019 年 12 月 18 日

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by