Using Groups of Rows in a Parfor Loop
2 ビュー (過去 30 日間)
古いコメントを表示
Below is my code that is attempting to populate a variable using section of rows. The original ContractFile is hundreds of thousands of rows - the thinking is I can populate the variable on different workers using the parfor loop which will populate sections of 10,000 rows at a time on a different worker. Example: Rows 1:10,000 to one worker, rows 10,001:20,000 to a different worker, etc. This code works as a regular for loop, but breaks as a parfor loop and I can't figure out why. Thanks!
parfor i = 1:Contracts
Rows = (i-1)*10000+(1:10000);
Var1(Rows,:) = ContractFile(Rows,2) .* ContractFile(Rows,8);
end
1 件のコメント
Matt J
2020 年 4 月 1 日
Why is the loop necessary? Why not simply,
Var1=ContractFile(:,2) .* ContractFile(:,8);
採用された回答
Matt J
2020 年 4 月 1 日
編集済み: Matt J
2020 年 4 月 1 日
As mentioned in my comment, your example does not make it clear why a loop is necessary at all. However, the reason for your difficulty is that your parfor code violates these rules. One way to fix it is as follows:
Var1=nan(10000,Contracts);
A=reshape(ContractFile(1:Contracts*10000,2),10000,[]);
B=reshape(ContractFile(1:Contracts*10000,8),10000,[]);
parfor i = 1:Contracts
Var1(:,i) = A(:,i).*B(:,i);
end
Var1=Var1(:);
5 件のコメント
Matt J
2020 年 4 月 2 日
編集済み: Matt J
2020 年 4 月 2 日
1. Var1=Var1(:) is equivalent to Var1=reshape(Var1,[],1). See also
2. You could pre-pad the array (e.g., with NaNs) to have an even multiple of 10000 elements and then proceed as above. Or, instead of reshaping, you could also partition your input data into cells and use cell array indexing. To do this splitting, I recommend mat2tiles (Download) because it will handle non-even multiples of 10000 rows, as demonstrated in the code below.
CF=mat2tiles(ContractFile,[10000,Inf]);
N=numel(C);
Var1=cell(numel(C),1);
parfor i=1:N
Var1{i}=CF{i}(:,2).*CF{i}(:,8);
end
Var1=cell2mat(Var1);
その他の回答 (0 件)
参考
カテゴリ
Help Center および File Exchange で Parallel for-Loops (parfor) についてさらに検索
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!