Randomness is lost in parfor loop on GPU?

1 ビュー (過去 30 日間)
Daigo
Daigo 2022 年 4 月 12 日
編集済み: Daigo 2022 年 4 月 15 日
I have a code with a following structure:
% my_code_seed#.m
myoutputs1 = my_function1(myinputs1);
seed = #;
rng(seed);
parfor iter = 1:50
myouputs2 = my_gpu_function2(myoutputs1, my_function_of_Gaussian);
result(iter).name = myoutputs2;
end
save(file_name, 'result');
What I want to do is to model some random behavior of something by using this code repeatedly. What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2". I run these codes in a parallel way on a GPU cluster to save the computation time. However, it seems like the results from the codes are very similar, even though I'm giving different values of seed.
Do you have an idea why rng(seed); is not working properly in this case? I appreciate any help.

回答 (1 件)

Steven Lord
Steven Lord 2022 年 4 月 13 日
What I'm doing is to copy this code and change the value of the seed. For example, the "my_code_seed1.m" uses "seed = 1", and "my_code_seed2.m" uses "seed = 2".
So if later on you find a bug in my_code_seed42.m you're going to go back and modify the previous 41 files? That's inefficient. Instead write the code once as a function that accepts a seed value and call that function with the various seeds as input.
As for the randomness in a parfor, see this documentation page on repeating random numbers in a parfor loop and the page on controlling randomness to which its first paragraph links.
  4 件のコメント
Daigo
Daigo 2022 年 4 月 13 日
編集済み: Daigo 2022 年 4 月 15 日
@Edric Ellis I ended up using parallel.gpu.RandStream.create to control the rng state for gpuArray inside of forloop. Here is a toy example:
function [r, time] = example_random_gpu(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(data(ii,:))/n;
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc;
end
function c = func_random(a, n, stream)
c = randn(stream, 1, n) + 1i*a*randn(stream)*gpuArray(linspace(-2,2,n));
end
The function example_random_gpu performs as exactly desired - the same r for the same val_seed, but a different r for a different val_seed.
However, I have another question regarding the data access. In this toy example, I have multiple parfor-loops accessing the same constant: data. Is there anyway to allow each worker to access the value of data more efficiently? I tried doing the following but the computation became much slower:
function [r, time] = example_random_gpu2(val_seed)
tic
% Example data to be used in parfor loop
n = 100;
data = reshape(1:n^2, [n, n]);
C = parallel.pool.Constant(data); % <----- ADDED
% Create random streams for gpuArray
streams = parallel.gpu.RandStream.create('Threefry',...
'NumStreams', n,...
'Seed', val_seed,...
'CellOutput', true);
r = zeros(3,n,'gpuArray');
% Loop
parfor ii = 1:n
a = sum(C.Value(ii,:))/n; % <---- Modified
r(ii,:) = func_random(a, n, streams{ii});
end
% Get the elapsed time
time = toc; % ---> Computation became slower ...
end

サインインしてコメントする。

タグ

製品


リリース

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by