Random Number Streams on a GPU

By default, the random number generation functions rand, randi, and randn use different generator settings for calculations on a GPU compared to those on a CPU. You can change the behavior of random number generators to generate reproducible sequences of random numbers on the GPU and CPU.

The table below summarizes the default settings for the GPU and CPU on client and worker MATLAB^® sessions:

	Generator	Seed	Normal Transform
Client CPU	Mersenne Twister	0	Ziggurat
Worker CPU	Threefry 4x64 generator with 20 rounds	0	Inversion
GPU (on client or worker)	Threefry 4x64 generator with 20 rounds	0	Box–Muller

In most cases, it does not matter that the default random number generator on the GPU is not the same as the default generators on the client or worker CPU. However, if you need to reproduce the same results on both the GPU and CPU, you can set the generators accordingly.

Client CPU and GPU

In a fresh MATLAB session, MATLAB generates different sequences of random numbers on the CPU and GPU.

Rc = rand(1,4)

Rc =
    0.8147    0.9058    0.1270    0.9134

Rg = rand(1,4,"gpuArray")

Rg =
    0.3640    0.5421    0.6543    0.7436

If you need to generate the same sequence of random numbers on both the GPU and CPU, then you can set the generator settings to match.

There are three random number generator algorithms available on the GPU: "Threefry", "Philox", and "CombRecursive". All three are also supported on the CPU. The following table lists the algorithms for these generators and their properties.

Value	Generator Name	Generator Keyword	Multiple Stream and Substream Support	Approximate Period in Full Precision
"Threefry"	Threefry 4x64 generator with 20 rounds	threefry4x64_20	Yes	2514 (2256 streams of length 2258)
"Philox"	Philox 4x32 generator with 10 rounds	philox4x32_10	Yes	2193 (264 streams of length 2129)
"CombRecursive"	Combined multiple recursive generator	mrg32k3a	Yes	2191 (263 streams of length 2127)

You can use rng and gpurng to set the generator algorithm and seed on the CPU and GPU, respectively. For more information about the GPU random number generators and their performance, see Generate Random Numbers on a GPU.

sc = rng(1,"Threefry");
Rc = rand(1,4)

Rc =
   0.1404    0.8197    0.1073    0.4131

sg = gpurng(1,"Threefry");
Rg = rand(1,4,"gpuArray")

Rg =
    0.1404    0.8197    0.1073    0.4131

The rand and randi functions now generate the same sequences of random numbers on the client CPU and GPU.

You can also use rng and gpurng to reset the generator algorithm and seed to the default values on the CPU and GPU respectively.

rsc = rng("default");
rsg = gpurng("default");

Worker CPU and GPU

A parallel worker CPU uses the same default random number generator type and seed as the client GPU and the worker GPU, if it has one. The GPU and CPU do not share the same stream. By default, rand and randi generate the same sequence of numbers on a GPU and a worker CPU.

The settings are different from those on the client CPU. For more information, see Control Random Number Streams on Workers

If you need to generate different random numbers on each worker, you can change the generator settings. In this example, each worker creates the same sequence on its GPU and CPU, but different sequences are generated on each worker.

p = parpool(2);
spmd
    rng(spmdIndex,"Threefry");
    Rc = rand(1,4)

    gpurng(spmdIndex,"Threefry");
    Rg = rand(1,4,"gpuArray")
end
delete(p)

Normally Distributed Random Numbers

For normally distributed random numbers created using the randn function, MATLAB produces different results on a client CPU, a worker CPU and a GPU. The transformation of uniform random numbers into normally distributed random numbers is controlled by the NormalTransform argument. You can control this on the GPU using the parallel.gpu.RandStream function.

On a client CPU, the default NormalTransform is "Ziggurat". On a worker CPU, the default is "Inversion".

Unless otherwise specified, GPU code uses the "BoxMuller" transform for the "Threefry" and "Philox" generators and the "Inversion" transform for the "CombRecursive" generator.

You can set the same generators and transforms on the CPU and the GPU to get the same normally distributed random number sequences. The only transform supported on both the CPU and GPU is "Inversion".

sc = RandStream("Threefry",NormalTransform="Inversion",Seed=1);
RandStream.setGlobalStream(sc)

sg = parallel.gpu.RandStream("Threefry",NormalTransform="Inversion",Seed=1);
parallel.gpu.RandStream.setGlobalStream(sg);

Rc = randn(1,4)

Rc =
   -1.0783    0.9144   -1.2412   -0.2196

Rg = randn(1,4,"gpuArray")

Rg =
   -1.0783    0.9144   -1.2412   -0.2196