How to select GPU when doing thread-based parallel?

Hi,

I am running programs using multiple GPUs that are shared with other users in our university's super computer. To improve speed, I run programs parallelly in many workers. I would like to put more workers to GPUs that are freer, and less on GPUs that are busier. For example, I would like 4 workers to use a GPU that is 100% free, and 2 workers to a GPU that has 50% occupied by another user.

I know how to do this when I use process-based pool. I can assign a GPU to a worker by doing:

spmd
    gpuDevice(GPUindex(labindex));
end

But I do not know how to do it using thread-based pool, i.e., a pool generated using

parpool("threads");

Using spmd is not available for a thread-based pool.

I would really appreciate your input on this. Thanks.

[Walter Roberson]

gpuArray does not appear to be supported on thread-based pools. https://www.mathworks.com/help/matlab/referencelist.html?type=function&capability=threadsupport

[Renping Li]

Thanks. I am doing most calculation using gpuArrays in the thread-based pool and it works well in my program though.

[Edric Ellis]

gpuArray should be documented as working on thread-based pools, thanks for pointing out this omission.

Answer 1

[Edric Ellis]

Unfortunately, spmd is indeed not yet supported for thread-based pools. Until it is, you can do something like the following to get a stable worker ID. Note that I am not advocating this as an example of sensible programming... this relies on the fact that thread-based workers are guaranteed to have different random number streams.

% Cause each worker to create a (hopefully) unique random number, store in
% 'c.Value' per worker.
c = parallel.pool.Constant(@rand);
% Retrieve all values back to the client
allVals = fetchOutputs(parfevalOnAll(@(c) c.Value, 1, c));

Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 2).

% This function, when evaluated on a worker, will indicate which position
% that worker got in the list ...
fcn = @(c) find(allVals == c.Value);
% ... like this:
fetchOutputs(parfevalOnAll(fcn, 1, c))

ans = 2×1

1 2

[Renping Li]

Thanks!

Answer 2

[B. S.]

Hi,

Please, how can I use your solution in a thread-based parfor pool to get thread ID ? Basically said, the code looks like the following:

parpool('Threads',8);

parfor i=1:100

A(,:,:,i)=rand(2,2)^i;

Thread_Id(i)= ??? % I want to get thread ID for each i and store them in a vector for later use.

end

Thank you in advance for your kind reply.