run background function on subset of allocated parpool nodes on cluster?

2 ビュー (過去 30 日間)
Science Machine
Science Machine 2022 年 7 月 23 日
コメント済み: Edric Ellis 2022 年 7 月 25 日
Is it possible to start a parpool, then run a background function on a subset of the parpool's allocated nodes (on a cluster)?
Suppose I allocate (total Procs) = (48 procs) x ( 6 nodes) using
parpool(48*6);
Then can I then invoke some command such as
myNum=[1e6,2e6]
Result1 = parfeval(backgroundPool_1,@rand,1,myNum(1)); % suppose bgPool_1 is a subset of parpool somehow
Result2 = parfeval(backgroundPool_2,@rand,1,myNum(2)); % suppose bgPool_1 is a subset of parpool somehow
where backgroundPool_1 would be like 48*6/2, half of the number of allocated nodes, and same for bgP_2.
  • I would not want to allocate two different parpools, because I am running interactively on a cluster and want all nodes to be pre-allocated on the slurm queue
  1 件のコメント
Edric Ellis
Edric Ellis 2022 年 7 月 25 日
This isn't possible today. Can you explain more about what you want to do on the pool subset if you could create one? What do you want to run there, and how would it use multiple workers of the subset? One thing I can imagine is doing something like this:
p = parpool(..);
[subPool1, subPool2] = somehowDividePool(p);
fut = parfeval(subPool1, @myFunctionThatCallsParfor, numOuts, in1, in2);
Here, myFunctionThatCallsParfor has a parfor loop inside, and it gets to use the workers from subPool1 to run that.
Is that the sort of thing you're interested in? If not, it would be useful if you could explain what you would like to be able to do.

サインインしてコメントする。

回答 (0 件)

カテゴリ

Help Center および File ExchangeParallel Computing Fundamentals についてさらに検索

製品


リリース

R2019a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by