Running parfor on multiple nodes using Slurm

50 ビュー (過去 30 日間)
Siamak Abolhassani
Siamak Abolhassani 2022 年 6 月 28 日
コメント済み: Siamak Abolhassani 2022 年 6 月 28 日
Hello everyone,
What I really like to do is to take advantage of running a Matlab script (written to be run in parallel on multiple cores using "parfor" concept) remotely on a supercomputer. The issue is not to run the script on just one node (ex. the node includes 48 cores) but is to run it on multiple nodes (more than 48 cores).
Attached you can find a simple 10-line Matlab script (parEigen.m) written by the "parfor" concept. I have attached the corresponding shell script I used, and the Slurm output from the supercomputer as well. From the Slurm output, you clearly see that I could succeed to run the script on 48 cores (1 node) on the supercomputer. However, I am looking for a solution to run the Matlab script remotely on more cores (multiple nodes) on the supercomputer.
I really appreciate any help you could provide with this. Please consider that I am a normal user of the supercomputer. I do not have access to the Matlab GUI and the parallel computing toolbox setting.
My Matlab script:
function [elapsedTime] = test_for()
nworker = str2double(getenv('SLURM_NTASKS')) - 1
defaultProfile = parallel.defaultClusterProfile
myCluster = parcluster(defaultProfile);
parpool(myCluster, nworker)
N = 1000;
A = zeros(N,1);
parfor i = 1 : N
E = eig(rand(100))+i;
A(i) = E(1);
elapsedTime = toc;
My shell script:
#SBATCH --job-name="pforTest"
#SBATCH --time=00:15:00
#SBATCH --ntasks=48
#SBATCH --cpus-per-task=1
#SBATCH --partition=compute
#SBATCH --mem-per-cpu=1GB
#SBATCH --account=researcher
module load matlab
matlab -r parEigen
Slurm output:
MATLAB is selecting SOFTWARE OPENGL rendering.
< M A T L A B (R) >
Copyright 1984-2021 The MathWorks, Inc.
R2021b ( 64-bit (glnxa64)
September 17, 2021
To get started, type doc.
For product information, visit
defaultProfile =
Starting parallel pool (parpool) using the 'local' profile ...
Connected to the parallel pool (number of workers: 47).
ans =
ProcessPool with properties:
Connected: true
NumWorkers: 47
Cluster: local
AttachedFiles: {}
AutoAddClientPath: true
IdleTimeout: 30 minutes (30 minutes remaining)
SpmdEnabled: true
ans =


Raymond Norris
Raymond Norris 2022 年 6 月 28 日
The local scheduler will only spawn workers on the same machine running the MATLAB client (e.g., on a Slurm compute node). In order to run a parallel job that spawns across mulitple nodes, you'll need the MATLAB Parallel Server. In doing so, you'll have the option to submit the job from MATLAB running on your desktop machine or from MATLAB running on your Slurm cluster (as you're doing now with the local scheduler).
  3 件のコメント
Siamak Abolhassani
Siamak Abolhassani 2022 年 6 月 28 日
Thank you for your answer, Raymond! That was great!


その他の回答 (0 件)


Find more on Parallel for-Loops (parfor) in Help Center and File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by