How do I allocate cpu resources to a batch job?
16 ビュー (過去 30 日間)
I want to run several batch jobs in the background. Each job runs a different script, but each script calls a 'mpi -np 16 "someApplication"' in parallel with 16 physical cpu cores. These 16 cores are fixed.
Do I need to set up a pool of 16+1 workers for each batch job or do I set up one worker with 16 cpu s, or what would be the best solution in order to run multiple jobs in the background for a server of several multiples of 16 cpus? Can a worker acces multiple cpu s if it is necessary?
Thanks in advance?
回答 (1 件)
Raymond Norris 2022 年 6 月 3 日
It would help if you could provide an example of the script and how you're running the job.
Let's assume you're using PBS, maybe it looks something like
#PBS -l ... (request nodes, ppn, etc.)
module load matlab
mpirun -np 16 matlab -r someApplication
* You wrote mpi -np but I'm assuming you meant mpiexec/mpirun. mpirun should be smart enough to not even need to specify -np 16.
Allocating 16 cores to MATLAB means MATLAB will see 16 cores, but it doesn't by default start a "pool" of workers. Rather, MATLAB will leverage it for the implicit multi-threading (e.g., fft).
If you start a local pool, you'd want to keep it to a max of 16 (which is really 17 including the MATLAB client). In this case, the workers will start MATLAB in singlethreaded mode by default. A worker can access multiple CPUs if you tell the pool to start with more threads. For example
local = parcluster("local");
local.NumThreads = 2;
pool = local.parpool(8);
Again, if you can provide a sample batch script and highlevel MATLAB code, it'll be easier to guide you.
Find more on Job and Task Creation in Help Center and File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!Start Hunting!