How do I allocate cpu resources to a batch job?
8 ビュー (過去 30 日間)
I want to run several batch jobs in the background. Each job runs a different script, but each script calls a 'mpi -np 16 "someApplication"' in parallel with 16 physical cpu cores. These 16 cores are fixed.
Do I need to set up a pool of 16+1 workers for each batch job or do I set up one worker with 16 cpu s, or what would be the best solution in order to run multiple jobs in the background for a server of several multiples of 16 cpus? Can a worker acces multiple cpu s if it is necessary?
Thanks in advance?
回答 (1 件)
Raymond Norris 2022 年 6 月 3 日
It would help if you could provide an example of the script and how you're running the job.
Let's assume you're using PBS, maybe it looks something like
#PBS -l ... (request nodes, ppn, etc.)
module load matlab
mpirun -np 16 matlab -r someApplication
* You wrote mpi -np but I'm assuming you meant mpiexec/mpirun. mpirun should be smart enough to not even need to specify -np 16.
Allocating 16 cores to MATLAB means MATLAB will see 16 cores, but it doesn't by default start a "pool" of workers. Rather, MATLAB will leverage it for the implicit multi-threading (e.g., fft).
If you start a local pool, you'd want to keep it to a max of 16 (which is really 17 including the MATLAB client). In this case, the workers will start MATLAB in singlethreaded mode by default. A worker can access multiple CPUs if you tell the pool to start with more threads. For example
local = parcluster("local");
local.NumThreads = 2;
pool = local.parpool(8);
Again, if you can provide a sample batch script and highlevel MATLAB code, it'll be easier to guide you.