parallel computing workers number vs. PSO particle number

Hi there,

I'm trying to run PSO in Matlab. I have a processor of 64 cores. I'm wondering how I should assign particle numbers for PSO. Is that true that at each iteration if I assign 64*n (n is an integer) particles, there won't be idle workers waiting each other, which brings efficiency compared with non-64*n particles? My simulation time varies from 25-40s per simulation.

Answer 1

[Walter Roberson]

Is that true that at each iteration if I assign 64*n (n is an integer) particles, there won't be idle workers waiting each other

No, that is not true. When you use parfor, the only way to avoid having cores idle waiting for other cores, is use a pool of size 1.

The question becomes how long they are going to wait. The answer to that is going to depend upon the variability in work loads.

When there are sufficient cores:

• the first 2/3 of the iterations are divided equally between the cores, large chunks
• the next 1/6 (I think it might have been) of the iterations are divided into smaller chunks and handed out to cores as the cores become available
• the remaining (1/6th?) iterations are handed out to cores individually as they become available.

It is possible in this scheme for cores to run out of individual iterations while one of the original large chunks is still executing.

[Wei Wang]

Got it. So if I still use PSO as an example, there is 100 particles per iteration, can a single particle work be divided into smaller chunks for several workers? A single particle is actually an input to my function f(x) so I guess the smallest chunk is a single particle. Is this correct?

Thanks for your help!

[Walter Roberson]

If you are using particleswarm() https://www.mathworks.com/help/gads/particleswarm.html#budidgf_sep_budidgf-fun then

When the 'UseVectorized' option is true, write fun to accept a pop-by-nvars matrix, where pop is the current population size. In this case, fun returns a vector the same length as pop containing the fitness function values.

https://www.mathworks.com/help/gads/particle-swarm-options.html#bued1yt-1

Parallel computation is likely to be faster than serial when your objective function is computationally expensive, or when you have many particles and processors. Otherwise, communication overhead can cause parallel computation to be slower than serial computation.

If your objective function can evaluate all the particles at once, you can usually save time by setting the UseVectorized option to true.

If you can vectorize, that would probably be more efficient, as 100 particles is not much. But it depends upon what you do with each particle.

[Wei Wang]

Based on my case, in which my function can just handle one input at a time and each simulation takes half a minute, I would choose parallel computing. Thanks for your advice!

[Walter Roberson]

I would suggest that you experiment with a parpool of 16 that is allocated 4 cores per worker. Use the Cluster Profile manager to reduce number of workers but increase numthreads.