Memory usage of parpool balloons in dual CPU system, then decreases after 20 minutes

Hi,

I just installed a second CPU on my motherboard in the hopes of leveraging its extra processing power using parallel computing in Matlab. However I noticed that memory usage is much higher when creating a parallel pool.

When creating a pool of any size, memory usage is approximately double in the dual CPU setting compared to the single CPU setting. Just to be clear, I'm comparing single and dual CPU memory usage with the same number of workers (4 workers in both cases, 8 workers in both cases etc).

I was wondering if this is something that is inherent to parallel computing in Matlab on dual CPU systems or whether there is a settings tweak that might help with this.

[edit] I just noticed that the memory usage declines to 'normal' levels after running for about 47 minutes. During this time Matlab is running the same script, running mainly permutation tests on one dataset; no new data is being loaded or saved. The decrease in memory load happens gradually, over the course of about one minute, which makes me wonder what's going on exactly. If certain data is simply being disregarded, I would expect it to happen more quickly.

Thanks, best, Paul

Answer 1

[Paul]

Alright so I found the answer myself. In the parpool documentation no less! The problem was the startup.m file I used. Removing it from the search path lead to much lower memory usage and quicker pool creation.