Why does parpool sometimes fail to initialize seemingly randomly and sometimes it works?

Hello everyone,

this is my first question so please be lenient with me.

I am using the following code to read multiple .his image sequences and then save each image of the sequences to a single .tif file.

parpool(8)
im=num_dithering*numICpoints;
parfor ii=1:num_dithering
    temp=zeros(sizesequence);
    for jj=1:numICpoints
        
        temp=temp+double(readHISfile1([projectionFolder,sprintf('proj_dither_%01d_IC_%01d.his',ii-1,jj-1)]));
    end
    temp=temp/numICpoints;
    for i=1:numOfProj
        imwrite(uint16(squeeze(temp(:,:,i))),[tempFolder,sprintf('proj_%04d_dither_%01d.tif',i,ii)])
    end
end
disp("saved raw projections")
delete(gcp)

I noticed that every few times i try to do this initializing the interactive session fails and i get this error message:

Error using parallel.internal.pool.InteractiveClient>iThrowWithCause (line

678)

Failed to initialize the interactive session.

Error using

parallel.internal.pool.InteractiveClient>iThrowIfBadParallelJobStatus

(line 818)

The interactive communicating job finished with no message.

I already checked if i run out of memory, but it seems all fine. When I get this error message I usually just rerun my script and then it works with no change of the code. Does anyone have an idea what the problem could be? I am using the Matlab version 9.9.0.1538559 (R2020b) Update 3. Thanks for your help!

Answer 1

[Sulaymon Eshkabilov]

(1) Try to increase the MATLAB's memory management based on the your system resources.

(2) Also try to configure the max number of available cores for parallel processing.

[Carlo Peiffer]

Hi @Sulaymon Eshkabilov,

As I already mentioned the workstation does not run out of memory and it is using all 8 cores. I actually made it work with 40 cores too. The peculiar thing is just that running the SAME code sometimes throws this error message and sometimes it works. Do you have any idea what the problem could be?

Cheers

[Shuangyuan Lu]

Do you have an answer, I also have the same issue, it is random. I found using longer idletimeout can lower the probability of parpool failing to start, but it can still happen.

parpool('local', 20, 'IdleTimeout', 10);