parallel computing fitness function inside parallel optimization

I have a calculation intensive fitness function to optimize in genetic optimization.

I plan to use matlab parallel server with 10 compute nodes, each on has 24core. Can I run each objFunction parallel in GA and parallel in each nodes?

Following is example code and target

• 40 poulation,
• UseParallel On in GA options
• Can assign each objFuntion evaluation to indivadual 24core compute and use another parfor inside objFunction?
• Therefore, it allowed me run 10 parallel objFunction on 10 node. Each objfunction run parallel in each node.

Thanks

% parallel on and 40 population
options = optimoptions('UseParallel',true,'PopulationSize',40);

[x,fval,exitflag,output] = gamultiobj(@objFunction,20,[],[],[],[],lb,ub,options);

%% fitness function required to run parallel
function output = objFunction(x)

a = 1:10000;
output = zeros(1,numel(a));

parfor i = 1:numel(a)
    output(i) = a(i) + mean(x);
end


end

Answer 1

[Walter Roberson]

Inside a parallel worker you can vectorize, but not run anything in parallel.

You would need to run multiple matlab sessions, each taking on part of the task.

[Rui]

Thanks Walter

[Walter Roberson]

See also MultiStart global optimizer -- you can run a number of different gamultiobj in parallel (but each one would not be able to use parfor in such a case.)

If you do use explicit parfor, remember that by default each worker only has access to a single core, and that slows down some vectorized calculations that would otherwise be able to automatically split their work between multiple cores. You can, however, deliberately create a profile with fewer workers each of which is permitted access to multiple cores.

[Sam Marshalik]

Rui, can you expand a bit more on what you are looking to do and how you ultimately want to run your code on MATLAB Parallel Server?

There is some level of nested parallelization with MATLAB Parallel Server. By default, the NumThreads value in the cluster profile is set to 1. Meaning that your parallel workers have access only to a single thread. You can change NumThreads to a value > 1 and do the following:

1. Run a multi-threaded function on my parallel worker. If you have code that is heavily multi-threaded, you can offload a single worker per machine and give each worker access to NumThreads = number of cores on that machine.
2. You can also use "parpool("Threads")" to start a thread pool on a parallel worker and run parallel worklows there. The catch is that not everything can run in a threaded environment (we are expanding this with every release). For example:

>> c = parcluster('cluster');
>> c.NumThreads = 4; % I am using 4 as an example, you can set that to a number that is right for you
>> c.parpool(1)
>> spmd, parpool("Threads", 4), end
Worker 1: 
  Starting parallel pool (parpool) using the 'Threads' profile ...
  Connected to parallel pool with 4 workers.
  
  ans = 
  
   ThreadPool with properties: 
  
             NumWorkers: 4
                   Busy: false

[Rui]

Hi Sam,

Thanks for suggestion for option of NumThreads. Can I run parallel in each worker during optmization? My finitness function is computing intensive. for example,

I understand matlab optimization can do parallel, by default, each worker use 1 core and non-parallel inside worker.

options = optimoptions('UseParallel',true);
[x,fval,exitflag,output] = gamultiobj(@objFunction,20,[],[],[],[],lb,ub,options);

What I am looking for is parallel inside each worker (each worker is one computer node) with MATLAB Parallel Server. For example (each computer node has 4 core).

Worker 1 (compute node 1): 4 core parallel parfor

Worker 2 (compute node 2): 4 core parallel parfor

Worker 3 (compute node 3): 4 core parallel parfor

....

[Sam Marshalik]

Hey Rui,

It is a bit tricky, but you can do this.

I first wanted to make sure that gamultiobj could run in a Threaded environment. I confirmed that it is possible by startung a local Thread Pool and running an example there. This means that a MATLAB Parallel Server process worker will be able to start a Threaded pool of workers and run the optimization problem in parallel using Threads.

The next question is how to actually offload this to the cluster. Can you let me know what scheduler you are using? Depending on what scheduler you are using the answer will be different.

I think using batch jobs will be a good way forward, like the below. Each batch job will start a single process worker and then start X number of Threads. The detail to be worked out is how to request the right resources from ths scheduler. You will want to ask for a total of X + 1 cores (1 for the MATLAB Parallel Server worker and X for the Threads).

c = parcluster('myCluster');
c.NumThreads = X;
batchJob = batch(p,@optimFunc, c.NumThreads)


function optimFunc(NumThreads)

parpool("Threads", NumThreads)
rng default % For reproducibility
M = diag([-1 -1]) + randn(2)/4; % Two problem variables
fun = @(x)[(x').^2 / 30 + M*x']; % Two objectives
intcon = 2;
lb = [0 0];
ub = [100 50];
options = optimoptions("gamultiobj","PlotFcn","gaplotpareto",...
    "PopulationSize",100, 'UseParallel', true);
nvars = 2;
A = [];
b = [];
Aeq = [];
beq = [];
nonlcon = [];
[x,fval] = gamultiobj(fun,nvars,A,b,Aeq,beq,lb,ub,nonlcon,intcon,options);

end