Can a worker/slave be used as a master node?

I am using the parallel computing toolbox. I have a general question though. I am trying to implement a Branch and Bounds algorithm for non-convex optimization. There has to be a master node and a bunch of worker nodes/slaves. Each time the slave solves a problem, it has to report back to the master node and the master node has to analyze and send a new problem back to any of the slaves which are unassigned.

Let's say we have 100 workers available. Can we use one of the workers out of the 100 as the master node? Or is the term 'Master Node' reserved for the Client (Desktop MATLAB)? Or have I got the whole concept wrong?

I really need guidance on this. Thank you.

Answer 1

[Walter Roberson]

If you were to use spmd then you could pick any of the nodes to labSend() to . spmd() allows arbitrary communication between nodes.

The other facilities in the Parallel Computing toolbox do not allow workers to send data to each other directly.

I notice from your other question that you found pollable data queue. That is best documented as allowing data to be sent from worker to client. It is also possible to construct one backwards to send data from the client to the worker. It is not possible to use one to send worker to worker, though (Mathworks employees have said): in order to send worker to worker you would have to have the worker send to the client and the client forward on to the other worker.

You could also call out to mex routines to interface to MPI, but spmd already uses MPI...

[Walter Roberson]

You can use any node as "master" if you use spmd.

Otherwise, you have to either use the client as master or as a switchboard to pass messages between the other nodes and the master node.

[Viswanath Hariharan]

Thank you so much for clarifying this for me.

Answer 2

[LINUS UNGEM BACHE]

i am trying to write down a parallel code of ant colony algorithm using the master/slave strategy. please i dont know how the master/slave works. Can you help me ?

[Walter Roberson]

spmd
    if labindex == 1
         for K = 1: 5
             labsend(K+1, sprintf('March %d by %d', K, K))
         end
    else
        cmd = labReceive();
        March_by = sscanf(cmd, 'March %d');
        do_some_work(March_by);
    end
end

[LINUS UNGEM BACHE]

thanks very much for the given code.

[LINUS UNGEM BACHE]

I have 8 workers and i want a master to send four reals constants and two array to each workers. please help me how can i do it ?

[Walter Roberson]

real_constant_1 = 5;
real_constant_2 = 17;
real_constant_3 = sqrt(-1);
real_constant_4 = 42;

spmd
    if labindex == 1
         for K = 1: 7
             first_array = rand(3,5);
             second_array = randn(11,4);
             this_data = {real_constant_1, real_constant_2, real_constant_3, real_constant_4, first_array, second_array};
             labsend(K+1, this_data)
         end
    else
        this_data = labReceive();
        [rc1, rc2, rc3, rc4, arr1, arr2] = deal(this_data{:});
        do_some_work(rc1, rc2, rc3, rc4, arr1, arr2);
    end
end

[LINUS UNGEM BACHE]

i test it and it work. thanks

[LINUS UNGEM BACHE]

but what if i want all the worker to send back the data to the master?

[Walter Roberson]

real_constant_1 = 5;
real_constant_2 = 17;
real_constant_3 = sqrt(-1);
real_constant_4 = 42;
spmd
    if labindex == 1
         for K = 1: 7
             first_array = rand(3,5);
             second_array = randn(11,4);
             this_data = {real_constant_1, real_constant_2, real_constant_3, real_constant_4, first_array, second_array};
             labsend(K+1, this_data);
         end
         for K = 1 : 7
             [response, source] = labReceive();
             all_responses(source-1) = response;
         end
    else
        this_data = labReceive();
        [rc1, rc2, rc3, rc4, arr1, arr2] = deal(this_data{:});
        result = do_some_work(rc1, rc2, rc3, rc4, arr1, arr2);
        labSend(1, result);
    end
end

[LINUS UNGEM BACHE]

thanks, please i need clarification on the second loop of for. what does the loop really do? and please can one worker receive data from all the others simultaneously?

[Walter Roberson]

It is not possible for a worker to receive data from more than one place at a time. Each worker has a queue. You can ask to receive the next from a particular worker, and you can ask to receive the next item marked with a particular tag, but you can only receive one item per labReceive call.

Because of that, in order to receive all of the replies you need to loop asking for one reply each time.

[LINUS UNGEM BACHE]

ok that's right. thanks

[LINUS UNGEM BACHE]

please, i have written a master/slave code of ant algorithm. but when i run it, the time is more larger than the serial one. what can be the cause of it? please i need a help

[Walter Roberson]

Communication between workers with synchronization takes time and resources.

[LINUS UNGEM BACHE]

So what can i do to have a good speed up?

[Walter Roberson]

You are being required to implement in this manner. Sometimes the reason for a requirement such as that is to cause you to think about implementation difficulties and how to overcome them.

Parallel processing works best when all of the below are true:

1. the amount of data being transferred back and forth is small compared to the computation (overhead small)
2. the computation on each worker is long enough (amortized costs)
3. the computation does not involve large arrays in ways that can be well vectorized (high performance multithreaded libraries are very efficient in serial instead of parallel)

[LINUS UNGEM BACHE]

Ok thanks

Answer 3

[gild kone]

I have a matlab code for solving eigenvalues and eigenvectors problems but the ways is that i can normally plot differents eigenvalues of my matrix which depend on the varible x=-1:1. My problem is that I don't know how to plot in 3D the corresponding eigenvectors ( ie xlabel: x=-1:1, ylabel: y=-1:1 and Zlabel: z=eigenvectors )

This the correpondint code :

close all;clear all; U =0.9; f= 39; nx=f;T=1; 
x=-3.086476993000499:3.096476993000599; y=x;
A=zeros(nx,nx);   B=zeros(nx,nx); mat=zeros(nx,nx);  

for kappa=1:numel(x)  % nu=1:f-1            

 Rep = T.*(2.0.*cos(x(kappa)/2.0).*cos(x(kappa).*f/2.0)); Imp = 0.0; 
 Req = T.*(1.0 + cos(x(kappa)));  Imq = T.*(sin(x(kappa)));
 A(1,1)=U ; A(1,2)=sqrt(2.0)*Req;  A(2,1)=sqrt(2.0)*Req; 
B(1,2)=sqrt(2.0)*Imq;   B(2,1)=-sqrt(2.0)*Imq;

 for i=2:nx-1
A(i,i+1)=Req; A(i+1,i)=Req ;B(i,i+1)=Imq; B(i+1,i)=-Imq;
 end
A(nx,nx)=Rep ; B(nx,nx)=Imp;
 for i=nx:-1:1
for j=nx:-1:1
    
mat(i,j)=A(i,j);
mat(i,j+nx)=B(i,j);
mat(i+nx,j)=-B(i,j);
mat(i+nx,j+nx)=A(i,j);

end
 end

[v,d0] = eig(mat);   % eigenvectors and eigenvalues
 %vv=v(:,kappa) ; v0=vv/norm(vv); C0=v0.*v0        % give us \c0\^2
        dd=d(:,kappa); d0=dd/norm(dd); z=d0*d0' ; surf(z); 
   %d0(kappa,:)=eig(mat); for eigenvalue only 
end
 plot(d0,'r-');

[Walter Roberson]

This is not relevant to the Question that was asked before, so it should be in its own Question.