Updating variable inside parfor

Hello,

I have a variable that I want to update inside a parfor loop:

a = 1;
parfor i = 1: 10
    a = a + 1;
    disp(a);
end

How can I do this? Can I use data queues to do this, if yes, how.

I am new to parallel computing.

My goal is to print the value of a everytime it is updated.

Edit: I was able to do this as below. But it uses a global variable. Is there a better way?

global a
a = 1;

q = parallel.pool.DataQueue;
afterEach(q, @my_increment);

parfor i = 1: 10
    send(q, i);
end

function my_increment(m)
global a
a = a + 1;
fprintf('a\n', x);
end

Would using persistent variables be better than this?

Answer 1

[Raymond Norris]

The example Use a DataQueue Object and parfor to Update a Wait Bar in

Send and listen for data between client and workers - MATLAB (mathworks.com)

will also show using a waitbar to display completion.

You could use a persistent variable (if you only want to see how many iterations are completed), which is what the above example uses

q = parallel.pool.DataQueue;
afterEach(q, @my_increment);
parfor i = 1:10
    send(q, i);
end

function my_increment(~)
persistent a
if isempty(a)
    a = 1;
else
    a = a + 1;
end
fprintf('a: %d\n', a);
end

But there are two isues with this

• This assumes that the loop variable always starts at 1. I would suggest you have a separate counter that always starts at 1 so that you're not dependent on the loop variable, whch might start at (say) 5.
• Because a in my_increment is persistent, it's also persistent between runs. Therefore, if you run your code a second time, you'll start where you left off last time (i.e., at 10). You'll want to clear your function to reset your persistent variable (and assuming you didn't mlock your function ;) )

For multiple runs, you'll need to call your code as such

my_fcn
clear my_fcn
my_fcn

[atharva aalok]

@Raymond Norris I am using this approach and it works very nicely.

I will accept this answer if you can put it as an independent answer.

[Sean Sullivan]

If you just want to keep track of the total number of iterations that have been started print that number, could you just print the loop variable?

parfor idx=1:10
    disp(idx)
end

[Walter Roberson]

@Sean Sullivan

parfor does not execute loops in order. Indeed, if you do not happen to have the Parallel Computing Toolbox, then the loops will be executed in serial in reverse order. If you do have PCT then parfor allocates chunks of iterations to workers -- so what might be displayed is 61, 81, 1, 41, 82, 83, 21 (for example). Displaying the parfor index does not give any useful information about how many iterations have been done so-far -- not unless you have been mentally counting the outputs as they go by.

Answer 2

[Matt J]

The best way is as you first had it:

a=1;
parfor i = 1: 10
    a = a + 1;
end

There is no way, however, that you can get the updates to display to the screen reliably in serial order (including your approach with a global variable). That is because the updates are not being processed serially. They are being processed in parallel.

[atharva aalok]

I don't want things to happen in serial order.

This works fine but I want to print a, as it gets changed.

When I add disp(a) to the loop after the increment statement, I get the following error.

How can I print a everytime it is incremented?

Error: The variable 'a' is perhaps intended as a reduction variable, but is actually an uninitialized temporary. For more information, see Parallel for Loops in MATLAB, "Temporary Variables Intended as Reduction
Variables".

[Torsten]

How should it be possible to parallelize a recursion which is always serial in nature ?

[atharva aalok]

What? Which recursion?

[Torsten]

a_new = a_old + 1

[atharva aalok]

Is there any other way of solving the problem. I have slightly updated the question.

Thank you

[Torsten]

If the result of index i in a for-loop depends on the result of index (i-1) of the for loop, it makes no sense to parallelize. If the operations in a for-loop are independent from each other, you can parallelize.

[atharva aalok]

For my particular application they are independent. (which is a roughly equivalent problem, but slightly complex)

I just want a way of solving the problem that I have posted.

[Torsten]

If the for-loop steps in your particular application are independent, why do you waste your time with the attempt to parallelize an application where the for-loop steps depend on each other ?

[Matt J]

This works fine but I want to print a, as it gets changed.

You cannot do that because there is no one single version of a that is being changed at a given moment. A different version of a is being incremented on many different processors at once. That's what it means to process in parallel.

If you were using two workers, it would be as if the following two loops are being run on separate computers at the same time,

a1=1;
for i=1:5  %Runs on one worker
    a1=a1+1;
end

a2=0;
for j=6:10  %Runs on another worker
    a2=a2+1;
end

and then at the very end the results are consolidated,

a=a1+a2;

Given the above, what does it mean to you to "print a as it gets changed"? The variable a doesn't really come into existence until the very end.

[atharva aalok]

Oh I see, thanks Matt.

What I want to achieve is maintain a value which is increased each time a loop is run and print that value (or print some character as many times as that value say '*')

I believe that using the function approach with global variable (or persistent variable) is a possible solution.

Do there exist other possible methods where we can bypass global variables?

[atharva aalok]

I want something that performs like the function approach that I have given in my Edit.

That is, some variable that keeps track of the total loops that have been started till now and I can print that value as the parfor is being executed.

Please have a look at the code given in edit.

[Torsten]

@Matt J

a1=1;
for i=1:5  %Runs on one worker
    a1=a1+1;
end
a2=0;
for j=6:10  %Runs on another worker
    a2=a2+1;
end

Are you sure MATLAB will be able to figure out that it can start with a2 = 0 in this special case ?

Imagine the dependency between left and right hand side were a little more complicated, e.g. a = 2*a instead of a = a+1 ? Would MATLAB be able to differ between these two cases ?

[Walter Roberson]

I wonder if https://www.mathworks.com/matlabcentral/fileexchange/71083-waitbar-for-parfor would be suitable for your purpose ?

[Matt J]

@atharva aalok I believe that using the function approach with global variable (or persistent variable) is a possible solution.

No, it is not. Why use parfor at all seeing as you seem to want things to happen serially? Why not just use a regular for loop?

[Matt J]

@Torsten Imagine the dependency between left and right hand side were a little more complicated, e.g. a = 2*a instead of a = a+1 ? Would MATLAB be able to differ between these two cases ?

Yes. See this link for a table of reduction operations that parfor knows how to parse,

https://www.mathworks.com/help/parallel-computing/reduction-variable.html

[Torsten]

@Matt J

Thank you.

[Walter Roberson]

The thing about reduction variables is that they are effectively write-only except for the implied read to determine the new value. If you were to do

a=a+1;

as a reduction, then the access to a elsewhere in the parfor loop would be effectively blocked -- and if not, then possibly wildly wrong. parfor converts the system into an implied

a_private(INDEX) = 1;
....
    end
a = sum(a_private);

[atharva aalok]

@Matt J

No I do not want things to happen serially.

I am running calculations in parallel and as @Walter Roberson pointed and my code example in the Edit, I just want to keep track of the total number of iterations that have been started and I want to print the number to see that visually.

[Walter Roberson]

Try the parfor waitbar.