parfor

Execute for-loop iterations in parallel on workers

Syntax

parfor loopVar = initVal:endVal; statements; end

parfor (loopVar = initVal:endVal,M); statements; end

parfor (loopVar = initVal:endVal,opts); statements; end

parfor (loopVar = initVal:endVal,cluster); statements; end

Description

parfor loopVar = initVal:endVal; statements; end executes for-loop iterations in parallel on workers in a parallel pool.

MATLAB^® executes the loop body commands in statements for values of loopVar between initVal and endVal. loopVar specifies a vector of integer values increasing by 1. If you have Parallel Computing Toolbox™, the iterations of statements can execute on a parallel pool of workers on your multi-core computer or cluster. As with a for-loop, you can include a single line or multiple lines in statements.

To find out how parfor can help increase your throughput, see Decide When to Use parfor.

parfor differs from a traditional for-loop in the following ways:

• Loop iterations are executed in parallel in a nondeterministic order. This means that you might need to modify your code to use parfor. For more help, see Convert for-Loops Into parfor-Loops.

• Loop iterations must be consecutive, increasing integer values.

• The body of the parfor-loop must be independent. One loop iteration cannot depend on a previous iteration, because the iterations are executed in a nondeterministic order. For more help, see Ensure That parfor-Loop Iterations are Independent.

• You cannot use a parfor-loop inside another parfor-loop. For more help, see Nested parfor and for-Loops and Other parfor Requirements.

parfor (loopVar = initVal:endVal,M); statements; end uses M to specify the maximum number of workers from the parallel pool to use in evaluating statements in the loop body. M must be a nonnegative integer.

By default, MATLAB uses the available workers in your parallel pool. You can change the default number of workers in your parallel pool using the PreferredPoolNumWorkers property of the default profile. For all factors that can affect your default pool size, see Factors That Affect Pool Size. You can override the default number of workers in a parallel pool by using the parpool function. When no workers are available in the pool or M is zero, MATLAB still executes the loop body in a nondeterministic order, but not in parallel. Use this syntax to switch between parallel and serial execution when testing your code.

With this syntax, to execute the iterations in parallel, you must have a parallel pool of workers. By default, if you execute parfor, you automatically create a parallel pool of workers on the parallel environment defined by your default profile. The default parallel environment is Processes. You can change your profile in Parallel Preferences. For more details, see Specify Your Parallel Preferences.

parfor (loopVar = initVal:endVal,opts); statements; end uses opts to specify the resources to use in evaluating statements in the loop body. Create a set of parfor options using the parforOptions function. With this approach, you can run parfor on a cluster without first creating a parallel pool and control how parfor partitions the iterations into subranges for the workers.

parfor (loopVar = initVal:endVal,cluster); statements; end executes statements on workers in cluster without creating a parallel pool. This is equivalent to executing parfor (loopVar = initVal:endVal,parforOptions(cluster)); statements; end.

Examples

Convert a for-Loop Into a parfor-Loop

Create a parfor-loop for a computationally intensive task and measure the resulting speedup.

In the MATLAB Editor, enter the following for-loop. To measure the time elapsed, add tic and toc.

tic
n = 200;
A = 500;
a = zeros(1,n);
for i = 1:n
    a(i) = max(abs(eig(rand(A))));
end
toc

Run the script, and note the elapsed time.

Elapsed time is 31.935373 seconds.

In the script, replace the for-loop with a parfor-loop.

tic
n = 200;
A = 500;
a = zeros(1,n);
parfor i = 1:n
    a(i) = max(abs(eig(rand(A))));
end
toc

Run the new script, and run it again. The first run is slower than the second run, because the parallel pool has to be started, and you have to make the code available to the workers. Note the elapsed time for the second run.

By default, MATLAB automatically opens a parallel pool of workers on your local machine.

Elapsed time is 10.760068 seconds. 

Observe that you speed up your calculation by converting the for-loop into a parfor-loop on four workers. You might reduce the elapsed time further by increasing the number of workers in your parallel pool. For more information, see Convert for-Loops Into parfor-Loops and Scale Up parfor-Loops to Cluster and Cloud.

Test parfor-Loops by Switching Between Parallel and Serial Execution

You can specify the maximum number of workers M for a parfor-loop. Set M = 0 to run the body of the loop in the desktop MATLAB, without using workers, even if a pool is open. When M = 0, MATLAB still executes the loop body in a nondeterministic order, but not in parallel, so that you can check whether your parfor-loops are independent and suitable to run on workers. This is the simplest way to allow you to debug the contents of a parfor-loop. You cannot set breakpoints directly in the body of the parfor-loop, but you can set breakpoints in functions called from the body of the parfor-loop.

Specify M = 0 to run the body of a parfor-loop in the desktop MATLAB, even if a pool is open.

 M = 0;                     % M specifies maximum number of workers
 y = ones(1,100);
 parfor (i = 1:100,M)
      y(i) = i;
 end

To control the number of workers in your parallel pool, see Specify Your Parallel Preferences and parpool.

Measure Data Transferred to Workers Using a parfor-Loop

To measure how much data is transferred to and from the workers in your current parallel pool, add ticBytes(gcp) and tocBytes(gcp) before and after the parfor-loop. Use gcp as an argument to get the current parallel pool.

Delete your current parallel pool if you still have one.

delete(gcp('nocreate'))

tic
ticBytes(gcp);
n = 200;
A = 500;
a = zeros(1,n);
parfor i = 1:n
    a(i) = max(abs(eig(rand(A))));
end
tocBytes(gcp)
toc

Run the new script, and run it again. The first run is slower than the second run, because the parallel pool has to be started, and you have to make the code available to the workers.

By default, MATLAB automatically opens a parallel pool of workers on your local machine.

Starting parallel pool (parpool) using the 'Processes' profile ... connected to 4 workers.
...
             BytesSentToWorkers    BytesReceivedFromWorkers
             __________________    ________________________

    1        15340                  7024                   
    2        13328                  5712                   
    3        13328                  5704                   
    4        13328                  5728                   
    Total    55324                 24168                   

You can use the ticBytes and tocBytes results to examine the amount of data transferred to and from the workers in a parallel pool. In this example, the data transfer is small. For more information about parfor-loops, see Decide When to Use parfor and Convert for-Loops Into parfor-Loops.

Run parfor on a Cluster Without a Parallel Pool

Create a cluster object using the parcluster function, and create a set of parfor options with it. By default, parcluster uses your default cluster profile. Check your default profile on the MATLAB Home tab, in Parallel > Select Parallel Environment.

cluster = parcluster;

To run parfor computations directly in the cluster, pass the cluster object as the second input argument to parfor.

When you use this approach, parfor can use all the available workers in the cluster, and workers become available as soon as the loop completes. This approach is also useful if your cluster does not support parallel pools. If you want to control other options, including partitioning of iterations, use parforOptions.

values = [3 3 3 7 3 3 3];
parfor (i=1:numel(values),cluster)
    out(i) = norm(pinv(rand(values(i)*1e3)));
end

Use this syntax to run parfor on a large cluster without consuming workers for longer than necessary.

Input Arguments

loopVar — Loop index

integer

Loop index variable with initial value initVal and final value endVal. The variable can be any numeric type and the value must be an integer.

Make sure that your parfor-loop variables are consecutive increasing integers. For more help, see Troubleshoot Variables in parfor-Loops.

The range of the parfor-loop variable must not exceed the supported range. For more help, see Avoid Overflows in parfor-Loops.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

initVal — Initial value of loop index

integer

Initial value loop index variable, loopVar. The variable can be any numeric type and the value must be an integer. With endVal, specifies the parfor range vector, which must be of the form M:N.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

endVal — Final value of loop index

integer

Final value loop index variable, loopVar. The variable can be any numeric type and the value must be an integer. With initVal, specifies the parfor range vector, which must be of the form M:N.

Data Types: single | double | int8 | int16 | int32 | int64 | uint8 | uint16 | uint32 | uint64

statements — Loop body

text

Loop body, specified as text. The series of MATLAB commands to execute in the parfor-loop.

You might need to modify your code to use parfor-loops. For more help, see Convert for-Loops Into parfor-Loops

Do not nest parfor-loops, see Nested parfor and for-Loops and Other parfor Requirements.

M — Maximum number of workers running in parallel

number of workers in the parallel pool (default) | nonnegative integer

Maximum number of workers running in parallel, specified as a nonnegative integer. If you specify an upper limit, MATLAB uses no more than this number, even if additional workers are available. If you request more workers than the number of available workers, then MATLAB uses the maximum number of workers available at the time of the call. If the loop iterations are fewer than the number of workers, some workers perform no work.

If parfor cannot run on multiple workers (for example, if only one core is available or M is 0), MATLAB executes the loop in a serial manner. In this case, MATLAB still executes the loop body in a nondeterministic order. Use this syntax to switch between parallel and serial when testing your code.

opts — parfor options

parforOptions object

parfor options, specified as a ClusterOptions object. Use the parforOptions function to create a set of parfor options.

Example: opts = parforOptions(parcluster);

cluster — Cluster

parallel.Cluster

Cluster, specified as a parallel.Cluster object, on which parfor runs. To create a cluster object, use the parcluster function.

Example: cluster = parcluster('Processes')

Data Types: parallel.Cluster

Tips

• Use a parfor-loop when:

  • You have many loop iterations of a simple calculation. parfor divides the loop iterations into groups so that each thread can execute one group of iterations.

  • You have some loop iterations that take a long time to execute.

• Do not use a parfor-loop when an iteration in your loop depends on the results of other iterations.

  Reductions are one exception to this rule. A reduction variable accumulates a value that depends on all the iterations together, but is independent of the iteration order. For more information, see Reduction Variables.

• When you use parfor, you have to wait for the loop to complete to obtain your results. Your client MATLAB is blocked and you cannot break out of the loop early. If you want to obtain intermediate results, or break out of a for-loop early, try parfeval instead.

• Unless you specify a cluster object, a parfor-loop runs on the existing parallel pool. If no pool exists, parfor starts a new parallel pool, unless the automatic starting of pools is disabled in your parallel preferences. If there is no parallel pool and parfor cannot start one, the loop runs serially in the client session.

• If the AutoAttachFiles property in the cluster profile for the parallel pool is set to true, MATLAB performs an analysis on a parfor-loop to determine what code files are necessary for its execution, see listAutoAttachedFiles. Then MATLAB automatically attaches those files to the parallel pool so that the code is available to the workers.

• You cannot call scripts directly in a parfor-loop. However, you can call functions that call scripts.

• Do not use clear inside a parfor loop because it violates workspace transparency. See Ensure Transparency in parfor-Loops or spmd Statements.

• You can run Simulink^® models in parallel with the parsim command instead of using parfor-loops. For more information and examples of using Simulink in parallel, see Running Multiple Simulations (Simulink).

• For GPU computations:

  • Do not use a parfor-loop if you have a single GPU and your loop iterations all use the same GPU. GPUs contain many microprocessors that can perform computations in parallel and trying to further parallelize GPU computations using a parfor-loop is unlikely to speed up your code.

  • Use a parfor-loop if you have multiple GPUs and your computations use GPU-enabled functions. For more information about using multiple GPUs in a parfor-loop, see Run MATLAB Functions on Multiple GPUs.

Extended Capabilities

Thread-Based Environment

Run code in the background using MATLAB® backgroundPool or accelerate code with Parallel Computing Toolbox™ ThreadPool.

This function fully supports thread-based environments. For more information, see Run MATLAB Functions in Thread-Based Environment.

Version History

Introduced in R2008a

R2024b: Use colon-vector indexing expressions with sliced variables

Starting in R2024b, you can now use colon-vector indexing expressions to index sliced input and output variables in parfor-loops. The colon-vector indexing expression must be in the form j:k or j:k:l.

For example, to assign values only to columns 3 to 7 of the output variable out, use the vector 3:7 as a subscript when you index the sliced variable.

out = zeros(10);
parfor i = 1:10
    out(i,3:7) = rand(1,5);
end

You can use either simple broadcast variables or scalar integer constants in the colon-vector indexing expressions. Temporary variables or complicated expressions are not supported.

R2024a: Specify maximum number of thread workers in parfor-loop

Starting in R2024a, you can now specify the maximum number of workers when running parfor-loops on a thread-based parallel pool.