Is it possible that optimisation using normal for loop gives better result in terms of execution time than parfor?

Hi

I used fmincon solver for optimising the equation

yhat = -(exp(-t./z(1))) + (b.*z(2)) - z(3);

where x are the coefficients to be optimized.

• Fmincon was called in the main function.
• Fmincon calls another function in which there is parafor loop. The parafor loop loads the input parameters(t and b) and ydata(expected o/p samples) and passes these and x to another function where yhat is calculated and returns the sum of square the error (yhat-ydata) . This value is then accumulated over the loop iteration and is returned back for fmincon solver to optimize.
• I have used the same code for optimisation using parfor and for loops and observed that execution time taken for parfor was more by 6s.

Is this because the loop was run only for 3 datasets each containing around 1000 samples?

Thanks Nadia

[Adam]

Was the parallel pool already open or did it have to be opened as part of the process? If it has to be opened specially for the process then it is certainly more than possible that it is slower than a for loop.

If the pool is already open then it could still potentially be slower given the overhead of copying data to and from workers, but it depends on the quantity of data being used.

[Nadia A]

Parallel pool was not open before I ran the script. I gave delete(gcp) at the end of the script for parfor section to ensure this. Parallel pool was not open from any other part of the script other than parfor.

[Adam]

In that case it will often be slower and not worth using compared to a normal for loop since the startup time is significant.

[Nadia A]

So incases where a parallel pool is already open and the quantity of data being used is less, parfor is effective than for. Is it?

[Adam]

It all depends on what the process is and how big is the overhead of transferring data to workers vs just doing the calculation in sequence. There isn't one easy answer so you have to experiment with the case you have. Having to open the parallel pool just adds an extra significant overhead to the time.

As with most parallel processing, it all comes down to how efficiently you do the parallelisation and how much processing you are actually doing. Generally there will be some crossover point beyond which parallel processing is faster and below which it isn't worth the overhead.