Parallel Computing is taking longer than without parallel

Hi, I'm testing the time consumed when using parallel computing, using the example (modified) below, from Matlab:

 clc; clear;

 tic

 A = [1 1; -1 2; 2 1]; b = [2; 2; 3]; lb = zeros(2,1);
 options = optimoptions('ga','MaxGenerations',10,'PopulationSize',200,'UseParallel','Always');
 [x,fval,exitflag] = ga(@lincontest6,2,A,b,[],[],lb,[],[],options);

 toc
 clear;
 tic

A = [1 1; -1 2; 2 1]; b = [2; 2; 3]; lb = zeros(2,1);
options = optimoptions('ga','MaxGenerations',10,'PopulationSize',200,'UseParallel','Never');
[x,fval,exitflag] = ga(@lincontest6,2,A,b,[],[],lb,[],[],options);

 toc

I expected that the first part would run faster than the lower one, but the opposite is happening. Im having result like this:

 Optimization terminated: maximum number of generations exceeded.
 Elapsed time is 31.912986 seconds.
 Optimization terminated: maximum number of generations exceeded.
 Elapsed time is 28.565836 seconds.

Before running I start a parallel pool with 2 workers (my pc has 4 cores). I tested with 3 and 4 and the same happens (first part slower than the second part). Anyone can explain why this is happening?? Thanks!!

Answer 1

[Alan Weiss]

Check out the first bullet item in this documentation. Your objective function, lincontest6, is very fast to compute, so parallel overhead dominates the calculation.

Alan Weiss

MATLAB mathematical toolbox documentation

Answer 2

[Walter Roberson]

It is common for working in parallel to take longer than working in serial, for two reasons:

1. Communications overhead. Even not counting the time it takes to set up the pool (which is not fast), the program and data have to be transferred to the separate processes for each worker, and the results have to be transferred back to the client. If not enough work is being done for each parfor iteration, then the cost of the communications overwhelms any gain from parallel processing;
2. When working with large enough matrices, for some of the more common computation patterns, MATLAB calls into high performance multi-threaded libraries that use all available cores. However, by default, when you run in parallel, each worker is only allocated one core and so although the same libraries are called all of the work ends up being done in serial... while still having had the overhead of setting up to call the libraries.