Thread Subject: Hessian calculated by fmincon within a parfor or spmd environment

Dear all,

My code works fine in serial mode, and I experiment with parfor and spmd to speed it up. The code runs the same sequence of function calls 7 times (corresponding to 7 independent models), and it uses containers (vectors and arrays) of different sizes at a time. Each model utilizes fmincon with the Hessian at the minimum returned. The problem within a parallel environment appears as soon as fmincon finds a minimum for the first time and attempts to calculate the Hessian for that model.

A relevant part of the main driver looks like this

% Attempt a parfor or spmd
matlabpool
spmd (7)
switch labindex

case 1
  lambda    = 0.0604;
  mu        = [0.0696 -0.0249 -0.0108]';
(code omitted)
  DNS1      = optimize_logL('DNS_unc',lambda,mu,state_par, ...
                            state_cov,meas_cov,opt_on,OPG_on);

case 2
  lambda    = [0.06248 0.0531];
  mu        = [0.0723 -0.0294 -0.012 0.063]';
(code omitted)

where optimize_logL is the constructor of a user defined class.

Through the function optimize_logL and additional function calls therein, fmincon is invoked. The relevant part is the following

options = optimset('FinDiffType','forward', 'MaxFunEvals',50000, ...
                   'TolX',1e-7, 'TolFun',1e-7, 'MaxIter',10000, ...
                   'Algorithm','interior-point', 'Hessian','bfgs', ...
                   'Diagnostics','on', 'UseParallel','always');

[xopt,logLmax,exitflag,output,~,grad,hessian] = ...
    fmincon(@objective_fun_con,QML.param_vec,A,b,Aeq,beq,lb,ub, ...
            @stability,options);

with the objective function and non-linear constraint below

objective function

% redefine parameter containers
[lambda mu state_par state_cov meas_cov] = extract_parameters(QML,x);

% create and evolve a temporary model with stability checks off
model_temp = choose_model(QML,QML.model_name,lambda,mu, ...
                                state_par,state_cov,meas_cov);
in_sample_yield(model_temp,false);

% return the log-likelihood value
logL = model_temp.logL;

non-linear constraint

% redefine parameter containers
[~, ~, state_par] = extract_parameters(QML,x);
(comments omitted)
c = abs(eig(state_par))' - 0.9999999999999999;
ceq = [];

The containers lambda, mu, etc have different sizes per model. The temporary handle model_temp is created to initiate a series of calculations common to every model, and it should be destroyed at function exit.

The error message I get is

Error using spmd_feval (line 8)
Error detected on lab(s) 6

Caused by:

    Matrix dimensions must agree.

    Error stack:
    parfinitedifferences>(parfor body) at 158
    Sending a stop signal to all the labs ... stopped.

or if I use 'UseParallel','never'

Error using spmd_feval (line 8)
Error detected on lab(s) 6

Caused by:

    CAT arguments dimensions are not consistent.

    Error stack:
    formJacobian.m at 26
    barrier.m at 596
    fmincon.m at 841
    optimize_model.m at 59
    optimize_logL>optimize_logL.optimize_logL at 160
    Sending a stop signal to all the labs ... stopped.

I assume some of the model containers are mixed, while they are not supposed to do so. Since the evaluation of the Hessian follows the calculation of the function minimum as a part of fmincon, I cannot add a few diagnostic fprintf statements to see what is actually happening.

A few clarifications and things that I have tried:

1) I do not use global variables.

2) There is no file opening or I/O to a file involved, only some output to the screen.

3) I tried to add delete and clear statements in the objective function before it exits, without any positive result.

4) I tried a model_temp = []; before the function end but still no different result.

5) All the variables/containers that are passed in optimize_logL at the main driver are (re)defined for every case block of the switch statement.

6) There is no difference in the error messages between spmd and parfor.

7) Each of the 7 models is a user-defined class, and optimize_logL is a separate class. All 8 classes inherit a common superclass where the containers lambda, mu, etc are properties with protected access, and function in_sample_yield has public access.

Any suggestions to tackle this problem are welcome.

Georgios

Answer 1

[Jill Reese]

If you want to leverage the inherent parallelism in fmincon via the 'UseParallel' parameter then you shouldn't invoke the parfor or spmd language constructs directly. Instead, simply open a matlabpool and invoke your code

    lambda    = 0.0604;
    mu        = [0.0696 -0.0249 -0.0108]';
  (code omitted)
    DNS1      = optimize_logL('DNS_unc',lambda,mu,state_par, ...
                              state_cov,meas_cov,opt_on,OPG_on);

When fmincon is called within your function with 'UseParallel' set to 'Always', then the finite difference derivative estimation will always be performed in parallel using the open pool of workers.