Hessian matrix produced by fmininuc

Dear Sir/Madam:

fminunc is producing a Hessian matrix such that, when I take its Cholesky factor:

[CF,pp]=chol(Hessian);

I get that pp>0. Is there a way to force the algorithm to produce a Hessian such that pp=0?

Many thanks !

Best wishes,

Luca Benati

[Torsten]

What does

eig(Hessian) 

return ?

Answer 1

[Vaibhav]

Hi Luca,

You can try a post-optimization step where you adjust the Hessian matrix to ensure its positive definiteness. This adjustment is necessary because the Hessian's definiteness directly affects its suitability for certain mathematical operations, including Cholesky decomposition. Below is a code snippet that optimizes a quadratic function using fminunc and then modifies the resulting Hessian matrix to ensure it is positive definite, allowing for successful Cholesky decomposition, by iteratively adding a scaled identity matrix until the Cholesky decomposition succeeds.

% Define a simple quadratic objective function
objFunc = @(x) (x(1)-1)^2 + (x(2)-2)^2;

% Initial guess
x0 = [0, 0];

% Use fminunc to minimize the objective function without specifying gradient or Hessian options
options = optimoptions('fminunc', 'Algorithm', 'quasi-newton', 'Display', 'iter');
[x, fval, exitflag, output, grad, hessian] = fminunc(objFunc, x0, options);

% Attempt Cholesky decomposition of the Hessian
[CF, pp] = chol(hessian);

if pp > 0
    fprintf('The Hessian is not positive definite. Attempting to modify...\n');
    
    % Modify the Hessian to be positive definite
    epsilon = 1e-6; % Starting value
    Hessian_modified = hessian;
    while true
        [CF, pp] = chol(Hessian_modified);
        if pp == 0
            fprintf('Successfully modified the Hessian to be positive definite.\n');
            break;
        else
            Hessian_modified = Hessian_modified + epsilon * eye(size(hessian));
            epsilon = epsilon * 10; % Increase epsilon
        end
    end
else
    fprintf('The Hessian is positive definite. No modification needed.\n');
end

Hope it helps!