Global Optimisation and nested function problem

Dear Support staff,

My question is about the availability of arguments (variables) mentioned and how they are even without being visible,availble for second funtion which is nested inside the second.

Please have a look at the script first-

%% Script - 1
function [fitted_params, model, fval, exitflag, output, optimisationLog] = Fitautocorrvins(radius, radialdist, optimisationLog)

    model = @objective_function;
    
    % lower bounds...
    % upper bounds...
    % initial guess...
    
    fmincoptions = optimoptions('fmincon', 'Display', 'iter', 'MaxIterations', 10000, 'MaxFunctionEvaluations', 10000, 'StepTolerance', 1e-9, 'FunctionTolerance', 1e-9);
        
    % Performing optimisation
    [fitted_params, fval, exitflag, output] = fmincon(model, initialpt, [], [], [], [], lwrbnds, uprbnds, [], fmincoptions);
    
    function [sse, fitted_ccurve] = objective_function(OrgParams)
        C1 = OrgParams(1);
        xi1 = OrgParams(2);
        beta = OrgParams(3);
        lambda = OrgParams(4);
        r1 = OrgParams(5);
      
        fitted_curve = (C1 * exp(-(radius/xi1)) .* (cos((radius - r1) ./ lambda)));
        error = fitted_curve - radialdist;
        sse = sum(error.^2);

    end

end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Script - 2
function [fitted_params, model, fval, exitflag, output, optimisationLog, fitted_Curve] = Fitautocorrvins(radius, radialdist, optimisationLog)

    model = @objective_function;
    
    % lower bounds...
    % upper bounds...
    % initial guess...
    
    fmincoptions = optimoptions('fmincon', 'Display', 'iter', 'MaxIterations', 10000, 'MaxFunctionEvaluations', 10000, 'StepTolerance', 1e-9, 'FunctionTolerance', 1e-9);
        
    % Performing optimisation
    [fitted_params, fval, exitflag, output] = fmincon(@(params)objective_function(params, radius, radialdist), initialpt, [], [], [], [], lwrbnds, uprbnds, [], fmincoptions);
    
    % calculating fitted_curve
    
    fitted_curve = ..................;
        
    function [sse, fitted_ccurve] = objective_function(OrgParams)
        C1 = OrgParams(1);
        xi1 = OrgParams(2);
        beta = OrgParams(3);
        lambda = OrgParams(4);
        r1 = OrgParams(5);
      
        fitted_curve = (C1 * exp(-(radius/xi1)) .* (cos((radius - r1) ./ lambda)));
        error = fitted_curve - radialdist;
        sse = sum(error.^2);

    end

end

    

• Both Script - 1 and Script - 2 results exactly same, my question is how does Script-1, when it comes to nested function "objective_function" gets to calculate "fitted_curve" even when "radius" and "radialdist" variables are not available to it? Just to mention, "radius" and "radialdist" are not defined as global variables. If I even put a breakpoint when "objective_function" is executing, I can see empty workspace clearly. How come it can still calculate fitted_curve and at the end answers to both script are exactly same !?!

Now comes the problem of "genetic algorithm" optimisation method. Please have a look at the two scripts-

%% Script - 3
function [fitted_params, model, fval, exitflag, output, optimisationLog] = Fitautocorrvins(radius, radialdist, optimisationLog)

    model = @objective_function;
    
    % lower bounds...
    % upper bounds...
    
    
    gaoptions = optimoptions('ga', 'Display', 'iter', 'PopulationSize', 1000, 'MaxGenerations', 1000, 'CrossoverFraction', 0.1, 'FunctionTolerance', 1e-2, UseParallel=true);
        
    % Performing optimisation
    
    [fitted_params, fval, exitflag, output] = ga(model, numel(lwrbnds), [], [], [], [], lwrbnds, uprbnds, [], gaoptions);
    
    % calculating fitted_curve
    
    fitted_curve = ..................;
        
    function [sse, fitted_curve] = objective_function(OrgParams)
        C1 = OrgParams(1);
        xi1 = OrgParams(2);
        beta = OrgParams(3);
        lambda = OrgParams(4);
        r1 = OrgParams(5);
      
        fitted_curve = (C1 * exp(-(radius/xi1)) .* (cos((radius - r1) ./ lambda)));
        error = fitted_curve - radialdist;
        sse = sum(error.^2);

    end

end
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% Script - 4
function [fitted_params, model, fval, exitflag, output, optimisationLog, fitted_curve] = Fitautocorrvins(radius, radialdist, optimisationLog)

    model = @objective_function;
    
    % lower bounds...
    % upper bounds...
    
    
    gaoptions = optimoptions('ga', 'Display', 'iter', 'PopulationSize', 1000, 'MaxGenerations', 1000, 'CrossoverFraction', 0.1, 'FunctionTolerance', 1e-2, UseParallel=true);
        
    % Performing optimisation
    
    [fitted_params, fval, exitflag, output] = ga(@(params)modified_oscillate(params, radius, radialdist), numel(lwrbnds), [], [], [], [], lwrbnds, uprbnds, [], gaoptions);
    
    % calculating fitted_curve
    
    fitted_curve = ..................;
        
    function [sse] = objective_function(OrgParams, radius, radialdist)
        C1 = OrgParams(1);
        xi1 = OrgParams(2);
        beta = OrgParams(3);
        lambda = OrgParams(4);
        r1 = OrgParams(5);
      
        fitted_curve = (C1 * exp(-(radius/xi1)) .* (cos((radius - r1) ./ lambda)));
        error = fitted_curve - radialdist;
        sse = sum(error.^2);

    end

end

• Why does GA fitted_params keeps changing? How do I make sure of a more consistent and reproducible results?
• Which is the right syntax while doing GA optimisation? ga(@(params)modified_oscillate(params, radius, radialdist)... or ga(model, numel(lb)............?

I am really grateful for your help in advance.

Regards

Vinay

[Stephen23]

"my question is how does Script-1, when it comes to nested function "objective_function" gets to calculate "fitted_curve" even when "radius" and "radialdist" variables are not available to it?"

They are available to it: nested functions have access to all variables in their parent function's workspace. That is the whole point of nested functions. This is explained in the documentation:

https://www.mathworks.com/help/matlab/matlab_prog/nested-functions.html

"Just to mention, "radius" and "radialdist" are not defined as global variables."

Not using globals .. is a good thing!

"Why does GA fitted_params keeps changing?"

GA by default relies on a random population at the start.

"Which is the right syntax while doing GA optimisation?"

It doesn't really matter, just pick one and stick with it... but stop mixing them up!

Scripts 1 & 3 uses a nested functions to parameterize the objective function.

Scripts 2 & 4 mix up nested functions and anonymous functions when trying to parameterize the objective function. If you are going to parameterize the function using an anonymous function:

• Although it may be a nested function, for clarity and efficiency the objective function should be a local function: https://www.mathworks.com/help/matlab/matlab_prog/local-functions.html
• Script 2 will throw an error because you did not modify the objective function to accept all of the required input arguments.
• MODEL is misleadingly defined but never used. Get rid of it.

[Vinay]

Dear Stephen,

It is absolutely clear now and you have well explained. I really appreciate your help.

Regards

Vinay

Answer 1

[Alan Weiss]

The reason objective_function can see those parameters is because you passed them into the Fitautocorrvins function, and that is how nested functions work. For more information, see Nested Functions.

I would like to suggest that, instead of using ga to optimize a least-squares problem, you instead use the purpose-built lsqnonlin or lsqcurvefit functions. And if you are trying to get a global solution, use MultiStart with lsqnonlin or lsqcurvefit. This approach with be much, much more efficient and reliable, especially if you give good bounds on the parameters.

Good luck,

Alan Weiss

MATLAB mathematical toolbox documentation

[Vinay]

Dear @Alan Weiss,

I appreciate your response and thank you for your time.

So, just to clarify- these both %% 1 and %%2 syntax are basically same and does not require explicit passing of "params", "radius", "radialdist" to the nested objective_function. Is that correct?

Regarding MultiStart, thank you for sharing this with me, I will try it now and will get back to you if I have more questions.

Regards

Function [..] = Fitautocorrvins[..]

    model = @objective_function;
    
    % Rest of the code %
    
    %% 1
    [fitted_params, fval, exitflag, output] = ga(@(params)objective_function(params, radius, radialdist), ...
        numel(lwrbnds), [], [], [], [], lwrbnds, uprbnds, [], gaoptions);
    
    %% 2
    [fitted_params, fval, exitflag, output] = ga(model, ...
        numel(lwrbnds), [], [], [], [], lwrbnds, uprbnds, [], gaoptions);
    
    % Rest of the code %
    
    function [sse] = objective_function(OrgParams)
        C1 = OrgParams(1);
        xi1 = OrgParams(2);
        beta = OrgParams(3);
        lambda = OrgParams(4);
        r1 = OrgParams(5);
      
        fitted_curve = (C1 * exp(-(radius/xi1)) .* (cos((radius - r1) ./ lambda)));
        error = fitted_curve - radialdist;
        sse = sum(error.^2);
    end
    
end

Vinay

[Stephen23]

"these both %% 1 and %%2 syntax are basically same"

Not quite:

1. correctly parameterizes the objective function using an anonymous function but will fail because the objective function has not been written to accept all of those input arguments. Also for clarity and efficiency the nested function should be moved to a local function. And get rid of the misleading MODEL, which is unused.
2. correctly parameterizes the objective function using a nested function.

Do not mix them up (until you know what you are doing): just pick one approach and stick with it!

"does not require explicit passing of "params", "radius", "radialdist" to the nested objective_function. Is that correct?"

No.

ORGPARAMS must be defined by GA every time it calls your objective function, whatever parameterization approach you choose. If GA (or FMINCON or whatever) cannot change ORGPARAMs then it will not work.

Any other arrays/values can be provided by either using a nested function or an anonymous function, as explained here:

https://www.mathworks.com/help/matlab/math/parameterizing-functions.html

Pick one of those methods and stick with it. Do not mix them up (until you know what you are doing):

• If you are using a nested function then you do not need to explictly provide the other arrays/values as input arguments: they are shared from the parent workspace to all nested workspaces automatically. That is the whole point of nested functions.
• If you are using an anonymous function then forget all about nested functions: make your objective function a local function and pass all required values as input aguments.

[Vinay]

Dear Stephen,

It is absolutely clear now and you have well explained. I really appreciate your help.

Regards

Vinay