Save multiple optimization plots

I am running fminunc several times in a for-loop to train an one-vs-all classifier. I would like to plot the cost of every iteration, which is no problem for one call of fminunc using the options

options = optimoptions(@fminunc,'...', 'PlotFcn','optimplotfval');

But what I would like is something like to plot every call of fminunc and export every plot into a figure for later use.

I.e. I have my loop which calls fminunc several times with the PlotFcn-Option:

for i=1:num_labels
    initial_theta = zeros(n + 1, 1);
    all_theta(i,:)=fminunc( @(theta)(lr_costfunction(theta, X, (y == i), lambda)), initial_theta, options );

and would like to add something like this pseudocode

    figure()=findobj('Tag','optimplotfval');
end

within the loop, as PlotFcn erases previous plots with each new run.

If there is a better/ other solution than my first thought feel free to correct me.

Answer 1

[Avadhoot]

Hi Dennis,

I see that you are trying to plot the cost of every iteration of "fminunc" for every call to the function. You want the plot to update after every call such that the previous plot is not erased. You have the correct idea of doing it through the "PlotFcn" option in the "optimoptions".

You can accomplish your objective by creating a custom plotting function which takes the state of the "fminunc" and the figure handle as inputs and updates the figure with the new values. For this you will need to create the figure outside your for loop and then after every call to "fminunc" you need to save the figure. You can refer to the following code to understand how to do it.

function myOptimPlotFunction(state, figHandle)
% Custom plotting function that updates the figure with the current cost
figure(figHandle); % Make the figure current
plot(state.iteration, state.fval, 'bo-'); % Plot current value
xlabel('Iteration');
ylabel('Cost');
drawnow; % Force MATLAB to render the figure
end

num_labels = ...; % Number of labels
    options = optimoptions(@fminunc, 'OutputFcn', @(state,~,~)myOptimPlotFunction(state, figHandle), 'PlotFcn', []);

for i = 1:num_labels
    figHandle = figure; % Create a new figure for each label
    initial_theta = zeros(n + 1, 1);
    [all_theta(i,:), ~, ~, output] = fminunc(@(theta)(my_costfunction(theta, X, (y == i), lambda)), initial_theta, options);
    
    % After optimization, save the figure
    saveas(figHandle, sprintf('cost_function_label_%d.png', i));
    
    % You might want to close the figure if you don't want it to be open after saving
    % close(figHandle);
end

Here are the main changes introduced in the code:

1. "myOptimPlotFunction" is the plotting function that will be used to plot the fval against iterations. Here "drawnow" is used to update the figure after every plot.
2. The "optimoptions" are specified such that it will use "myOptimPlotFunction" as the "PlotFcn".
3. Inside the for loop, a saveas command is executed to save the figure after every call to "fminunc".
4. "my_costfunction" is the placeholder cost function you can replace with your cost function.

This will update the plot accordingly and the file will also be saved after every "fminunc" call.

For more information on "drawnow","saveas" and custom plotting functions, refer to the below documentation:

• drawnow:https://www.mathworks.com/help/matlab/ref/drawnow.html
• saveas:https://www.mathworks.com/help/matlab/ref/saveas.html
• Custom plotting function:https://www.mathworks.com/help/optim/ug/output-functions.html

I hope this helps.