How to implement linesearch in optimization algorithms?

I am trying to implement a linesearching procedure in an optimization algorithm. $\eta_n=\eta^{m_n}$ where $m_n$ is the smallest integer $m$ such that $\eta_n\|A(x)-A(y)\| \le \mu\|x-y\|.$ Please how do I execute this in function mode?

[Malay Agarwal]

Could you please explain what exactly it is you want to achieve? The question is not very clear.

[Olawale Oyewole]

I want to implement (14) in this attachment.

[Torsten]

And what does Step 1 has to do with linesearching ?

[Olawale Oyewole]

x1 = 5;
tol = 1e-6;
[uu,eru]=SFS(x1,tol)

Index exceeds the number of array elements. Index must not exceed 1.

Error in solution>F (line 43)
y=[3*x(1)*(log(x(1)*x(2)/x(3))),3*x(2)*(log(x(1)*x(2)/x(3))),-3*x(3)*(log(x(1)*x(2)/x(3)))];

Error in solution>SFS (line 9)
in_zn=A(uu,F(uu),-lambda);

function [uu,eru]=SFS(x1,tol)
uu=x1; n=1; eru=1;
sigma=0.5;
while (eru > tol)
    in_zn=A(uu,F(uu),-lambda);
    zn=PC(in_zn);
    alph=0.213;
    for mk=1:1:n
        alpha=alph^(mk);
        in_yn=B(uu,zn);
        yn=A(uu,in_yn,alpha);
        left=inner(F(uu),B(uu,yn),uu)+inner(F(yn),B(yn,uu),yn);
        right=-(sigma*alpha/lambda)*error(uu,zn)^2;
        if right <= left
            break
        else
        end
    end
    cam=inner(F(yn),B(yn,uu),yn); %my code is diplaying an error here "Variable might be used before it is defined. Whereas yn is already defined in the for loop.
    if cam > 0
        wn=uu-(cam*F(yn)/norm(F(yn))^2);
    else
        wn=uu;
    end
    x2=PC(wn);
    erx(n)=error(x2,uu);
    uu=x2;
    n=n+1;
end
end
function z=A(x,y,t)
v=t*y/x;
z=x*exp(v);
end
function z=B(x,y)
z=[x(1)*log(y(1)/x(1)),x(2)*log(y(2)/x(2)),x(3)*log(y(3)/x(3))];
end
function y=F(x)
y=[3*x(1)*(log(x(1)*x(2)/x(3))),3*x(2)*(log(x(1)*x(2)/x(3))),-3*x(3)*(log(x(1)*x(2)/x(3)))];
%function y=P(x,y,v)
%y=v*y/x;
end
function z=PC(x)
t=dot(x,-1/2); s=0;
if t <= s
    z=x;
else
    z=x-((t-s)/norm(2)^2)*2;
end
end
function z=error(x,y)
a=(log(x(1)/y(1)))^2;
b=(log(x(2)/y(2)))^2;
c=(log(x(3)/y(3)))^2;
z=sqrt(a+b+c);
end
function z=inner(u,v,x)
a=[x(1)^(-2),x(2)^(-2),x(3)^(-2)];
b=diag(a);
z=u*b*v';
end

[Olawale Oyewole]

(14) is referred to as a line search technique in the sense of fixed point and optimization theory.

Answer 1

[Harimurali]

Hi Olawale,

I understand that you want to implement a line searching procedure for the given optimization algorithm. The error being shown while running the MATLAB code given in the comments is because in line 8, in the function call of A, you are calling the function F with a scalar value, but the function F expects a vector as an argument.

As for the implementation of the line searching procedure in MATLAB for the given optimization algorithm, where is the smallest integer m such that .

You can use a loop to increment the value of muntil the inequality is satisfied by following these steps:

• Let the initial value of m = 1
• In the loop, calculate the left-hand side of the inequality using the values provided for and the functions and
• If the inequality is satisfied, then the smallest value of m that is is obtained. Otherwise, m is incremented, and the loop continues until the inequality is satisfied

Hope this helps.

[Olawale Oyewole]

Thanks for your help. Can you please type this into the code I earlier sent and I will take it from there? Thanks once again.

[Olawale Oyewole]

Do you mean I use the loop as currently defined?

[Harimurali]

Sorry, I am not sure how to incorporate this into the code that you have given. I noticed that you used the variable uu which contains a scalar value in two ways:

• while calling the function A,where it is used as a scalar
• while calling the function F, where it is used as a vector

The second one errors out. Could you try modifying the code so that these conflicts are resolved?

[Olawale Oyewole]

We can leave the error part for now as I will try to change everything to scalar and rework the code if I need to have vector values. My question now is the loop part. Do you mean this?

for m=1

alpha=alph^(m);

in_yn=B(uu,zn);

yn=A(uu,in_yn,alpha);

left=inner(F(uu),B(uu,yn),uu)+inner(F(yn),B(yn,uu),yn);

right=-(sigma*alpha/lambda)*error(uu,zn)^2;

if right <= left

break

else

end

[Harimurali]

I am not able to assess the logical correctness of this loop due to the multiple dependencies, which are the calls to the functions inner, A, B, F, and error. This looks very different from the equation you have provided in the question. As this loop is part of the implementation of the whole algorithm, there is a lot of coupling with the rest of the code. You could try to implement and run the loop separately and then integrate it with the rest of the code. Also, due to the scalar-vector conflict, I feel that a major rework of the code may be required.

[Olawale Oyewole]

Thanks.