Find Extremum of Multivariate Function and Its Approximation

Open Live Script

This example shows how to find the extremum of a multivariate function and its approximation near the extremum point. This example uses symbolic matrix variables to represent the multivariate function and its derivatives. Symbolic matrix variables are available starting in R2021a.

Consider the multivariate function $f (x) = \sin (x^{T} A x)$ , where $x$ is a 2-by-1 vector and $A$ is a 2-by-2 matrix. To find a local extremum of this function, compute a root of the derivative of $f (x)$ . In other words, find the solution of the derivative $\nabla f (x_{0}) = 0$ .

Create Function and Find Its Derivative

Create the vector $x$ and the matrix $A$ as symbolic matrix variables. Define the function $f (x) = \sin (x^{T} A x)$ .

syms x [2 1] matrix
syms A [2 2] matrix
f = sin(x.'*A*x)

f =  $\sin (x^{T} A x)$

Compute the derivative D of the function $f (x)$ with respect to the vector $x$ . The derivative D is displayed in compact matrix notation in terms of $x$ and $A$ .

D = diff(f,x)

D =  $\cos (x^{T} A x) (x^{T} A + x^{T} A^{T})$

Convert `symmatrix` Objects to `sym` Objects

The symbolic matrix variables x, A, f, and D are symmatrix objects. These objects represent matrices, vectors, and scalars in compact matrix notation. To show the components of these variables, convert the symmatrix objects to sym objects using symmatrix2sym.

xsym = symmatrix2sym(x)

xsym = 
 $(\begin{array}{c} x_{1} \\ x_{2} \end{array})$

Asym = symmatrix2sym(A)

Asym = 
 $(\begin{array}{c} A_{1, 1} & A_{1, 2} \\ A_{2, 1} & A_{2, 2} \end{array})$

fsym = symmatrix2sym(f)

fsym =  $\sin (x_{1} (A_{1, 1} x_{1} + A_{1, 2} x_{2}) + x_{2} (A_{2, 1} x_{1} + A_{2, 2} x_{2}))$

Dsym = symmatrix2sym(D)

Dsym =  $(\begin{array}{c} \cos (x_{1} (A_{1, 1} x_{1} + A_{1, 2} x_{2}) + x_{2} (A_{2, 1} x_{1} + A_{2, 2} x_{2})) (2 A_{1, 1} x_{1} + A_{1, 2} x_{2} + A_{2, 1} x_{2}) & \cos (x_{1} (A_{1, 1} x_{1} + A_{1, 2} x_{2}) + x_{2} (A_{2, 1} x_{1} + A_{2, 2} x_{2})) (A_{1, 2} x_{1} + A_{2, 1} x_{1} + 2 A_{2, 2} x_{2}) \end{array})$

Substitute Numeric Values and Find the Minimum

Suppose you are interested in the case where the value of $A$ is [2 -1; 0 3]. Substitute this value into the function fsym.

fsym = subs(fsym,Asym,[2 -1; 0 3])

fsym =  $\sin (3 {x_{2}}^{2} + x_{1} (2 x_{1} - x_{2}))$

Substitute the value of $A$ into the derivative Dsym

Dsym = subs(Dsym,Asym,[2 -1; 0 3])

Dsym =  $(\begin{array}{c} \cos (3 {x_{2}}^{2} + x_{1} (2 x_{1} - x_{2})) (4 x_{1} - x_{2}) & - \cos (3 {x_{2}}^{2} + x_{1} (2 x_{1} - x_{2})) (x_{1} - 6 x_{2}) \end{array})$

Then, apply the symbolic function solve to get a root of the derivative.

[xmin,ymin] = solve(Dsym,xsym,'PrincipalValue',true);
x0 = [xmin; ymin]

x0 = 
 $(\begin{array}{c} 0 \\ 0 \end{array})$

Plot the function $f (x)$ together with the extremum solution $x_{0}$ . Set the plot interval to $- 1 < x_{1} < 1$ and $- 1 < x_{2} < 1$ as the second argument of fsurf. Use fplot3 to plot the coordinates of the extremum solution.

fsurf(fsym,[-1 1 -1 1])
hold on
fplot3(xmin,ymin,subs(fsym,xsym,x0),'ro')
view([-68 13])

Figure contains an axes object. The axes object contains 2 objects of type functionsurface, parameterizedfunctionline.

Approximate Function Near Its Minimum

You can approximate a multivariate function around a point $x_{0}$ with a multinomial using the Taylor expansion.

$f (x) \approx f (x_{0}) + \nabla f (x_{0}) \cdot (x - x_{0}) + \frac{1}{2} (x - x_{0})^{T} H (f (x_{0})) (x - x_{0})$

Here, the term $\nabla f (x_{0})$ is the gradient vector, and $H (f (x_{0}))$ is the Hessian matrix of the multivariate function $f (x)$ calculated at $x_{0}$ .

Find the Hessian matrix and return the result as a symbolic matrix variable.

H = diff(f,x,x.')

H =  $- \sin (x^{T} A x) (A^{T} x + A x) (x^{T} A + x^{T} A^{T}) + \cos (x^{T} A x) (A^{T} + A)$

Convert the Hessian matrix $H (f (x_{0}))$ to the sym data type, which represents the matrix in its component form. Use subs to evaluate the Hessian matrix for $A$ = [2 -1; 0 3] at the minimum point $x_{0}$ .

Hsym = symmatrix2sym(H);
Hsym = subs(Hsym,Asym,[2 -1; 0 3]);
H0 = subs(Hsym,xsym,x0)

H0 = 
 $(\begin{array}{c} 4 & - 1 \\ - 1 & 6 \end{array})$

Evaluate the gradient vector $\nabla f (x_{0})$ at $x_{0}$ .

D0 = subs(Dsym,xsym,x0)

D0 =  $(\begin{array}{c} 0 & 0 \end{array})$

Compute the Taylor approximation to the function near its minimum.

fapprox = subs(fsym,xsym,x0) + D0*(xsym-x0) + 1/2*(xsym-x0).'*H0*(xsym-x0)

fapprox = 
 $x_{1} (2 x_{1} - \frac{x_{2}}{2}) - x_{2} (\frac{x_{1}}{2} - 3 x_{2})$

Plot the function approximation on the same graph that shows $f (x)$ and $x_{0}$ .

hold on
fsurf(fapprox,[-1 1 -1 1])
zlim([-1 3])

Figure contains an axes object. The axes object contains 3 objects of type functionsurface, parameterizedfunctionline.

Find Extremum of Multivariate Function and Its Approximation

Create Function and Find Its Derivative

Convert symmatrix Objects to sym Objects

Substitute Numeric Values and Find the Minimum

Approximate Function Near Its Minimum

Convert `symmatrix` Objects to `sym` Objects