Finding a maximum is two separate problems: locating the candidates, and deciding which is which. The first is a first-derivative condition, the second a second-derivative one, and both change shape when the domain carries a constraint. The multiplier method turns a constrained problem into an unconstrained one, and the implicit function theorem is what makes it legal.
Optimisation under constraints is most of applied mathematics. Solving the constraint and substituting works only when you can solve it; the multiplier method never needs to, which is why it survives into settings where the constraint is a differential equation.
Support vector machines, portfolio optimisation, maximum entropy distributions, and the equilibrium conditions of thermodynamics are all multiplier computations. The multiplier itself usually has a meaning: a price, a temperature, a marginal cost.
Lagrange introduced the method in his 1788 Mécanique analytique as a way to handle mechanical constraints. The inequality version — the Karush–Kuhn–Tucker conditions of the 1940s–50s — is the foundation of modern nonlinear programming and is where the subject went next.
If $f$ is differentiable at an interior point $a$ of its domain and $a$ is a local maximum or minimum, then $Df(a)=0$.
For each direction $u$, the function $t\mapsto f(a+tu)$ has an interior extremum at $t=0$, so its one-variable derivative $Df(a)u$ vanishes. True for every $u$, so $Df(a)=0$. Such an $a$ is called a critical point.
A symmetric matrix $H$ is positive definite if $h^{T}Hh>0$ for all $h\ne0$, negative definite if the inequality reverses, and indefinite if $h^{T}Hh$ takes both signs. By the spectral theorem from MAT247H1, the sign pattern of the eigenvalues decides which.
Let $f$ be $C^2$ with $\nabla f(a)=0$.
The proof is Taylor. With $\nabla f(a)=0$, $f(a+h)-f(a)=\tfrac12h^{T}Hf(a)h+o(\|h\|^2)$. If $H$ is positive definite its smallest eigenvalue $\mu>0$ gives $h^{T}Hh\ge\mu\|h\|^2$, and for small $h$ that beats the remainder. Indefinite: pick eigenvectors of opposite sign and the two restrictions of $f$ go opposite ways.
All of $x^4+y^4$, $-x^4-y^4$ and $x^4-y^4$ have zero gradient and zero Hessian at the origin, and the origin is a minimum, a maximum and a saddle respectively. When the Hessian is singular, the second-order test cannot possibly decide, and you must look at $f$ itself.
Let $f$ and $g_1,\dots,g_k$ be $C^1$ near $a$, let $M=\{x:g_1(x)=\dots=g_k(x)=0\}$, and suppose the gradients $\nabla g_1(a),\dots,\nabla g_k(a)$ are linearly independent. If $f$ restricted to $M$ has a local extremum at $a$, then there are scalars $\lambda_1,\dots,\lambda_k$ with
The geometry in one line: at a constrained extremum the level set of $f$ is tangent to $M$, so their normals are parallel.
Maximise $f(x,y,z)=xyz$ on $x^2+y^2+z^2=3$. The condition $\nabla f=\lambda\nabla g$ reads $yz=2\lambda x$, $xz=2\lambda y$, $xy=2\lambda z$. Multiplying the three by $x,y,z$ and comparing gives $x^2=y^2=z^2=1$, so the candidates are the eight points $(\pm1,\pm1,\pm1)$. The maximum value is $1$, attained where an even number of coordinates are negative.
Maximise $x_1\cdots x_n$ subject to $x_1+\dots+x_n=s$ with all $x_i>0$. The multiplier equations force all $x_i$ equal, so the maximum of the product is $(s/n)^n$. Rearranged, that is exactly $\sqrt[n]{x_1\cdots x_n}\le(x_1+\dots+x_n)/n$.
A continuous $f$ on a compact $K$ attains its maximum, by Unit 3. To find it:
Then compare finitely many values. No second-order test is needed: the extreme value theorem already promised the maximum exists, so the largest candidate is it.
Maximise $f(x,y)=x^2-y^2+2x$ on $[-1,1]^2$. Interior: $\nabla f=(2x+2,-2y)$ vanishes at $(-1,0)$, which is on the boundary, so there is no interior critical point. Edges: on $x=1$, $f=3-y^2$ with maximum $3$ at $y=0$; on $x=-1$, $f=-1-y^2$ with maximum $-1$; on $y=\pm1$, $f=x^2+2x-1$ with maximum $2$ at $x=1$. Largest value $3$ at $(1,0)$.
• Applying the first-order condition at a boundary point. It holds at
interior points only; the maximum of $x$ on $[0,1]$ has nonzero
derivative.
• Concluding "saddle" from a singular Hessian. Singular means the test is
silent, not that the point is a saddle.
• Forgetting to check that the constraint gradients are independent.
Where they are dependent the multiplier equation can fail outright.
• Solving the multiplier equations and forgetting the constraint itself.
It is one of the equations of the system.
• Assuming a critical point exists. On a non-compact set $f$ may have no
extremum at all, and $e^x$ on $\mathbb{R}$ has neither.
Original problems written for this guide in the style of Spivak and Munkres exercises. Write a full solution before opening the answer.
Find and classify the critical points of $f(x,y)=x^3-3xy+y^3$.
Find the point of the plane $x+2y+3z=6$ nearest the origin.
Maximise $x+y+z$ on the intersection of $x^2+y^2=1$ and $z=x+y$.
Classify the origin for $f(x,y)=x^2y^2$.
Find the largest-area axis-aligned rectangle inscribed in $x^2/a^2+y^2/b^2=1$.
Minimise $x$ on $\{y^2=x^3\}$ and explain why the multiplier equation breaks.
If the constraint is $g(x)=c$, show $\lambda=dM/dc$ where $M(c)$ is the constrained maximum value.
Show the maximum of $x^{T}Ax$ on the unit sphere, for symmetric $A$, is the largest eigenvalue.
Course coverage. This unit is part of the complete MAT257Y1 topic map. It should cover the following items before you move on:
Spivak treats maxima briefly and Lagrange multipliers as an exercise; Munkres gives the tangent space argument in full. For the second-order constrained test, look up the bordered Hessian in an optimisation text. Confirm what the syllabus expects.
Implicit function theorem gives the tangent space, which gives Lagrange. Taylor gives the second-order test. Extreme value theorem is what guarantees the maximum you are hunting for exists at all.
Use these after the main practice set. They are designed to force definition-level reasoning, theorem selection, and mixed application rather than routine calculation only.
Why does the first-order condition need an interior point?
State the Lagrange condition geometrically.
When does the second-order test say nothing?
Why check compactness before hunting for a maximum?