Lagrange multipliers

Page 1

1 Lagrange multipliers The objective of the notes is to explain why looking for a stationary point of a Lagrangian means looking for a saddle point . We first recall that for a C1 function a stationary point is a point where the gradient is zero, i.e. all partial derivatives are zero. For a convex function there is a unique stationary point and is a minimum. For a concave function there is a single stationary point, which is a maximum. Yet a stationary point may also be a saddle point, i.e. neither a minimum non a maximum (this is true even for functions of a single variable!). The theory of Lagrange multipliers in Hilbert spaces is rather tricky, so I limit the expla- nation to the much simpler case of finite dimensional spaces, and simple quadratic functionals. 1.1 A very simple problem Let’s consider the convex functionJ(x1,x 2) = 1 2(x2 1+x2 2)−5x1. We want to find the minimum of J subject to the constraint x1 +x2 = 0, i.e. find (ˆx1, ˆx2) such that J(ˆx1, ˆx2) = min x1∈R,x2∈R x1+x2=0 J(x1,x 2). In a more compact way, one may write (ˆx1, ˆx2) = arg min x1∈R,x2∈R x1+x2=0 J(x1,x 2). The problem is very simple, and with very few computations you find the result: x1 = 5/2 andx2 =−5/2. But suppose we stubbornly want to use Lagrange multipliers. We then define the Lagrange functional L(x1,x 2,λ ) =J(x1,x 2)−λ(x1 +x2) = 1 2(x2 1 +x2 2)− 5x1−λ(x1 +x2). Note that we could have defined instead ˆL(x1,x 2,λ ) =J(x1,x 2) +λ(x1 +x2) = 1 2(x2 1 +x2 2)− 5x1 +λ(x1 +x2). and (since the constraint is bilateral, i.e. it is an equality not an inequality) the result would be the same for x1 and x2, and the Lagrange multiplier λ would have the opposite sign. Now let’s first find a stationary point with respect to x1 and x2. For any value of λ, L is a convex quadratic function of x1 and x2. So the…