2
$\begingroup$

I understand the concept behind Lagrange multipliers and the standard arguments used to prove it. But today I tried to derive it explicitly, and am having trouble with it. Let $f$ be the function we are trying to minimize under the constraint $g=0$. We know that if $x$ is a local minimizer, than $$\nabla f \cdot v =0$$ for any $v$ in the tangent space of the manifold defined by the constraint, as $$\nabla g \cdot v =0$$

Now let $u$ be an arbitrary vector. We can formulate its projection onte the said tangent space by $$v=u-\frac{\nabla g\cdot u}{\nabla g\cdot\nabla g}\nabla g$$ where $\frac{\nabla g\cdot u}{\nabla g\cdot\nabla g}$ is the scalar projection. We name it $\lambda$ for convenience. Thus $$v=u-\lambda \nabla g$$ This means that the minizier satisfies $$\nabla f \cdot v =\nabla f\cdot(u-\lambda\nabla g)=0$$ $$\nabla f\cdot u=\lambda\nabla f\cdot\nabla g$$ Which is obviously not the correct form.

Could someone please correct me? I am aware of how the proof usually goes, but I would like something of this form, where the equality is expressed through an arbitrary direction $u$ on both sides, something like $$\nabla f\cdot u =\lambda\nabla g\cdot u$$ for an arbitrary direction $u$.

EDIT:

I am aware of the argument meantioned by littleO in the comment bellow. But I want to derive an expression like specified in the last equation, $\nabla f\cdot u =\lambda\nabla g\cdot u$, for an arbitrary direction $u$, through something similar to the third expression $v=u-\frac{\nabla g\cdot u}{\nabla g\cdot\nabla g}\nabla g$.

  • 0
    Your first two equations state the $\nabla f$ is in the orthogonal complement of the null space of the row vector $(\nabla g)^T$. From linear algebra we know the orthogonal complement of the null space of a matrix $A$ is the range of $A^T$. It follows that $\nabla f$ is in the range of $\nabla g$. – 2017-01-27
  • 0
    Yes, I am aware and understand these arguments. I am trying to derive it in a way I set out to, and am really bothered that I am unable to. Where do I make a mistake in my procedure? I want to derive at a form specified in the last equation of my question, through something similar to the third equation. Could you help please? I know there are other ways, but this is the way I set out to do it, and failed, and I must figure it out and clear my confusion (with this specific calculation). – 2017-01-27
  • 1
    your $\lambda$ depends on $u$. You can write $\nabla f \cdot u = \frac{\nabla f \cdot \nabla g}{\nabla g\cdot \nabla g}\nabla g \cdot u$ – 2017-01-27
  • 0
    Can you explicity write out the procedure for me, how to arrive at it? I would be most grateful. (I see how lambda depends on u. But I cannot arrive at your equation.) – 2017-01-27
  • 1
    Do you agree that it shouldn't depend on $u$? I took your expression $\nabla f\cdot u = \lambda \nabla f \cdot \nabla g$ and plugged in your expression for $\lambda = \nabla g\cdot u /\nabla g^2$ – 2017-01-27
  • 0
    I see... your comment and the answer bellow showed just how foolish my mistake was. Thank you both. – 2017-01-27

1 Answers 1

2

The very naming of $\lambda$ is what caused your trouble. Lets take what you have arrived at, and simply substitute $\lambda=\frac{\nabla g\cdot u}{\nabla g\cdot\nabla g}$ (which depends on the choice of $u$). $$\nabla f\cdot u=\frac{\nabla g\cdot u}{\nabla g\cdot\nabla g}\nabla f\cdot\nabla g$$ As you can see, you where already there. Because $$(\nabla g\cdot u)(\nabla f\cdot\nabla g)=(\nabla f\cdot\nabla g)(\nabla g\cdot u)$$ We have $$\nabla f\cdot u=\frac{\nabla f\cdot \nabla g}{\nabla g\cdot\nabla g}\nabla g\cdot u$$ This is the correct quantity to label a scalar, as it does not depend on the choice of $u$ $$\lambda=\frac{\nabla f\cdot \nabla g}{\nabla g\cdot\nabla g}$$ Thus, we arrive at the expression you desired $$\nabla f\cdot u=\lambda\nabla g\cdot u$$