1
$\begingroup$

I have confusion regarding Lagrange's multiplier. I was referring to this wiki article http://en.wikipedia.org/wiki/Lagrange_multiplier.

It says that if I have the two contours of the original function $f(x,y)$ to be minimized and the constraint $g(x,y)$ then the gradient of $f(x,y)$ and gradient of $g(x,y)$ are parallel when the contour lines of $f$ and $g$ touch and the tangent vectors of the contour lines are parallel.

And it gives the condition

$\nabla_{x,y}f = -\lambda\cdot \nabla_{x,y}g$

where did this condition come from? I mean they have placed $\lambda$ to make the values equal. But how come we have the negative sign? I didn't get this equation.

Also how did this condition lead to the following

$\mathcal L(x,y,\lambda) = f(x,y) + \lambda \cdot g(x,y)$

Also I have one more question, the wiki article says that we can choose the value of $x$,$y$ such that the contour of $g$ and $f$ touch each other tangentially. They have given an example in the figure at the top right showing it, for maximization. But I am not sure why they need to touch tangentially what if they cross each other can we take that point. For example consider the same figure lets say I want to minimize the function with the constraints. If I take the point when the two contours intersect which is for value $d_2$ as given for the wikipedia figure, definitely I can take this as $x$,$y$ even though they don't touch tangentially. I am a bit confused. So can anyone please explain?

  • 0
    You can simply redefine your constant variable in the equation so that $\lambda' = -\lambda$. This condition is the *definition* of being parallel.2012-06-24
  • 0
    $L$ is the function you now "want to maximize"... take the derivative (gradient) and set it equal to zero, and you get the original condition (i.e. the original condition is the extremum for the function $L$).2012-06-24
  • 0
    Can you give me some resources from where I can see the derivation for the condition or definition of being parallel which you are referring?2012-06-24
  • 0
    Vectors originate from the origin, so if $v$ and $w$ are parallel then $v=\lambda w$ (or they are translates of each other if you ignore the requirement that they have the same origin). Resource: any textbook on vectors.2012-06-24
  • 0
    @user31820: I really need to ask you this: what's your definition of "parallel"?2012-06-24
  • 0
    The optimality condition is that at an extremum, the gradients $\nabla f(x)$ and $\nabla g(x)$ are collinear, assuming that $\nabla g(x) \neq 0$. The $\lambda$ above is just a way of expressing this. The Lagrangean function $L$ has an interpretation in terms of the set $\{(g(x),f(x))\}_{x \in \mathbb{R}^n}$, but you may think of it as a convenient way of expressing the optimality condition.2012-06-24
  • 0
    a good resource is the book by sandaram, http://books.google.de/books/about/A_First_Course_in_Optimization_Theory.html?id=yAfug81P-8YC&redir_esc=y2012-06-24

0 Answers 0