Is there any way of a graphic understanding of the non-negative condition in the KKT conditions? Why do I need this only for the inequality constraints?
Best Regards
Gesetzt
KKT Non-Negative-Condition
0
$\begingroup$
optimization
karush-kuhn-tucker
-
0Are you referring to the "dual feasibility" conditions? (using the terminology from [here](https://en.wikipedia.org/wiki/Karush%E2%80%93Kuhn%E2%80%93Tucker_conditions) – 2017-02-27
1 Answers
0
From my understanding, you want why the $\lambda$ has to be non-negative for an inequality constraint. Assuming this is what you want, consider the following first order expansion:
$f(x+d) \approx f(x) + \nabla f^T d$ (by Taylor's theorem), and
$c(x+d) \approx c(x) + \nabla c^T d$ (by Taylor's theorem)
where $c$ is the constraint function (assumed $\geq0$), $f$ is the function you are trying to minimise and $d$ is the direction of descent.
Now, for a feasible direction, $\nabla f^T d \lt0$ (because you want a decrease in the function) and $\nabla c^T d \geq0$ (because you want to stay feasible). Now, when won't one have a feasible $d$?
Only when, $\nabla f(x)= \lambda*\nabla c(x)$ where $\lambda \geq0$. Pictorially, the following figure may help you: courtesy of Numerical Optimisation by Nocedal and Wright
Now what happens if $\lambda \lt0?$ You have the whole open half space as feasible and you don't reach a solution.