1
$\begingroup$

Suppose I have the problem of finding the minimum of some function $f(x,y)$ subject to three inequality constraints $g_1(x,y),g_2(x,y),g_3(x,y) \leq 0$. Further assume the functions $f,g_2,g_3$ to be convex on $\mathbf{R}^2$ ($g_1$ is not convex).

A theorem in my book says that if I find a KKT-point where $g_1$ is inactive (the corresponging Lagrange multiplier is zero), then I know this point to be a local minimum. (Under assumptions that $f,g_2,g_3$ are convex in the neighbourhood of this point).

A corollary to the same theorem says that if all of $f,g_1,g_2,g_3$ are convex, then the local minimum is in fact the global minimum.

Now to what confuses me: In the solution to the above described problem, the found KKT-point (with $g_1$ being inactive) is said to be the global minimum as a cause of the above theorem and corollary. Since $g_1$ is not convex, I cannot see how this works.

1 Answers 1

0

You really don't even need KKT. Let's consider two problems: the original, \begin{array}{lll} P_1: & \text{minimize} & f(x,y) \\ & \text{subject to} & g_1(x,y) \leq 0 \\ & & g_2(x,y) \leq 0 \\ & & g_3(x,y) \leq 0 \end{array} and the same model with $g_1$ completely removed: \begin{array}{lll} P_2: & \text{minimize} & f(x,y) \\ & \text{subject to} & g_2(x,y) \leq 0 \\ & & g_3(x,y) \leq 0 \end{array} Let $f_1^*$ and $f_2^*$ be the optimal values $P_1$ and $P_2$, respectively (including by convention $+\infty$ if a model is infeasible, and $-\infty$ if it is unbounded). Clearly, it must be true that $f_1^*\geq f_2^*$. After all, $P_2$ is a relaxation of $P_1$; it has fewer constraints. So $P_2$ is certainly going to have an objective that's less than or equal to that of $P_1$.

Now suppose $(x^*,y^*)$ is an optimal point for $P_2$. That means $$f(x^*,y^*)=f_2^*, \quad g_2(x^*,y^*)\leq 0, \quad g_3(x^*,y^*)\leq 0$$ Further suppose it happens to be the case that $g_1(x^*,y^*)\leq 0$. Then $(x^*,y^*)$ is a feasible point of $P_1$ as well! Therefore, $$f(x^*,y^*)\geq f_1^*.$$ But now we have $$f_1^*\geq f_2^* = f(x^*,y^*) \geq f_1^* \quad\Longrightarrow\quad f_1^*=f_2^*.$$ So if $g_1(x^*,y^*)\leq 0$, that means the optimal point of $P_2$ is also a globally optimal point of $P_1$.

Note that it doesn't matter if $g_1$ is convex. In fact, we haven't used convexity at all here. The above argument holds even if $f$, $g_2$, and $g_3$ are all non-convex. The general principle is this:

Any point that is a) optimal for the relaxed problem and b) feasible for the original problem is also c) optimal for the original problem.

Let's bring KKT back into it---and this means we have to bring convexity into it. You said that you found a KKT point $(x,y)$ where $g_1$ is inactive---its Lagrange multiplier $\lambda_1$ is zero. That means you could remove the term $\lambda_1 g_1(x,y)$ from the Lagrangian altogether without modifying its value or the other KKT conditions. You're left with the Lagrangian for $P_2$, with its KKT conditions satisfied. This establishes that the point is a local minimum for $P_2$; and since that problem is convex, it is global as well.