Convex Optimization

Question

Prove that the set $\{Ax: x\geq 0\in\mathbb{R}^n\}$ is closed and convex cone.

I have no problem with proving that the set is convex, the issue comes when I try to show that it is closed. I am stuck.

Answer 1

I assume that $A$ is an $m \times n$ matrix.

Let $P$ be the positive cone in $\mathbb{R}^m$, i.e. $P = \{x \in \mathbb{R}^m : x_i \ge 0 \text{ for } i = 1,2,\ldots,m\}$. Then $P$ is closed, since it is the intersection of a finite number of closed half-spaces.

Define a mapping $f: \mathbb{R}^n \to \mathbb{R}^m$ by $f(x) = Ax$. The set you're interested in is $f^{-1}(P)$, I think. Since $P$ is closed and $f$ is continuous, $f^{-1}(P)$ is also closed.

Answer 2

Here is one way:

Let $\operatorname{cone} S = \{ \sum_k \lambda_k s_k | \lambda_k \ge 0, s_k \in S \}$, where the sums are over finite sets of indices. It is straightforward to check that $\operatorname{cone} S$ is a convex cone.

The problem is to show that $\operatorname{cone} \{ A e_1,\cdots, A e_n \}$ is closed.

In general, even if $S \subset \mathbb{R}^n$ is closed (or compact), the set $\operatorname{cone} S$ in not necessarily closed. For example we have $\operatorname{cone} \overline{B}(e_1,1) = \{x | x_1 >0 \} \cup \{0\}$ which is not closed.

If $S$ is finite, however, $\operatorname{cone} S$ is closed. The following proof is inspired by Lemma 2.6 in Appendix in Dem'yanov & Malozemov's "Introduction to minimax".

Let $K = \operatorname{cone} \{b_1,...,b_p \}$.

First suppose the $b_k$ are linearly independent, then if we let $B= \begin{bmatrix} b_1 \cdots b_p \end{bmatrix}$, there is some $c>0$ such that $\|Bx\| \ge c \|x\|$.

Suppose $y_k \in K$ such that $y_k \to y$. We have $y_k = B x_k$ for some $x_k \ge 0$ and since the $y_k$ are bounded, we see that the $x_k$ are bounded and hence by taking a subsequence and renumbering as appropriate, we have $x_k \to x$ for some $x \ge 0$. From continuity, we have $y=Bx$ and hence $y \in K$.

To deal with the general case, we need a technical result which is essentially a version of Carathéodory's theorem for cones. Suppose $x \in K \setminus \{0\}$ and $x= \sum_{k \in I} x_k b_k$ such that $x_k >0$ for $k \in I$. If the $\{b_k\}_{k \in I}$ are linearly dependent, there is some $\alpha \neq 0$ such that $\sum_{k \in I} \alpha_k b_k = 0$. Hence we can choose some $\mu$ such that $x_k + \mu \alpha_k \ge 0$ for $k \in I$ and $x_{k_0} + \mu \alpha_{k_0} = 0$ for at least one ${k_0} \in I$. Let $I'$ be the subset of indices in $I$ that are strictly positive. (Since $x \neq 0$, we know that the resulting index set is non empty.) We can repeat this procedure until the remaining $b_k$ are linearly independent.

Now let ${\cal I}$ be the collection of index sets $I$ such that the $\{b_k\}_{k \in I}$ are linearly independent and note that ${\cal I}$ is finite. We see that $K = \cup_{I \in {\cal I}} \operatorname{cone} \{b_k\}_{k \in I}$ which is the union of a finite number of closed sets, hence closed.