Need to show that, $f : \mathbf{R}^2 \rightarrow \mathbf{R}^2 $ with $f(x) = \frac{x}{x_1 + x_2 + 1}$ with $dom f = \left \{ \left( \begin{matrix} x_1 \\ x_2 \end{matrix} \right) |x_1 + x_2 +1 > 0 \right\}$ maps convex sets to convex set.
What approach should I take to solve this problem. Any hint is appreciated.
The function that you have is known as a linear-fractional function [1]. More generally written as, $$f(x) = \frac{Ax + b}{c^{\top}x + d}\;\;\;\;\;\textrm{dom} f = \{x | c^{\top} x + d > 0\}$$
It can be shown that this entire class of functions map convex sets to convex sets.
Case 1: Prove if $C \in \textrm{dom} f$ is a convex set, $f(C)$ is convex set.
Let $x_1, x_2 \in C$. We know that $\lambda x_1 + (1-\lambda)x_2 \in C, \forall\lambda \in [0,1]$. Similarly, if we can show that $\lambda_ff(x_1) + (1 - \lambda_f)f(x_2) \in f(C)$, then that will show that $f(C)$ is a convex set.
One way to do it is to find an explicit map between $\lambda_f$ and $\lambda$, which we can find in this case: $$\lambda_f = \frac{\lambda (c^{\top}x_1 + d)}{\lambda (c^{\top}x_1 + d) + (1 - \lambda) (c^{\top}x_2 + d)}$$
Therefore, if we substitute this and simplify we would get, $$\lambda_ff(x_1) + (1 - \lambda_f)f(x_2) = \frac{A(\lambda x_1 + (1-\lambda)x_2) + b}{c^{\top}(\lambda x_1 + (1-\lambda)x_2) + d} \in f(C)$$
Case 2: Prove that $I \in \textrm{Range}(f)$ is a convex set, then $f^{-1}(I)$ is a convex set.
Essentially, in this case, you just have to work the proof of case 1 backwards. I will leave this for you to show.
[1] Slide 14 in this pdf