You start with the joint PDF, as you have done $ f : \mathbb{R}_{\geq 0}^3 \rightarrow \mathbb{R} : \mathbf{x} \rightarrow \alpha^3 e^{-\alpha(x_1+x_2+x_3)} $
Then you pick an appropriate (invertible!) transform $T$, also as you have done, and find its inverse $ \begin{eqnarray} T(\mathbf{x}) &=& \begin{pmatrix} -1 & 1 & 0 \\ -1 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix}\;\mathbf{x} &\text{,}& T^{-1}(\mathbf{x}) &=& \begin{pmatrix} -1 & 0 & 1 \\ 0 & 0 & 1 \\ -1 & 1 & 1 \end{pmatrix}\;\mathbf{x} \text{.} \end{eqnarray} $
The transformed PDF $g$ (i.e. $g$ such that $g(T(\mathbf{x})) = f(\mathbf{x})$) is thus $ g(\mathbf{y}) = f(T^{-1}(\mathbf{y}))\frac{1}{\det(T)} = \alpha^3 e^{-\alpha(3y_3 + y_2 - 2y_1)} $
Now we have to figure out $\text{dom}\, g \subset \mathbb{R}^3$ such that $T^{-1}(\text{dom}\, g) = \text{dom}\, f$. Requiring $T^{-1}(\mathbf{y}) \in \mathbb{R}_{\geq 0}^3$ for all $y \in \text{dom}\, g$ produces the conditions $ \begin{eqnarray} y_3 - y_1 &\geq& 0 \text{,}\\ y_3 &\geq& 0 \text{,}\\ y_3 + y_2 - y_1 &\geq& 0 \text{.} \end{eqnarray} $ This yields $y_3 \geq \max\{0,y_1, y_1 - y_2\}$ as necessary and sufficient, i.e. you get $ \text{dom}\, g = \left\{\mathbf{y} \,:\, \mathbf{y} \in \mathbb{R}^3, y_3 \geq \max\{0,y_1, y_1 - y_2\}\right\} \text{.} $
To find the PDF $h$ of $(X_2-X_1,X_3-X_1)=(Y_1,Y_2)$ you must now integrate $g$ over $y_3$, but without leaving the domain of $g$, i.e. compute $ \begin{eqnarray} h(y_1,y_2) &=& \int_{c}^\infty \alpha^3 e^{-\alpha(3y_3 + y_2 - 2y_1)} dy_3 \:\text{where}\: c = \max\{0,y_1, y_1 - y_2\} \\ &=& \frac{\alpha^2}{3} e^{-\alpha(y_2 - 2y_1)} \int_{c}^\infty 3\alpha e^{-3\alpha y_3} dy_3 \\ &=& \frac{\alpha^2}{3} e^{-\alpha(y_2 - 2y_1)} e^{-3\alpha c} \\ &=& \frac{\alpha^2}{3} e^{-\alpha(3c - 2y_1 + y_2)} \end{eqnarray} $
Note that if $y_1,y_2\geq 0$ then $c=y_1$, if $y_1 \leq 0,y_1 \leq y_2$ then $c=0$ and if $y_2 \leq 0,y_2 \leq y_1$ then $c=y_1-y_2$. This gives $ h(y_1,y_2) = \frac{\alpha^2}{3}\begin{cases} e^{-\alpha(y_1 + y_2)} & \text{if } y_1,y_2 \geq 0 \\ e^{-\alpha(2|y_1| + y_2)} & \text{if } y_1 \leq 0, y_1 \leq y_2 \\ e^{-\alpha(y_1 + 2|y_2|)} & \text{if } y_2 \leq 0, y_2 \leq y_1 \end{cases} $