Lets consider the example of two sets, $X_2$ and $Y_2$, each of which have two elements which satisfy $x_1\geq x_2\geq 0, y_1\geq y_2\geq 0$. Then $d=\sum_{k=1}^{2}\left|x_{i_{k}}-y_{j_{k}}\right| = |x_{i_1}-y_{j_1}|+|x_{i_2}-y_{j_2}|$ and to determine all possible values we have only two cases to check, one where $i_k=j_k, k\in \{1,2\}$ and the other in which $i_k\neq j_k, k\in \{1,2\}$. In the first case, $d = |x_1 - y_1| + |x_2 - y_2|$, and in the second case $d = |x_1 - y_2| + |x_2 - y_1|$, which we break down into two further cases: $x_1 \geq y_1$ and $x_1\leq y_1$. In the first case $|x_1 - y_2| + |x_2 - y_1| = |x_1 - y_1| + |y_1-y_2| + |x_2 - y_1|\geq |x_1 - y_1|+|x_2-y_2|$ while in the second we have $x_2\leq x_1\leq y_1$ so $|x_1 - y_2| + |x_2 - y_1| = |x_1 - y_2| + |x_1 - y_1| + |x_2 - x_1|\geq |x_1 - y_1|+|x_2-y_2|$ with the inequality in both cases coming from the triangle inequality. Certainly, this method of case-by-case analysis is too laborious to use directly for your problem. However, the case of two elements turns out to be the crucial one in order to apply induction, which proves the theorem for any number of elements. The key observation is that the above proof can be modified to show that $d=\sum_{k=1}^{n}\left|x_{i_{k}}-y_{j_{k}}\right| = |x_{i_1}-y_{j_1}|+\sum_{k=2}^{n}\left|x_{i_k}-y_{j_k}\right|$ is at least the result of using some permutation such that $i_1=j_1$. I leave the details of this to you.