We have:
\begin{align}
\mathbb Z_2\oplus\mathbb Z_2
&= \mathbb Z_2^{1\rm st}\oplus \mathbb Z_2^{2\rm nd}\\
&= \{0^{1\rm st},1^{1\rm st}\}\oplus\{0^{2\rm nd},1^{2\rm nd}\}\\
&= \{0^{1\rm st}+0^{2\rm nd}, 0^{1\rm st}+1^{2\rm nd}, 1^{1\rm st}+0^{2\rm nd}, 1^{1\rm st}+1^{2\rm nd}\}
\end{align}
Note that at this stage, the "$+$" notation in those elements is a mere convention, that will be justified afterwards. That is, we actually define the elements to be the complete expressions $x^{1\rm st}+y^{2\rm nd}$. Also note that addition is by definition component-wise, that is,
$$(a^{1\rm st}+b^{2\rm nd}) + (c^{1\rm st}+d^{2\rm nd})
= (a+c)^{1\rm st}+(b+d)^{2\rm nd}$$
Now we see that $0^{1\rm st}+0^{2\rm nd}$ is the neutral element, so we just denote it as $0$. Moreover we notice that $\{0^{1\rm st}+x^{2\rm nd}:x\in \mathbb Z_2^{2\rm nd}\}$ is isomorphic to $\mathbb Z_2^{2\rm nd}$, therefore we omit the $0^{1\rm st}$, and analogously for $0^{2\rm nd}$.
Therefore we arrive at
$$\mathbb Z_2\oplus\mathbb Z_2 = \{0, 1^{1\rm st},1^{2\rm nd}, 1^{1\rm st}+1^{2\rm nd}\}$$
Note however that $1^{1\rm st}\ne 1^{2\rm nd}$, therefore you cannot simplify this to $\{0,1,1,1+1\}=\{0,1\}$
Also note that now the "$+$" notation for the elements is justified after the fact: $x^{1\rm st}$, when standing alone, is just a shorthand for the expression $x^{1\rm st} + 0^{2\rm nd}$, and $y^{2\rm nd}$ is just a shorthand for the expression $0^{1\rm st}+y^{2\rm nd}$. But if we interpret $x^{1\rm st}+y^{2\rm nd}$ not as a single expression, but as the sum of the standalone values $x^{1\rm st} + 0^{2\rm nd}$ and $0^{1\rm st}+y^{2\rm nd}$, then we get
$$(x^{1\rm st} + 0^{2\rm nd}) + (0^{1\rm st}+y^{2\rm nd}) =
(x+0)^{1\rm st}+(0+y)^{2\rm nd} = x^{1\rm st}+y^{2\rm nd}$$
so we recover the original expression, which justifies the notation.