You can compute this manually if you want:
$\left[\begin{array}{cc} a & b \\c & d \end{array}\right]^2=\left[\begin{array}{cc} a^2+bc & b(a+d) \\c(a+d) & bc+d^2 \end{array}\right]$
To get the identity matrix, either $a=-d$, so $a^2+bc=1$ (and these can be picked freely, leaving plenty of options) or $b=0$ and $c=0$, so $a=\pm1, d=\pm 1$.
More conceptually, you're asking this: "What linear transformation, applied twice, brings you back to where you started?" You could swap the $x$ and $y$ axes:
$\left[\begin{array}{cc} 0 & 1 \\1 & 0 \end{array}\right]$
flip the space around the $x$ axis:
$\left[\begin{array}{cc} -1 & 0 \\0 & 1 \end{array}\right]$
Or a number of other things! Just think of a transformation that is undone by applying it again, and find the matrix that corresponds to it.