The intention of this answer is to show you that trying all possibilities is in fact not that long. (Of course, it si more elegant, when you find a solution which avoids trying all possibilities.) Let me start by plotting 5x5 table with all possibilites for the remainders of a and b. (I do not know of a good way of making tables here - I tried something anyway.)
$ \begin{array}{c|ccccc} b \backslash a & 0 & 1 & 2 & 3 & 4 \\ \hline 0 & & & & & \\ 1 & & & & & \\ 2 & & & & & \\ 3 & & & & & \\ 4 & & & & & \\ \end{array} $
If we rewrite our expression as $ab(a-b)(a+b)(a^2+b^2)$, we see that all possibilities where $a=0$ or $b=0$ are ok (marked by $\circ$).
$ \begin{array}{c|ccccc} b \backslash a & 0 & 1 & 2 & 3 & 4 \\ \hline 0 & \circ & \circ & \circ & \circ & \circ \\ 1 & \circ & & & & \\ 2 & \circ & & & & \\ 3 & \circ & & & & \\ 4 & \circ & & & & \\ \end{array} $
Also possibilities where a=b are ok (since $a-b\equiv 0\pmod 5$ and so are those where $a=5-b$ (since $a+b\equiv 0\pmod 5$), hence we can omit both diagonals (marked by $\bullet$).
$ \begin{array}{c|ccccc} b \backslash a & 0 & 1 & 2 & 3 & 4 \\ \hline 0 & \circ & \circ & \circ & \circ & \circ \\ 1 & \circ & \bullet & & & \bullet \\ 2 & \circ & & \bullet & \bullet & \\ 3 & \circ & & \bullet & \bullet & \\ 4 & \circ & \bullet & & & \bullet \\ \end{array} $
There are only 8 possibilities left, and since the roles of a and b are symmetric, we only have to try: (1,2), (1,3), (4,2), (4,3). In all these cases $a^2+b^2\equiv 0 \pmod 5$.