There is a slightly more general version of this result: if a finite field has an odd number $q$ of elements, then the equation $x^2+1=0$ has a solution in the field if and only if $q\equiv1\pmod4$. In your case you can of course take $\mathbb Z/p\mathbb Z$ as finite field, in which case $q=p$.
Here's a simple proof. Group all nonzero elements of your field in packets $\{a,-a,a^{-1},-a^{-1}\}$. It is easy to see that if one would have generated the packet from any of its three elements other than $a$, it would still give the same packet. In other words one has partitioned the $q-1$ nonzero elements into packets. Not all packets have four distinct elements though: while it cannot happen that $a=-a$ (since $q$ is odd), it can happen either that $a=a^{-1}$ or that $a=-a^{-1}$; in both cases the packet has two elements instead of four.
Now $a=a^{-1}$ happens if and only if $a^2=1$, and since $a^2-1=(a-1)(a+1)$, this occurs precisely for $a=1$ and $a=-1$, giving one packet of this type, regardless of the field. For the other type $a=-a^{-1}$, this happens if and only if $a^2=-1$. That need not occur at all, but if it does, there are precisely two solutions to this quadratic equation, which then together define a single packet of this second type. Now since all remaining packets are of size $4$, and all packet sizes add up to $q-1$, one easily sees that a packet of the second type exists if and only if $q-1$ is a multiple of $4$.