This is more a hint since I unfortunately don't have much time, but I think this should do the job for odd primes.
For a henselian local ring $R$ with $2\in R^*$ and residue field $k$ we have $R^*/R^{2*}\cong k^*/k^{2*}$.
The composition of projections \begin{equation*} R^*\twoheadrightarrow (R/\mathfrak m)^*\cong k^*\twoheadrightarrow k^*/k^{2*} \end{equation*} is surjective. An element $a$ is in the kernel of the composition map if its image $x$ is a square in $k^*$, say $x=y^2$. But then the polynomial $T^2-x\in k[T]$ factors as $(T-y)(T+y)$, which may then be lifted to $T^2-a=(T-b)(T+b)\in R[T]$. (Note that we need char $k\neq 2\Leftrightarrow (T-y)$ and $(T+y)$ are coprime.) Clearly $b^2=a$ and $b$ is a unit, hence $a\in R^{2*}$. We conclude that the above map factors through an isomorphism $R^*/R^{2*}\cong k^*/k^{2*}$.
Therefore the problem can be reduced from $\mathbb Z_p$ to $\mathbb Z/p$. There some version of the quadratic reciprocity law should do.