My proof. Let $\varepsilon >0$. Pick $\delta=5\varepsilon$.
If $\left| x-4\right| < \delta$ for $x\in\mathbb{R} ^{\geq 0}$ then $\left| \sqrt {x}-2\right| =\dfrac {\left| x-2\right| } {\left| \sqrt {x}+2\right| }=\dfrac {\left| x-2\right| } {\left| \sqrt {x}\right| +2}=\delta/5\leq \dfrac {5\varepsilon } {\varepsilon }=\varepsilon$.
Note that for $0 Can you check my proof?