I am trying to show that if a continuous function $f$ is defined on an open interval $J$ and $f$ is nonzero for some $p \in J$ that there exists a closed interval about $p$ on which f is nonzero. For definiteness, assume $f(p) > 0$. I want to argue the following: simply choose a sufficiently small $\epsilon$ such that $V := (f(p) - \varepsilon , f(p) + \varepsilon)$ doesn't contain $0$. Then continuity guarantees that one can find an open interval $U := (p - \delta, p + \delta)$ such that $f(U) \subset V$. Now, $f$ is nonzero on on $U$, and I believe it is a true statement that $f$ is also nonzero on $[p - \delta, p + \delta]$ but I am unsure of how to formally justify this last part.
So, my question is, Is the basic thrust of my argument correct and how can one formally justify that if $f$ is nonzero on $U$ then $f$ is also nonzero on what is effectively the closure of $U$?