If we knew that a local extremum occurred at $p_1$, we could conclude that the gradient vanishes here (assuming this to be a sufficiently nice function, with continuous first and second order partial derivatives). To establish that $f(p_1)=0$ is an extreme value, we can show that $f$ is either always non-negative or always non-positive.
To do this, suppose we have $p_3,p_4\in\mathbb{R}^2$ with $f(p_3)<0$ and $f(p_4)>0$. We can choose a curve $c\colon[0,1]\to\mathbb{R}^2$ so that $c(0)=p_3$, $c(1)=p_4$, and so that our curve does not pass through $p_1$ or $p_2$. Then $f\circ c\colon[0,1]\to\mathbb{R}$ is a continuous function so that $f(0)<0$ and $f(1)>0$. Can you make a conclusion about $f\circ c$ that violates our assumption about where $f$ vanishes?
Once we obtain our contradiction, we conclude that either $f\geq 0$ on $\mathbb{R}^2$ or $f\leq 0$ on $\mathbb{R}^2$. In either case, $p_1$ and $p_2$ give local extrema of $f$, so the gradient of $f$ is zero at these points.
Can you make any similar statements when we increase the number of points where $f$ vanishes? If $f$ is zero only at $p_1,\ldots,p_n$ for some $n\in\mathbb{N}$, can you still conclude that the gradient vanishes at these points? What if the zero set of $f$ is some more general subset $C\subset\mathbb{R}^2$ of the plane? Are there conditions on this subset that will allow us to conclude that the gradient vanishes at these points?
