Here the function is $f(x,y)=\frac{y^2}{4}+x^2$. How do we find the direction of a function in which the function neither decreases nor increases at $(x,y)=(1,2)$?
I suppose we start with first finding the gradient of the function. $\nabla f=(2x,\frac12 y)=(2,1)$. I think the direction orthogonal to this gradient would achieve the purpose. But how do we derive such vector?
You are right. The directional derivative at $\mathbf{p}=(1,2)$ in the direction of a vector $\vec v=(a,b)$ is $ \nabla_{\vec v}f(\mathbf{p})=\nabla f(\mathbf{p})\cdot \vec v$, so we have: $$ \nabla_{\vec v}f(\mathbf{p})=(2,1)\cdot(a,b)=2a+b $$ that is $0$ iff $$ 2a+b=0 \quad \iff \quad b=-2a $$ that is iff the vector $\vec v$ is orthogonal to the gradient. You can chose $\vec v=(1,-2)$