Question on gradient vectors

Question

Let $F(x,y)$ be a function in $R \times R \rightarrow R$.

In this case: $\nabla F = (\frac{\partial}{\partial x} , \frac{\partial}{\partial y})^T$.

Well, let's suppose that $\nabla F$ at some point $(a,b)$ $= (n, -n)^T$. In this case, that would intuitively mean:

• When we add $\epsilon$ to $x$ then $F(x,y)$ increases by n
• When we add $\epsilon$ to $y$ then $F(x,y)$ decreases by n

So, it would intuitively mean that by adding this $\epsilon$ amount, then $F(x+\epsilon,y+\epsilon)$ stays at the same value, right?

However, when we take the magnitude of this gradient vector, we get a different value other than 0. Isn't the value of the magnitude supposed to be 0 to match the intuition (meaning that the function would neither increase nor decrease when we move a little bit along its variables)?

Answer 1

You are confusing the magnitude of the amount you "move/step" in the domain over which the function is defined, with the change in the function itself, so if $F(x,y) = c$ and you consider the set of points $\{ (x,y) \; : \; F(x,y) = c \}$ then moving though these points in a direction such that your new point is still a member of this set then you have certainly moved position, but the function won't have changed in value.

Consider the function $F(x,y) = x^2 - y^2$ at the point $(x_0,y_0) = (1,1)$ then you have gradient $\nabla F = (2,-2)$, now consider an infinitesimal (but non-zero!) step of constant step size $\delta_x = \delta_y = \delta$, then \begin{align*} F(x_0 + \delta,y_0 + \delta) &\approx F(x_0,y_0) + \nabla F(x_0,y_0)\cdot(\delta_x,\delta_y) \\ &\approx F(x_0,y_0) + 2 (\delta - \delta) \\ &= F(x_0,y_0) \end{align*} To summarise there are directions in the space of variables such that a small change in these directions will lead to no net change in a function defined, in general these will be of the form \begin{align*} \xi(x,y) \frac{\partial F}{\partial x} + \eta(x,y)\frac{\partial F}{\partial y} = 0, \end{align*} and sometimes it so happens that these functions (at a certain point) are particularly simple as in your example.