Say $$E = f(X) \\ \text{when}\ X \to X+\delta \\\text{where}\ \|\delta\| \to 0\ \text{is a vector}$$ then $$\Delta E \approx f(X) +\delta^T \nabla_X f(X) $$
Is this correct ? Then my question is , by the definition of gradient, the gradient should be the direction which increases your function value the most. But now I am not moving my X along with that direction. Howcome the equation above is correct ?
Recall from a 1-D derivative, the gradient tells you what increase you will get if you "move in this direction that I tell you"
But now I am not moving along with the gradient direction. Why is this still the approximation for $$\Delta E$$
My thought is that the equation should be $$\Delta E = f(X) +\delta^T \nabla_{\delta}f(X)$$
Am I missing something ?