Say $E = f(X) \\ \text{when}\ X \to X+\delta \\\text{where}\ \|\delta\| \to 0\ \text{is a vector}$ then $\Delta E \approx f(X) +\delta^T \nabla_X f(X) $
Is this correct ? Then my question is , by the definition of gradient, the gradient should be the direction which increases your function value the most. But now I am not moving my X along with that direction. Howcome the equation above is correct ?
Recall from a 1-D derivative, the gradient tells you what increase you will get if you "move in this direction that I tell you"
But now I am not moving along with the gradient direction. Why is this still the approximation for $\Delta E$
My thought is that the equation should be $\Delta E = f(X) +\delta^T \nabla_{\delta}f(X)$
Am I missing something ?