3
$\begingroup$

I am looking for a simple direct proof of the fact that \frac{\frac{f(x + \Delta x) - f(x)}{\Delta x} -f'(x)}{\Delta x} \stackrel{\Delta x \to 0}{\to} \frac{1}{2}f''(x), or, equivalently, f(x+\Delta x) = f(x) + f'(x)\Delta x + \frac{1}{2}f''(x)\Delta x^2 + o(\Delta x^2).

holds for a twice-differentiable $f(x)$.

I remember there were times when I could derive this directly from the following definition of a derivative: f(x+\Delta x) = f(x) + f'(x)\Delta x + o(\Delta x) in a couple of simple lines.

A long time has passed since then and now I need to either recollect this magical "obvious" proof of mine or find out I was wrong then and the actual proof is more involved.

2 Answers 2

3

The limit (in what follows I let $h=\Delta x$ for sake of simplicity): $\lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h}$ comes in the indeterminate $0/0$ form.

If you're allowed to apply de l'Hôpital's rule, you can write: $\begin{split} \lim_{h\to 0} \frac{\frac{f(x+h)-f(x)}{h} -f^\prime (x)}{h} &= \lim_{h\to 0} \frac{f(x+h)-f(x) - f^\prime (x)\ h}{h^2}\\ & \stackrel{H}{=} \lim_{h\to 0} \frac{f^\prime (x+h) -f^\prime (x)}{2h} \quad \text{(here you took derivatives w.r.t. } h \text{)}\\ & = \frac{1}{2}\ f^{\prime \prime} (x) \end{split}$ therefore: $f(x+h)=f(x)+f^\prime (x)\ h+\frac{1}{2}\ f^{\prime \prime} (x)\ h^2 +\text{o}(h^2)\; .$

  • 2
    Thanks, this is correct indeed, but I would like to find a proof which does not use the l'Hospitals rule. I mean, something which would really only require the direct definition of the differentiation. The point is, I actually need to prove this result for a multidimensional case (but I hope that if I see the one-dimensional proof, I'll be able to easily generalize it).2012-03-08
1

After some tinkering I've come up with a proof which is still not the one I am hoping to find (if there is one at all), but it is closer to what I need in the sense that it is obviously generalizable to the multidimensional case, it is reasonably short, and it (kind-of) only relies on the very basic definitions of the gradient/derivative.

We start by defining a function $g(\mathbf{h}): \mathbb{R}^n \to \mathbb{R}$ as follows:

$ g(\mathbf{h}) = f(\mathbf{x} + \mathbf{h}) - f(\mathbf{x}) - \nabla f(\mathbf{x})^T\mathbf{h} - \frac{1}{2}\mathbf{h}^T\nabla^2f(\mathbf{x})\mathbf{h} $

It now remains to show now that $g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert^2)$.

Note that $g(\mathbf{0}) = 0$, $\nabla g(\mathbf{0}) = \mathbf{0}_n$ and $\nabla^2 g(\mathbf{0}) = \mathbf{0}_{n\times n}$. Then $ \frac{\nabla g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{\nabla g(\mathbf{h}) - \nabla g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla^2 g(\mathbf{0}) = \mathbf{0}, $ and consequently $\nabla g(\mathbf{h})$ is $o(\Vert\mathbf{h}\Vert)$. Next, repeating the same step for $g(\mathbf{h})$ we have that $ \frac{g(\mathbf{h})}{\Vert\mathbf{h}\Vert} = \frac{g(\mathbf{h}) - g(\mathbf{0})}{\Vert\mathbf{h}\Vert} \to \nabla g(\mathbf{h}) = o(\Vert\mathbf{h}\Vert), $ which implies that $g(\mathbf{h})$ is indeed $o(\Vert \mathbf{h} \Vert^2)$. Substituting $o(\Vert \mathbf{h} \Vert^2)$ instead of $g(\mathbf{h})$ into the first equation, we obtain the necessary result.

Now this is still not simple enough, to my mind, and there should be something better and "more obvious".