0
$\begingroup$

We want to find the zero of a function, and Newton method allow us to do it : Newton Method on Wikipedia. But I don't understand how it work. The explanation goes as following : Let $x_0$ (with good conditions). Then, around $x_0$, we have that $$f(x)\approx f(x_0)+f'(x_0)(x-x_0).$$ Then, as I understand, the say : $$f(x)=0\implies f(x_0)+f'(x_0)(x-x_0)=0\implies x=x_0-\frac{f(x_0)}{f'(x_0)}.$$

The thing is $f(x)=f(x_0)+f'(x_0)(x-x_0)$ only when $x$ is very close of $x_0$, it has no sense to say that $$f(x)=0\implies f(x_0)+f'(x_0)(x-x_0)=0 \ \ !$$

Indeed, for example around $1$, we have that $x^2\approx 2x-1$ when $x=1$, but $x^2=0\not\Rightarrow 2x-1=0$. So, what are they doing here ?

Anyways, let's continue. They says that the solution of $f'(x_0)(x-x_0)+f(x_0)$ is a better approximation of the zero of $f(x)$, but why ? How is it possible ? And to finish, if we continue, let $x_1=x_0-\frac{f(x_0)}{f'(x_0)}$. Then, $f(x)\approx f(x_1)+f'(x_1)(x-x_1)$, and thus $$f(x)=0\implies f(x_1)+f'(x_1)(x-x_1)=0\implies x=x_1-\frac{f(x_1)}{f'(x_1)},$$ so, let $x_2=x_1-\frac{f(x_1)}{f'(x_1)}$, then $$f(x)\approx f(x_2)+f'(x_2)(x-x_2).$$

we continue, and we arrive to $x_{k+1}=x_k-\frac{f(x_k)}{f'(x_k)}$, and thus should converge to the zero of $f$. Why ? i.e. we should have that $$\lim_{k\to \infty }f\left(x_k-\frac{f(x_k)}{f'(x_k)}\right)=0,$$ but I don't understand why. This algorithm doesn't has sense to me. Thanks for your explanations.

  • 0
    Have you tried $f(x) = x^3 - x$ starting from $x = 1/\sqrt 5$? – 2017-02-03

3 Answers 3

2

The idea behind the method is geometrical in nature. You look at some point $x_0$ and hope it is some approximation for a zero of $f$. Instead of looking now at $f(x_0)$, you look at the tangent line to the graph of $f$ in $(x_0, f(x_0)) $. This is a linear function (for which it is easy to find a root), and the idea of the method is that the zero of this linear function is closer to the root of $f$ then $x_0$. On Wikipedia (your link) you see a picture which tries to illustrate this idea. Now you iterate.

It will, of course, only work if $f$ satisfies additional assumption (you mentioned this, as well), and of course these assumptions have to be used in order to prove the result.

0

If $x_0$ is close enough to the root, then $x_1:=x_0-\dfrac{f(x_0)}{f'(x_0)}$ is closer to the root.

The reason is that starting from the root approximation, you consider that the curve is approximately equal to its tangent, and this allows you to find a better estimate by extrapolation.

If the curve was linear, the extrapolation would indeed give the exact root. For a plain curve, the new approximation is in general much better.

enter image description here

Side note:

In the secant method, rather than estimating the curve from a point and a slope, you estimate it from two known points, also using a linear model.

In the Brent-Dekker method, they use three points and estimate the curve as a parabola through these points to get a yet better estimate of the root.

You could think of an improved Newton method where you use two points and two slopes, and estimate the curve with a cubic (by means of Hermite interpolation).

  • 1
    Agreed, comment deleted. – 2017-02-03
0

The function that you iterate when doing Newton's method is generally a contraction. A contraction is a mapping from some space with a concept of distance (in this case, the real number line) back into the same space, which moves every pair of points closer together. (For a function from an interval of the real numbers back into that same interval, having a slope with absolute value less than 1 makes it contractive.)

You can prove (and can easily visualize) that a contractive function F mapping some interval on the number line back to itself will have one and only one fixed point - a point for which $F(x) = x$, and that iterating F again and again will "suck" all points into that fixed point.

In the case of Newton's method, most functions f will lead to a Newton's method function $F(x) = x - f(x) / f'(x)$ that is contractive in some interval around the zero of f. That's why repeating the process leads you to the zero of f.