Taylor Series Based Numerical Method for Non-Linear Equations

Question

There is a numerical method for solving non-linear equations which uses the Taylor Series:

$$f(x_n+e_n)=f(x_n)+f'(x_n)e_n+\cdots+\frac{f^{(k)}(x_n)}{k!} e_n^k+\cdots+=0$$

The solution is then found through the iteration:

$$x_{n+1}=x_n+e_n$$

For $k=1$ we have:

$$e_{n1}=-\frac{f_{n}}{f'_{n}}$$

For $k=2$ we have:

$$e_{n2}=-\frac{f_{n}}{f'_{n}+\frac{f''_{n}}{2!}e_{n1}}$$

and so on...

My questions are:

1. What is the name of that method?

2. Solving for $k=2$ we get:

$$e_{n2}=-\frac{f_{n}}{f'_{n}+\frac{f''_{n}}{2!}e_{n2}}$$

so why we set $e_{n2}=e_{n1}$?

Answer 1

Halley's method is used for computing square roots or $n$th roots. If you used $e_{n2}$ in the denominator instead of $e_{n1},$ there would be a square root in each iteration, defeating the purpose of using the method. However, Halley preferred the method with $e_{n2}$ in the denominator (called Halley's irrational formula) for computing cube roots (see Halley's method). For a cube root, the "irrational formula" iteration for finding the cube root of $a$ is $$ x_{n+1} = x_n/2 + \sqrt{12ax_n - 3x_n^4}/(6x_n) $$ whereas the standard Halley's method iteration is $$ x_{n+1} = (x_n^4 + 2ax_n) / (2x_n^3 + a). $$ I have compared these methods on a computer for computing the cube root of 2 and the "irrational formula" does have a somewhat faster convergence rate. But the total time on a computer is slower because of the square root at each iteration.

Answer 2

The Householder (?) methods are a generalization of Newton's and Halley's method with the closed formula $$ x_{n+1} = x_n+k\frac{(1/f)^{(k-1)}(x_n)}{(1/f)^{(k)}(x_n)} \text{ i.e., } e_{n,k}=k\frac{(1/f)^{(k-1)}(x_n)}{(1/f)^{(k)}(x_n)} $$

Let the local power series of $f$ and $g=1/f$ be $f(x+e)=\sum a_ke^k$ and $g(x+e)=\sum b_ke^k$. The product of both has to be the constant $1$. If the closest singularity of $g$, i.e., the closest root of $f$ as complex function, is real (and simple, and the following limit exists etc.), then it is located at $x_+=x+e$ with $$ e=\lim_{k\to\infty}\frac{b_{k-1}}{b_k}=\lim_{k\to\infty}k\frac{g^{(k-1)}(x)}{g^{(k)}(x)} $$ Now introduce the $e_k=\frac{b_{k-1}}{b_k}$ so that for $j1$ the Cauchy product formula reads as \begin{align} 0&=a_0b_k+a_1b_{k-1}+a_2b_{k-2}+…+a_kb_0 \\ &=a_0b_k+(a_1+a_2e_{k-1}+a_3e_{k-1}e_{k-2}+…+a_ke_{k-1}…e_1)b_{k-1} \end{align} so that $$ e_k=\frac{b_{k-1}}{b_k}=-\frac{a_0}{a_1+e_{k-1}(a_2+e_{k-2}(a_3+…))} =-\frac{f}{f'+e_{k-1}(\frac{f''}{2!}+e_{k-2}(\frac{f'''}{3!}+…(e_1\frac{f^{(k)}}{k!})))} $$

Answer 3

I will try to address the "why $e_{n1}$ instead of $e_{n2}$" issue mentioned by OP. The method you seem to start with is to Taylor expand $f$ to $k$'th order

$$T_k[f](x_n +e_n) = f(x_n) + f'(x_n)e_n + f''(x_n)\frac{e_n^2}{2!} + \ldots + \frac{f^{(k)}(x_n)}{k!}e_n^k$$

and then solve for $e_n$ such that $T_k[f](x_n+e_n) \approx f(x_n + e_n) = 0$ and use $x_{n+1} = x_n + e_n$ as an itteration to find a zero of $f$. This is a perfectly valid starting point for constructing a root-finding method, however doing it directly like this has some issues that makes it not very useful in general. To see why this is so, let's go through the first few values of $k$ and see what it gives us.

If we do this for $k=1$ then we obtain good old Newton's method

$$e_n = - \frac{f(x_n)}{f'(x_n)} \implies x_{n+1} = x_n - \frac{f(x_n)}{f'(x_n)}$$

If we do this for $k=2$ we obtain the equation $$e_n = -\frac{f(x_n)}{f'(x_n) + \frac{f''(x_n)}{2}e_n}\tag{1}$$ which can be solved for $e_n$ to give

$$e_n = \frac{-f'(x_n) \pm \sqrt{f'^2(x_n) - 4f(x_n)f''(x_n)}}{f''(x_n)}$$ Here we see a possible problem: if $f'^2(x_n) - 4f(x_n)f''(x_n) < 0$ then the iteration would break down as we will get imaginary numbers: the equation has no solution in the reals (which could happen for all even $k$). This makes this method of limited use as this would happen for quite a large range of functions. If we instead take $e_n$ on the right hand side of $(1)$ to be the result for $k=1$, i.e. $e_n = -f(x_n)/f'(x_n)$, then we obtain Halleys method

$$e_n = -\frac{f(x_n)}{f'(x_n) - \frac{f''(x_n)f(x_n)}{2f'(x_n)}}$$

which is a good method as long as $f''$ exist and it's not to hard/expensive to compute it.

For higher $k$ we run into the problem of having to solve a complicated polynomial equation. This is formally possible for $k=3$ and $k=4$, but this leads to very complicated expressions and for $k>4$ we would need a numerical solver to solve the polynomial equation for which this method looses all of it's appeal (we need a root-finder inside of a root-finder). One possible solution to this is to do it like in Householder's method which is a generalization of the procedure done above for $k=2$. This method can be though of as using the formula for $e_n$ for a lower $k$ to construct the formula for $e_n$ instead of trying to solve the polynomial equation (this is illustrated quite well in the last formula in LutzL's answer). This does not have the problems I mentioned above and as long as the higher order derivatives needed in the formula are easy/fast to compute then this is a good method.