1
$\begingroup$

In Newton's method, if the residual delta x is iteratively updated ($X \leftarrow X + \Delta x$) at fixed slope, does it converge?

In newton's method, the estimation of X should be done iteratively. In each iteration the derivative (slope) of a function, the residual are calculated and used to update X. My question is if what will happen if we performed the derivative only one time.

More precisely, assume the following: f(X)=A+B my target is to estimate X

solution 1::

  • outer loop calculate the slope of f(X)
  • inner loop
  • ----calculate A
  • ----calculate B
  • ----calculate delta x using f(x), A, B and the slope
  • end inner loop
  • X=X+ delta x
  • end outer loop

solution 2::

  • loop
  • calculate the slope of f(X)
  • calculate A
  • calculate B
  • calculate delta x using f(x), A, B and the slope
  • X = X + delta x
  • end loop

does the accuracy of X in both cases similar? and why?

  • 1
    What do you mean by "at fixed slope"? – 2017-02-28
  • 0
    In newton's method, the estimation of X should be done iteratively. In each iteration the derivative (slope) of a function, the residual are calculated and used to update X. My question is if what will happen if we performed the derivative only one time. – 2017-02-28

1 Answers 1

3

This is an interesting question.

The "normal" iterative scheme is $$x_{n+1}=x_n-\frac{f(x_n)}{f'(x_n)}$$ with a permanently updated slope.

You are asking about $$x_{n+1}=x_n-\frac{f(x_n)}{f'(x_0)}$$ This was used a lot at a time (I knew that time) where computers did not exist and everything had to be done by hand. It works quite well provided $\cdots$ a few things. For sure, the convergence is slower than the "normal" method.

For illustration purposes, I selected $f(x)=e^{-x}-x$ starting using $x_0=1$. Below are given the iterates for the two cases. $$\left( \begin{array}{ccc} n & \text{with update} & \text{without update}\\ 1 & 0.53788284273999024150 & 0.53788284273999024150 \\ 2 & 0.56698699140541323884 & 0.57158499872254489612 \\ 3 & 0.56714328598912294403 & 0.56650033488134524122 \\ 4 & 0.56714329040978386946 & 0.56723703773937297193 \\ 5 & 0.56714329040978387300 & 0.56712963572569512729 \\ 6 & 0.56714329040978387300 & 0.56714527957469274352 \\ 7 & 0.56714329040978387300 & 0.56714300064189749926 \\ 8 & 0.56714329040978387300 & 0.56714333262131691955 \\ 9 & 0.56714329040978387300 & 0.56714328426068052466 \\ 10 & 0.56714329040978387300 & 0.56714329130554557275 \end{array} \right)$$

  • 0
    Thanks for your reply. I wonder if the accuracy is changed between the two ways – 2017-02-28
  • 1
    @agwadElt. This was done using the same conditions. As you see, updating the derivative makes the solution reached at the fifth iteration while the no-update would take 24 iterations. However, if you want, you can update the derivative from time to time. – 2017-02-28
  • 0
    I think baby Rudin's proof of the inverse function theorem is based on this modified version of Newton's method. In order to show existence of a point $x$ that satisfies $f(x) = y$, Rudin basically uses the modified Newton's method to find a solution (and he proves it converges to a solution using the contraction mapping principle). Somebody correct me if I'm wrong. – 2017-02-28
  • 0
    @ClaudeLeibovici. in my problem, there are other variables affects the calculations of delta x. Is the accuracy of final value X affected? and why? – 2017-02-28
  • 0
    @agwadElt. Are speaking about more than one equation and one unknown ? This would be a very different story. Please, clarify. – 2017-02-28
  • 0
    More precisely, do you think the results will be different between the following two solutions: f(X)=A+B solution 1--> outer loop calculate slope of f(X) ----inner loop ----calculate A ----calculate B ----calculate delta x end inner loop X=X+ delta x end outer loop solution 1--> – 2017-02-28
  • 0
    More precisely, do you think the results will be different between the following two solutions: f(X)=A+B solution 1:: outer loop calculate slope of f(X) ----inner loop ----calculate A ----calculate B ----calculate delta x using f(x), A, B and the slope end inner loop X=X+ delta x end outer loop solution 2:: loop calculate the slope of f(X) calculate A calculate B calculate delta x using f(x), A, B and the slope X=X+ delta x end loop does the accuracy of X in both cases similar? and why? – 2017-02-28
  • 0
    @ClaudeLeibovici. I have edited the original post to clarify this point, kindly see it. – 2017-02-28
  • 0
    @agwadElt. This looks more than dangerous since what you solve is $\Phi(x)=F(x)-A(x)-B(x)=0$ – 2017-02-28
  • 0
    So, which solution is more accurate and why? – 2017-02-28
  • 0
    @agwadElt. What means accurate ? We need the solution and, to me, the second solution could lead to chaos. If, for any reason, you cannot express analytically $A'(x)$ and/or $B'(x)$ which could be (e.g.) implicit functions, use numerical derivatives to get $\Phi'(x)$. – 2017-03-01
  • 0
    @ClaudeLeibovici. I mean the resulted X will be accurate in which solution. why do you think second solution could lead to a chaos? – 2017-03-01
  • 0
    @ClaudeLeibovici. why does the first solution not lead to a chaos? – 2017-03-01