Variants of Newton’s method for functions of several variables

Abstract

Some variants of Newton’s Method are developed in this work in order to solve systems of nonlinear equations, based in trapezoidal and midpoint rules of quadrature. We prove the quadratic convergence of one of these methods. Moreover, different numeric tests confirm theoretic results and allow us to compare these variants with Newton’s classical method.

Introduction

Let us consider the problem of finding a real zero of a function $F:D\subseteq {\mathbb{R}}^n\to {\mathbb{R}}^n$, that is, a real solution $\overline{x}$, of the nonlinear equation system F(x) = 0, of n equations with n variables. This solution can be obtained as a fixed point of some function $G:{\mathbb{R}}^n\to {\mathbb{R}}^n$ by means of the fixed point iteration method

$$x^{(k+1)}=G(x^{(k)})\text{,}k=0\text{,}1\text{,}\dots \text{,}$$

where x⁽⁰⁾ is the initial estimation. The best known fixed point method is the classical Newton’s method, given by

$$x^{(k+1)}=x^{(k)}-J_F(x^{(k)}{)}^{-1}F(x^{(k)})\text{,}k=0\text{,}1\text{,}\dots \text{,}$$

where J_F(x^(k)) is the Jacobian matrix of the function F evaluated in x^(k).

In the following, we remember the most common notions about the convergence of an iterative method.

Definition 1

Let {x^(k)}_k⩾0 be a sequence in ${\mathbb{R}}^n$ convergent to $\overline{x}$. Then, convergence is called

• linear, if there exists M, 0 < M < 1, and k₀ such that

  $$\parallel x^{(k+1)}-\overline{x}\parallel ⩽M\parallel x^{(k)}-\overline{x}\parallel \forall k⩾k_0\text{.}$$

• quadratic, if there exists M, M > 0, and k₀ such that

  $$\parallel x^{(k+1)}-\overline{x}\parallel ⩽M\parallel x^{(k)}-\overline{x}{\parallel }^2\forall k⩾k_0\text{.}$$

• cubic, if there exists M, M > 0, and k₀ such that

  $$\parallel x^{(k+1)}-\overline{x}\parallel ⩽M\parallel x^{(k)}-\overline{x}{\parallel }^3\forall k⩾k_0\text{.}$$

  And so on.

In practice, as the limit $\overline{x}$ is unknown, we analyze for each p the behaviour of the quotients

$$\frac{\parallel x^{(k+1)}-x^{(k)}\parallel }{\parallel x^{(k)}-x^{(k-1)}{\parallel }^p}\text{,}k=1\text{,}2\text{,}3\text{,}\dots \text{,}$$

where p = 1, 2, 3, …, which are called convergence test or convergence rate.

If p = 1 and the convergence rate eventually tends to C_L, 0 < C_L < 1, the sequence {x^(k)}_k⩾0 is said to be linearly convergent to $\overline{x}$.

If p = 2 and there exists C_C, C_C > 0, such that the convergence rate eventually tends to C_C, the sequence {x^(k)}_k⩾0 is said to be quadratically convergent to $\overline{x}$. In a similar way for p = 3 (cubic convergence C_Cu) and so on.

Some results about convergence and order of convergence of Newton’s method can be found in [1], [2].

Several modifications have been made on this classical method in order to accelerate the convergence or to reduce the number of operations and evaluations of functions in each step of the iterative process.

The variants on Newton’s method developed by Weerakoon and Fernando in [3] and the analysis of convergence made by Ford and Pennline in [4], for a nonlinear equation, suggest us their extension to functions of several variables. In Section 2 we develop some adjustments on Newton’s method to find solutions of systems of nonlinear equations, and in Section 3 quadratic convergence is proved for one of these methods. Last section is dedicated to the numeric results obtained by applying classical Newton’s method and its alternatives to several systems of nonlinear equations. From these results we compare the different methods and obtain some conclusions.