Maximum efficiency for a family of Newton-like methods with frozen derivatives and some applications

Abstract

A generalized k-step iterative application of Newton’s method with frozen derivative is studied and used to solve a system of nonlinear equations. The maximum computational efficiency is computed. A sequence that approximates the order of convergence is generated for the examples, and it numerically confirms the calculation of the order of the method and computational efficiency. This type of method appears in many applications where the authors have heuristically chosen a given number of steps with frozen derivatives. An example is shown in which the total variation (TV) minimization model is approximated using the schemes described in this paper.

Introduction

In this paper, a family of Newton-like methods with frozen derivatives is analyzed. This type of method appears in many applications where the authors heuristically choose a given number of steps with frozen derivatives (see for instance this incomplete list of Refs. [14], [21], [22], [23], [24], [25]). Our goal is to give a rigorous estimate of the maximum efficiency of these schemes for a given problem.

Let $F:D\subset {\mathbb{R}}^m\to {\mathbb{R}}^m$ be a nonlinear operator and assume that F has at least bounded third-order derivatives with continuity on an open convex set D. Suppose that the equation $F(x)=0$ has a solution $\alpha \in D$ at which $F^{\prime }(\alpha )$ is nonsingular.

Setting $e=x-\alpha$, we develop $F(x)$ and its derivatives in a neighborhood of $\alpha$. That is, assuming that ${\left[F^{\prime }(\alpha )\right]}^{-1}$ exists, we have

$$F(x)=\mathrm{\Gamma }\left(e+A_2{e}^2+O(e^3)\right)\text{,}$$

where $\mathrm{\Gamma }=F^{\prime }(\alpha )$ and $A_2=\frac{1}{2}{\mathrm{\Gamma }}^{-1}{F}^{″}(\alpha )\in {\mathcal{L}}_2({\mathbb{R}}^m\text{,}{\mathbb{R}}^m)$. From (1), the derivative of $F(x)$ can be written as $F^{\prime }(x)=\mathrm{\Gamma }\left(I+2A_{2}e+O(e^2)\right)$, and expanding the inverse of the derivative of $F(x)$ in terms of e in formal power developments, we get

$${\left[F^{\prime }(x)\right]}^{-1}=\left(I-2A_{2}e+O(e^2)\right){\mathrm{\Gamma }}^{-1}\text{.}$$

A direct computation of the local order of convergence for Newton’s method and for its generalization in k steps are presented in Section 2. The optimal number of steps in order to maximize the computational efficiency index is computed in Section 3. In Section 4, we analyze several researchers’ examples that illustrate our theoretical results. Section 5 is devoted to the local and semilocal convergence of these schemes. The last section shows an application related to image denoising, where we approximate the total variation (TV) minimization model using the described schemes. A nonlinear primal–dual method is used to remove some of the singularity caused by the nondifferentiability of the quantity $\nabla u/|\nabla u|$ in the definition of the TV norm. A finite difference scheme is then applied and the associated nonlinear system of equations is approximated by the most efficient candidate of our family for this particular problem.