A stability analysis for perturbed nonlinear iterative methods

Authors:
Paul T. Boggs and J. E. Dennis

Journal:
Math. Comp. **30** (1976), 199-215

MSC:
Primary 65H10

DOI:
https://doi.org/10.1090/S0025-5718-1976-0395209-4

MathSciNet review:
0395209

Full-text PDF

Abstract | References | Similar Articles | Additional Information

Abstract: This paper applies the asymptotic stability theory for ordinary differential equations to Gavurin's continuous analogue of several well-known nonlinear iterative methods. In particular, a general theory is developed which extends the Ortega-Rheinboldt concept of consistency to include the widely used finite-difference approximations to the gradient as well as the finite-difference approximations to the Jacobian in Newton's method. The theory is also shown to be applicable to the Levenberg-Marquardt and finite-difference Levenberg-Marquardt methods.

**[1]**P. T. BOGGS (1971) "The solution of nonlinear operator equations by*A*-stable integration techniques,"*SIAM J. Numer. Anal.*, v. 8, pp. 767-785. MR**45**#6179. MR**0297121 (45:6179)****[2]**P. T. BOGGS (1976) "The convergence of the Ben-Israel iteration for nonlinear least squares problems,"*Math. Comp.*(To appear.) MR**0416018 (54:4095)****[3]**W. E. BOSARGE, JR. (1968)*Infinite Dimensional Iterative Methods and Applications*, IBM Publications 230-2347, Houston.**[4]**K. M. BROWN & J. E. DENNIS, JR. (1970), "Derivative-free analogues of the Levenberg-Marquardt and Gauss algorithms for nonlinear least squares approximation,"*Numer. Math.*, v. 18, pp. 289-297. MR**46**#2859. MR**0303723 (46:2859)****[5]**E. A. CODDINGTON & N. LEVINSON (1955)*Theory of Ordinary Differential Equations*, McGraw-Hill, New York. MR**16**, 1022. MR**0069338 (16:1022b)****[6]**J. E. DENNIS, JR. (1971) "Algorithms for nonlinear problems which use discrete approximations to derivatives,"*Proc. ACM*1971*Nat'l. Conference*, Chicago.**[7]**J. E. DENNIS, JR. (1970) "On the convergence of Newton-like methods,"*Numerical Methods for Nonlinear Algebraic Equations*, edited by Philip Rabinowitz, Gordon-Breach, New York. MR**0331759 (48:10091)****[8]**M. K. GAVURIN (1968) "Nonlinear functional equations and continuous analogs,"*Izv. Vysš. Učebn. Zaved. Matematika*, v. 1958, no. 5 (6), pp. 18-31; English transl., Technical Report 68-70, University of Maryland, College Park, Md. MR**25**#1380. MR**0137932 (25:1380)****[9]**A. A. GOLDSTEIN (1967),*Constructive Real Analysis*, Harper & Row, New York. MR**36**#705. MR**0217616 (36:705)****[10]**G. H. GOLUB & V. PEREYRA (1972), "The differentiation of pseudo inverses and nonlinear least squares problems whose variables separate,"*SIAM J. Numer. Anal.*, v. 10, pp. 413-432. MR**49**#1753. MR**0336980 (49:1753)****[11]**P. HENRICI (1962),*Discrete Variable Methods in Ordinary Differential Equations*, John Wiley, New York. MR**24**#B1772. MR**0135729 (24:B1772)****[12]**J. HURT (1967) "Some stability theorems for ordinary difference equations,"*SIAM J. Numer. Anal.*, v. 4, pp. 582-596. MR**36**#4839. MR**0221787 (36:4839)****[13]**K. LEVENBERG (1944) "A method for the solution of certain non-linear problems in least squares,"*Quart. Appl. Math.*, v. 2, pp. 164-168. MR**6**, 52. MR**0010666 (6:52a)****[14]**D. W. MARQUARDT (1963) "An algorithm for least-squares estimation of nonlinear parameters,"*J. Soc. Indust. Appl. Math.*, v. 11, pp. 431-441. MR**27**#3040. MR**0153071 (27:3040)****[15]**GUNTER H. MEYER (1968), "On solving nonlinear equations with a one-parameter operator imbedding,"*SIAM J. Numer Anal.*, v. 5, pp. 739-752. MR**39**#3697. MR**0242366 (39:3697)****[16]**J. M. ORTEGA (1973) "Stability of difference equations and convergence of iterative processes,"*SIAM J. Numer. Anal.*, v. 10, pp. 268-282. MR**49**#4281. MR**0339523 (49:4281)****[17]**J. M. ORTEGA & W. C. RHEINBOLDT (1970)*Iterative Solution of Nonlinear Equations in Several Variables*, Academic Press, New York. MR**42**#8686. MR**0273810 (42:8686)****[18]**A. M. OSTROWSKI (1966)*Solution of Equations and Systems of Equations*, 2nd ed., Pure and Appl. Math., vol. 9, Academic Press, New York. MR**35**#7575. MR**0216746 (35:7575)****[19]**C. RADHAKRISHNA RAO & SUJIT KUMAR MITRA (1971)*Generalized Inverse of Matrices and Its Applications*, John Wiley, New York. MR**49**#2780. MR**0338013 (49:2780)****[20]**R. A. TAPIA (1971) "The differentiation and integration of nonlinear operators,"*Nonlinear Functional Analysis and Applications*(Proc. Advanced Sem., Math. Res. Center, Univ. of Wisconsin, Madison, Wis., 1970), Academic Press, New York, pp. 45-101. MR**44**#3160. MR**0285943 (44:3160)**

Retrieve articles in *Mathematics of Computation*
with MSC:
65H10

Retrieve articles in all journals with MSC: 65H10

Additional Information

DOI:
https://doi.org/10.1090/S0025-5718-1976-0395209-4

Keywords:
Nonlinear iterative methods,
stability analysis,
consistent approximations,
steepest descent,
Newton's method,
nonlinear least squares methods

Article copyright:
© Copyright 1976
American Mathematical Society