Analysis of the finite precision

bi-conjugate gradient algorithm

for nonsymmetric linear systems

Authors:
Charles H. Tong and Qiang Ye

Journal:
Math. Comp. **69** (2000), 1559-1575

MSC (1991):
Primary 65F10, 65N20

DOI:
https://doi.org/10.1090/S0025-5718-99-01171-0

Published electronically:
August 19, 1999

MathSciNet review:
1665975

Full-text PDF

Abstract | References | Similar Articles | Additional Information

Abstract: In this paper we analyze the bi-conjugate gradient algorithm in finite precision arithmetic, and suggest reasons for its often observed robustness. By using a tridiagonal structure, which is preserved by the finite precision bi-conjugate gradient iteration, we are able to bound its residual norm by a minimum polynomial of a perturbed matrix (i.e. the residual norm of the exact GMRES applied to a perturbed matrix) multiplied by an amplification factor. This shows that occurrence of near-breakdowns or loss of biorthogonality does not necessarily deter convergence of the residuals provided that the amplification factor remains bounded. Numerical examples are given to gain insights into these bounds.

**[1]**Z. Bai,*Error Analysis of the Lanczos Algorithm for the Nonsymmetric Eigenvalue Problem,*Math. Comp., 62:209-226 (1994). MR**94c:65045****[2]**R. E. Bank and T. F. Chan,*An Analysis of the Composite Step Biconjugate Gradient Algorithm for Solving nonsymmetric Systems,*Numer. Math., 66:295-319 (1993). MR**94i:65043****[3]**T. Barth and T. A. Manteuffel,*Variable Metric Conjugate Gradient Methods*, Proceedings of the 10th International Symposium on Matrix Analysis and Parallel Computing, Keio University, Yokohama, Japan, March 14-16, 1994.**[4]**J. Cullum and A. Greenbaum,*Relation between Galerkian and norm-minimizing iterative methods for solving linear systems*SIAM J. Matrix Anal. Appl. 17:223-247 (1996). MR**97b:65035****[5]**D. Day,*Semi-duality in the two-sided Lanczos algorithm.*Ph.D. Thesis, University of Californial, Berkeley, 1993.**[6]**I. S. Duff, R. G. Grimes, and J. G. Lewis,*Sparse Matrix Test Problems,*ACM Trans. Math. Softw., 15:1-14 (1989).**[7]**R. Fletcher,*Conjugate Gradient Methods for Indefinite Systems,*in Proc. Dundee Conference on Numerical Analysis, 1975, Lecture Notes in Mathematics 506, G. A. Watson, ed., Springer-Verlag, Berlin, pp. 73-89 (1976). MR**57:1841****[8]**R. W. Freund and N. M. Nachtigal,*QMR : a Quasi-minimal Residual Method for non-Hermitian Linear Systems,*Numer. Math., 60:315-339 (1991). MR**92g:65034****[9]**G. H. Golub and M. L. Overton,*Convergence of a two-stage Richardson iterative procedure for solving systems of linear equations*. in Numerical Analysis, Lect. Notes Math 912, (ed. G.A. Watson), Springer, New York Heidelberg Berlin pp.128-139. MR**83f:65045****[10]**G. H. Golub and M. L. Overton,*The convergence of inexact Chebyshev and Richardson iterative methods for solving linear systems.*Numer. Math. 53:571-593 (1988). MR**90b:65054****[11]**G. H. Golub, C. F. Van Loan,*Matrix Computations,*The Johns Hopkins University Press, Baltimore, 1983. MR**85h:65063****[12]**A. Greenbaum,*Behavior of Slightly Perturbed Lanczos and Conjugate-Gradient Recurrences,*Lin. Alg. and its Appl., 113:7-63 (1989). MR**90e:65044****[13]**A. Greenbaum,*Accuracy of Computed Solutions from Conjugate-Gradient-Like Methods,*PCG '94, Matrix Analysis and Parallel Computing, Keio University, March 14-16, 1994.**[14]**A. Greenbaum and Z. Strakos,*Predicting the behavior of finite precision Lanczos and Conjugate Gradient computations,*SIAM J. Matrix Anal. Appl. 13:121-137 (1992). MR**92j:65043****[15]**A. Greenbaum, V. Druskin, and L. Knizhnerman,*Private communication.***[16]**C. Lanczos,*Solution of Systems of Linear Equations by Minimized Iterations,*J. Res. Natl. Bur. Stand. 49:33-53 (1952). MR**14:501g****[17]**R. Lehoucq,*Analysis and implementation of an implicitly restarted iteration*Ph.D. Thesis, Rice University, Houston, Texas, May 1995.**[18]**Y. Notay,*On the convergence rate of the conjugate gradients in presence of rounding errors*, Numer. Math. 65:301-317 (1993). MR**94j:65050****[19]**C. Paige,*Error Analysis of the Lanczos Algorithm for Tridiagonalizing a Symmetric Matrix,*J. Inst. Math. Appl., 18:341-349 (1976). MR**58:19082****[20]**C. Paige,*Accuracy and Effectiveness of the Lanczos Algorithm for the Symmetric Eigenproblem,*Linear Alg. Appl. 34(1980):235-258. MR**82b:65025****[21]**Y. Saad,*Analysis of some Krylov subspace approximations to the matrix exponential operators*, SIAM J. Numer. Anal. 29:209-228 (1992). MR**92m:65050****[22]**P. Sonneveld,*CGS, a Fast Lanczos-type Solver for Nonsymmetric Linear Systems,*SIAM J. Sci. Stat. Comput., 10:36-52 (1989). MR**89k:65052****[23]**C. H. Tong,*A Comparative Study of Preconditioned Lanczos Methods for Nonsymmetric Linear Systems,*Tech. Report SAND91-9240B, Sandia National Lab., Livermore, 1992.**[24]**L. N. Trefethen,*Approximation theory and numerical linear algebra,*in Algorithms for Approximation II, J.C. Mason and M.G. Cox eds., Chapman and Hall, London, 1990, pp. 336-360. MR**91j:65063****[25]**H. A. Van der Vorst,*Bi-CGSTAB : A Fast and Smoothly Converging Variant of Bi-CG for the Solution of Nonsymmetric Linear Systems,*SIAM J. Sci. Stat. Comput., 13:631-644 (1992). MR**92j:65048****[26]**H. A. Van der Vorst,*The convergence behaviour of preconditioned CG and CGS*in Lect. Notes in Math. 1457, ed. O. Axelsson and L. Kolotilina, pp. 121-136, Springer, Berlin Heidelberg New York (1990), pp. 1-99. MR**92a:65141****[27]**Q. Ye,*A convergence analysis of nonsymmetric Lanczos algorithms*, Math. Comp. 56:677-691 (1991). MR**91m:65115**

Retrieve articles in *Mathematics of Computation of the American Mathematical Society*
with MSC (1991):
65F10,
65N20

Retrieve articles in all journals with MSC (1991): 65F10, 65N20

Additional Information

**Charles H. Tong**

Affiliation:
Sandia National Laboratories, Livermore, CA 94551

Email:
chtong@california.sandia.gov

**Qiang Ye**

Affiliation:
Department of Mathematics, University of Manitoba, Winnipeg, Manitoba, Canada R3T 2N2

Email:
ye@gauss.amath.umanitoba.ca

DOI:
https://doi.org/10.1090/S0025-5718-99-01171-0

Keywords:
Bi-conjugate gradient algorithm,
error analysis,
convergence analysis,
nonsymmetric linear systems

Received by editor(s):
October 6, 1998

Published electronically:
August 19, 1999

Additional Notes:
The first author’s research was supported by Research Grant Council of Hong Kong.

The second author’s research was supported by Natural Sciences and Engineering Research Council of Canada. Part of this work was completed while this author visited Stanford University during the summer of 1995. He would like to thank Professor Gene Golub for providing this opportunity and for his great hospitality.

Article copyright:
© Copyright 2000
American Mathematical Society