Abstract
Linear discrete ill-posed problems are difficult to solve numerically because their solution is very sensitive to perturbations, which may stem from errors in the data and from round-off errors introduced during the solution process. The computation of a meaningful approximate solution requires that the given problem be replaced by a nearby problem that is less sensitive to disturbances. This replacement is known as regularization. A regularization parameter determines how much the regularized problem differs from the original one. The proper choice of this parameter is important for the quality of the computed solution. This paper studies the performance of known and new approaches to choosing a suitable value of the regularization parameter for the truncated singular value decomposition method and for the LSQR iterative Krylov subspace method in the situation when no accurate estimate of the norm of the error in the data is available. The regularization parameter choice rules considered include several L-curve methods, Regińska’s method and a modification thereof, extrapolation methods, the quasi-optimality criterion, rules designed for use with LSQR, as well as hybrid methods.
Similar content being viewed by others
References
Bakushinskii, A.B.: Remarks on choosing a regularization parameter using quasi-optimality and ratio criterion. USSR Comp. Math. Math. Phys. 24(4), 181–182 (1984)
Bauer, F., Kindermann, S.: Recent results on the quasi-optimality principle. J. Inverse Ill-Posed Probl. 17, 5–18 (2009)
Bauer, F., Lukas, M.A.: Comparing parameter choice methods for regularization of ill-posed problem. Math. Comput. Simul. 81, 1795–1841 (2011)
Bauer, F., Reiß, M.: Regularization independent of the noise-level: an analysis of quasi-optimality. Inverse Probl. 24, 055009 (16 pp) (2008)
Björck, Å.: Numerical Methods for Least Squares Problems. SIAM, Philadelphia (1996)
Brezinski, C., Rodriguez, G., Seatzu, S.: Error estimates for linear systems with applications to regularization. Numer. Algorithms 49, 85–104 (2008)
Brezinski, C., Rodriguez, G., Seatzu, S.: Error estimates for the regularization of least squares problems. Numer. Algorithms 51, 61–76 (2009)
Calvetti, D., Golub, G.H., Reichel, L.: Estimation of the L-curve via Lanczos bidiagonalization. BIT 39, 603–619 (1999)
Calvetti, D., Hansen, P.C., Reichel, L.: L-curve curvature bounds via Lanczos bidiagonalization. Electron. Trans. Numer. Anal. 14, 20–35 (2002)
Calvetti, D., Lewis, B., Reichel, L.: GMRES, L-curves, and discrete ill-posed problems. BIT 42, 44–65 (2002)
Calvetti, D., Reichel, L.: Tikhonov regularization of large linear problems. BIT 43, 263–283 (2003)
Castellanos, J.L., Gómez, S., Guerra, V.: The triangle method for finding the corner of the L-curve. Appl. Numer. Math. 43, 359–373 (2002)
Donatelli, M.: Fast transforms for high order boundary conditions in deconvolution problems. BIT 50, 559–576 (2010)
Eldén, L.: A weighted pseudoinverse, generalized singular values, and constrained least squares problems. BIT 22, 487–501 (1982)
Engl, H.W., Hanke, M., Neubauer, A.: Regularization of Inverse Problems. Kluwer, Dordrecht (1996)
Golub, G.H., Meurant, G.: Matrices, moments and quadrature. In: Griffiths, D.F., Watson, G. A. (eds.) Numerical Analysis 1993, pp. 105–156. Longman, Essex (1994)
Golub, G.H., Meurant, G.: Matrices, Moments and Quadrature with Applications. Princeton University Press, Princeton (2010)
Golub, G.H., von Matt, U.: Tikhonov regularization for large scale problems. In: Golub, G.H., Lui, S.H., Luk, F., Plemmons, R. (eds.) Workshop on Scientific Computing, pp. 3–26. Springer, New York (1997)
Golub, G.H., von Matt, U.: Generalized cross-validation for large scale problems. J. Comput. Graph. Stat. 6, 1–34 (1997)
Hanke, M.: Conjugate Gradient Type Methods for Ill-Posed Problems. Longman, Essex (1995)
Hanke, M.: Limitations of the L-curve method in ill-posed problems. BIT 36, 287–301 (1996)
Hanke, M., Hansen, P.C.: Regularization methods for large-scale problems. Surv. Math. Ind. 3, 253–315 (1993)
Hansen, P.C.: Regularization, GSVD and truncated GSVD. BIT 29, 491–504 (1989)
Hansen, P.C.: Analysis of the discrete ill-posed problems by means of the L-curve. SIAM Rev. 34, 561–580 (1992)
Hansen, P.C.: Rank-Deficient and Discrete Ill-Posed Problems. SIAM, Philadelphia (1998)
Hansen, P.C.: Regularization tools version 4.0 for Matlab 7.3. Numer. Algorithms 46, 189–194 (2007)
Hansen, P.C., Jensen, T.K., Rodriguez, G.: An adaptive pruning algorithm for the discrete L-curve criterion. J. Comput. Appl. Math. 198, 483–492 (2006)
Hansen, P.C., O’Leary, D.P.: The use of the L-curve in the regularization of discrete ill-posed problems. SIAM J. Sci. Comput. 14, 1487–1503 (1993)
Hämarik, U., Palm, R., Raus, T.: A family of rules for parameter choice in Tikhonov regularization of ill-posed problems with inexact noise level. J. Comput. Appl. Math. 236, 2146–2157 (2012)
Hnětynková, I., Plešinger, M., Strakoš, Z.: The regularizing effect of the Golub-Kahan iterative bidiagonalization and revealing the noise level in the data. BIT 49, 669–696 (2009)
Kindermann, S.: Convergence analysis of minimization-based noise level-free parameter choice rules for linear ill-posed problems. Electron. Trans. Numer. Anal. 38, 233–257 (2011)
Lukas, M.A.: Robust generalized cross-validation for choosing the regularization parameter. Inverse Probl. 22, 1883–1902 (2006)
Morigi, S., Reichel, L., Sgallari, F., Zama, F.: Iterative methods for ill-posed problems and semiconvergent sequences. J. Comput. Appl. Math. 193, 157–167 (2006)
Neuman, A., Reichel, L., Sadok, H.: Implementations of range restricted iterative methods for linear discrete ill-posed problems. Linear Algebra Appl. 436, 3974–3990 (2012)
Neuman, A., Reichel, L., Sadok, H.: Algorithms for range restricted iterative methods for linear discrete ill-posed problems. Numer. Algorithms 59, 325–331 (2012)
Paige, C.C., Saunders, M.A.: LSQR: an algorithm for sparse linear equations and sparse least squares. Trans. Math. Software 8, 43–71 (1982)
Phillips, D.L.: A technique for the numerical solution of certain integral equations of the first kind. J. ACM 9, 84–97 (1962)
Reichel, L., Rodriguez, G., Seatzu, S.: Error estimates for large-scale ill-posed problems. Numer. Algorithms 51, 341–361 (2009)
Reichel, L., Sadok, H.: A new L-curve for ill-posed problems. J. Comput. Appl. Math. 219, 493–508 (2008)
Reichel, L., Ye, Q.: Simple square smoothing regularization operators. Electron. Trans. Numer. Anal. 33, 63–83 (2009)
Regińska, T.: A regularization parameter in discrete ill-posed problems. SIAM J. Sci. Comput. 17, 740–749 (1996)
Shaw, Jr., C.B.: Improvements of the resolution of an instrument by numerical solution of an integral equation. J. Math. Anal. Appl. 37, 83–112 (1972)
Tikhonov, A., Arsenin, V.: Solutions of Ill-Posed Problems. Wiley, New York (1977)
Vogel, C.R.: Non-convergence of the L-curve regularization parameter selection method. Inverse Probl. 12, 535–547 (1996)
Author information
Authors and Affiliations
Corresponding author
Additional information
Dedicated to Claude Brezinski and Sebastiano Seatzu on the occasion of their 70th birthdays.
Research of L. Reichel was supported in part by NSF grant DMS-1115385 while research of G. Rodriguez was supported in part by MIUR-PRIN grant no. 20083KLJEZ-003.
Rights and permissions
About this article
Cite this article
Reichel, L., Rodriguez, G. Old and new parameter choice rules for discrete ill-posed problems. Numer Algor 63, 65–87 (2013). https://doi.org/10.1007/s11075-012-9612-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11075-012-9612-8