Minimax entropy solutions of ill-posed problems
Author:
Fred Greensite
Journal:
Quart. Appl. Math. 67 (2009), 137-161
MSC (2000):
Primary 47A52, 45Q05; Secondary 65J20, 34A55
DOI:
https://doi.org/10.1090/S0033-569X-09-01120-7
Published electronically:
January 8, 2009
MathSciNet review:
2497601
Full-text PDF Free Access
Abstract |
References |
Similar Articles |
Additional Information
Abstract: Convergent methodology for ill-posed problems is typically equivalent to application of an operator dependent on a single parameter derived from the noise level and the data (a regularization parameter or terminal iteration number). In the context of a given problem discretized for purposes of numerical analysis, these methods can be viewed as resulting from imposed prior constraints bearing the same amount of information content. We identify a new convergent method for the treatment of certain multivariate ill-posed problems, which imposes constraints of a much lower information content (i.e., having much lower bias), based on the operatorβs dependence on many data-derived parameters. The associated marked performance improvements that are possible are illustrated with solution estimates for a Lyapunov equation structured by an ill-conditioned matrix. The methodology can be understood in terms of a Minimax Entropy Principle, which emerges from the Maximum Entropy Principle in some multivariate settings.
References
- A. Bouhamidi and K. Jbilou, Sylvester Tikhonov-regularization methods in image restoration, J. Comput. Appl. Math. 206 (2007), no. 1, 86β98. MR 2333837, DOI https://doi.org/10.1016/j.cam.2006.05.028
- Heinz W. Engl, Martin Hanke, and Andreas Neubauer, Regularization of inverse problems, Mathematics and its Applications, vol. 375, Kluwer Academic Publishers Group, Dordrecht, 1996. MR 1408680
- Joel N. Franklin, Well-posed stochastic extensions of ill-posed linear problems, J. Math. Anal. Appl. 31 (1970), 682β716. MR 267654, DOI https://doi.org/10.1016/0022-247X%2870%2990017-X
- Zoran Gajic and Muhammad Tahir Javed Qureshi, Lyapunov matrix equation in system stability and control, Mathematics in Science and Engineering, vol. 195, Academic Press, Inc., San Diego, CA, 1995. MR 1343974
- Martin Hanke, Limitations of the $L$-curve method in ill-posed problems, BIT 36 (1996), no. 2, 287β301. MR 1432249, DOI https://doi.org/10.1007/BF01731984
- Per Christian Hansen, The discrete Picard condition for discrete ill-posed problems, BIT 30 (1990), no. 4, 658β672. MR 1082808, DOI https://doi.org/10.1007/BF01933214
- Per Christian Hansen, Analysis of discrete ill-posed problems by means of the ${\mathsf L}$-curve, SIAM Rev. 34 (1992), no. 4, 561β580. MR 1193012, DOI https://doi.org/10.1137/1034115
- E. T. Jaynes, Probability theory, Cambridge University Press, Cambridge, 2003. The logic of science; Edited and with a foreword by G. Larry Bretthorst. MR 1992316
- Athanasios Papoulis, Probability, random variables, and stochastic processes, 2nd ed., McGraw-Hill Series in Electrical Engineering. Communications and Information Theory, McGraw-Hill Book Co., New York, 1984. MR 836136
- Albert Tarantola, Inverse problem theory and methods for model parameter estimation, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2005. MR 2130010
- Charles F. Van Loan, Generalizing the singular value decomposition, SIAM J. Numer. Anal. 13 (1976), no. 1, 76β83. MR 411152, DOI https://doi.org/10.1137/0713009
- J. M. Varah, Pitfalls in the numerical solution of linear ill-posed problems, SIAM J. Sci. Statist. Comput. 4 (1983), no. 2, 164β176. MR 697171, DOI https://doi.org/10.1137/0904012
- C. R. Vogel, Non-convergence of the $L$-curve regularization parameter selection method, Inverse Problems 12 (1996), no. 4, 535β547. MR 1402108, DOI https://doi.org/10.1088/0266-5611/12/4/013
- Grace Wahba, Practical approximate solutions to linear operator equations when the data are noisy, SIAM J. Numer. Anal. 14 (1977), no. 4, 651β667. MR 471299, DOI https://doi.org/10.1137/0714044
- S. C. Zhu, Y. N. Wu, and D. Mumford, Minimax entropy principle and its application to texture modeling, Neural Computation 9 (1997), 1627-1660.
References
- A. Bouhamidi and K. Jbilou, Sylvester Tikhonov-regularization methods in image restoration, J. Comput. Appl. Math. 206 (2007), 86-98. MR 2333837
- H. W. Engl, M. Hanke, and A. Neubauer, Regularization of inverse problems, Kluwer Academic Publishers, Dordrecht, The Netherlands, 1996. MR 1408680 (97k:65145)
- J. N. Franklin, Well-posed stochatic extensions of ill-posed problems, J. Math. Anal. Appl. 31 (1970), 682β716. MR 0267654 (42:2556)
- Z. Gajic and M. T. J. Quereshi, Lyapunov Matrix Equation in System Stability and Control, Academic Press, San Diego, 1995. MR 1343974 (96g:93001)
- M. Hanke, Limitations of the L-curve method in ill-posed problems, BIT 36 (1996), 287-301. MR 1432249 (97j:65098)
- P. C. Hansen, The discrete Picard condition for discrete ill-posed problems, BIT 30 (1990), 658β672. MR 1082808 (91m:65119)
- P. C. Hansen, Analysis of discrete ill-posed problems by means of the L-curve, SIAM Review 34 (1992), 561-580. MR 1193012 (93k:65035)
- E. T. Jaynes, Probability Theory, Cambridge University Press, Cambridge, United Kingdom, 2003. MR 1992316 (2004g:62006)
- A. Papoulis, Probability, Random Variables, and Stochastic Processes, McGraw-Hill, New York, 1984. MR 836136 (87d:60001)
- A. Tarantola, Inverse Problem Theory, SIAM, Philadelphia, 2005. MR 2130010 (2007b:62011)
- C. F. van Loan, Generalizing the singular value decomposition, SIAM J. Numer. Anal. 11 (1976), 76-83. MR 0411152 (53:14891)
- J. M. Varah, Pitfalls in the numerical solution of linear ill-posed problems, SIAM J. Sci. Stat. Comput. 4 (1983), 164-176. MR 697171 (84g:65052)
- C. Vogel, Non-convergence of the L-curve regularization parameter selection method, Inverse Problems 12, (1996) 535-547. MR 1402108 (97k:65149)
- G. Wahba, Practical approximate solutions to linear operator equations when the data are noisy, SIAM J. Numer. Anal. 14 (1977), 651-667. MR 0471299 (57:11036)
- S. C. Zhu, Y. N. Wu, and D. Mumford, Minimax entropy principle and its application to texture modeling, Neural Computation 9 (1997), 1627-1660.
Similar Articles
Retrieve articles in Quarterly of Applied Mathematics
with MSC (2000):
47A52,
45Q05,
65J20,
34A55
Retrieve articles in all journals
with MSC (2000):
47A52,
45Q05,
65J20,
34A55
Additional Information
Fred Greensite
Affiliation:
Department of Radiological Sciences, University of California, Orange, California 92868
Email:
fred.greensite@uci.edu
Keywords:
Inverse problems,
ill-posed problems,
Sylvester equation
Received by editor(s):
August 7, 2007
Published electronically:
January 8, 2009
Article copyright:
© Copyright 2009
Brown University
The copyright for this article reverts to public domain 28 years after publication.