## Posterior consistency for Gaussian process approximations of Bayesian posterior distributions

HTML articles powered by AMS MathViewer

- by Andrew M. Stuart and Aretha L. Teckentrup PDF
- Math. Comp.
**87**(2018), 721-753 Request permission

## Abstract:

We study the use of Gaussian process emulators to approximate the parameter-to-observation map or the negative log-likelihood in Bayesian inverse problems. We prove error bounds on the Hellinger distance between the true posterior distribution and various approximations based on the Gaussian process emulator. Our analysis includes approximations based on the mean of the predictive process, as well as approximations based on the full Gaussian process emulator. Our results show that the Hellinger distance between the true posterior and its approximations can be bounded by moments of the error in the emulator. Numerical results confirm our theoretical findings.## References

- Robert J. Adler,
*The geometry of random fields*, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Ltd., Chichester, 1981. MR**611857** - Christophe Andrieu and Gareth O. Roberts,
*The pseudo-marginal approach for efficient Monte Carlo computations*, Ann. Statist.**37**(2009), no. 2, 697–725. MR**2502648**, DOI 10.1214/07-AOS574 - N. Aronszajn,
*Theory of reproducing kernels*, Trans. Amer. Math. Soc.**68**(1950), 337–404. MR**51437**, DOI 10.1090/S0002-9947-1950-0051437-7 - S. R. Arridge, J. P. Kaipio, V. Kolehmainen, M. Schweiger, E. Somersalo, T. Tarvainen, and M. Vauhkonen,
*Approximation errors and model reduction with an application in optical diffusion tomography*, Inverse Problems**22**(2006), no. 1, 175–195. MR**2194190**, DOI 10.1088/0266-5611/22/1/010 - Ivo Babuška, Fabio Nobile, and Raúl Tempone,
*A stochastic collocation method for elliptic partial differential equations with random input data*, SIAM Rev.**52**(2010), no. 2, 317–355. MR**2646806**, DOI 10.1137/100786356 - I. Bilionis, N. Zabaras, B. A. Konomi, and G. Lin,
*Multi-output separable Gaussian process: Towards an efficient, fully Bayesian paradigm for uncertainty quantification*, Journal of Computational Physics**241**(2013), 212–239. - Nikolay Bliznyuk, David Ruppert, Christine Shoemaker, Rommel Regis, Stefan Wild, and Pradeep Mugunthan,
*Bayesian calibration and uncertainty analysis for computationally expensive models using optimization and radial basis function approximation*, J. Comput. Graph. Statist.**17**(2008), no. 2, 270–294. MR**2439960**, DOI 10.1198/106186008X320681 - Vladimir I. Bogachev,
*Gaussian measures*, Mathematical Surveys and Monographs, vol. 62, American Mathematical Society, Providence, RI, 1998. MR**1642391**, DOI 10.1090/surv/062 - T. Bui-Thanh, K. Willcox, and O. Ghattas,
*Model reduction for large-scale systems with high-dimensional parametric input space*, SIAM J. Sci. Comput.**30**(2008), no. 6, 3270–3288. MR**2452388**, DOI 10.1137/070694855 - Albert Cohen, Ronald Devore, and Christoph Schwab,
*Analytic regularity and polynomial approximation of parametric and stochastic elliptic PDE’s*, Anal. Appl. (Singap.)**9**(2011), no. 1, 11–47. MR**2763359**, DOI 10.1142/S0219530511001728 - P. R. Conrad, Y. M. Marzouk, N. S. Pillai, and A. Smith,
*Asymptotically exact MCMC algorithms via local approximations of computationally intensive models*, J. Amer. Statist. Assoc.**111**(2016), 1591–1607. - Paul G. Constantine,
*Active subspaces*, SIAM Spotlights, vol. 2, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 2015. Emerging ideas for dimension reduction in parameter studies. MR**3486165**, DOI 10.1137/1.9781611973860 - Paul G. Constantine, Eric Dow, and Qiqi Wang,
*Active subspace methods in theory and practice: applications to kriging surfaces*, SIAM J. Sci. Comput.**36**(2014), no. 4, A1500–A1524. MR**3233940**, DOI 10.1137/130916138 - S. L. Cotter, G. O. Roberts, A. M. Stuart, and D. White,
*MCMC methods for functions: modifying old algorithms to make them faster*, Statist. Sci.**28**(2013), no. 3, 424–446. MR**3135540**, DOI 10.1214/13-STS421 - M. Dashti and A.M.Stuart,
*The Bayesian Approach to Inverse Problems*, Handbook of Uncertainty Quantification (R. Ghanem, D. Higdon, and H. Owhadi, eds.), Springer. - Mark Girolami and Ben Calderhead,
*Riemann manifold Langevin and Hamiltonian Monte Carlo methods*, J. R. Stat. Soc. Ser. B Stat. Methodol.**73**(2011), no. 2, 123–214. With discussion and a reply by the authors. MR**2814492**, DOI 10.1111/j.1467-9868.2010.00765.x - Markus Hansen and Christoph Schwab,
*Sparse adaptive approximation of high dimensional parametric initial value problems*, Vietnam J. Math.**41**(2013), no. 2, 181–215. MR**3089816**, DOI 10.1007/s10013-013-0011-9 - W. K. Hastings,
*Monte Carlo sampling methods using Markov chains and their applications*, Biometrika**57**(1970), no. 1, 97–109. MR**3363437**, DOI 10.1093/biomet/57.1.97 - Dave Higdon, Marc Kennedy, James C. Cavendish, John A. Cafeo, and Robert D. Ryne,
*Combining field data and computer simulations for calibration and prediction*, SIAM J. Sci. Comput.**26**(2004), no. 2, 448–466. MR**2116355**, DOI 10.1137/S1064827503426693 - Jari Kaipio and Erkki Somersalo,
*Statistical and computational inverse problems*, Applied Mathematical Sciences, vol. 160, Springer-Verlag, New York, 2005. MR**2102218** - Marc C. Kennedy and Anthony O’Hagan,
*Bayesian calibration of computer models*, J. R. Stat. Soc. Ser. B Stat. Methodol.**63**(2001), no. 3, 425–464. MR**1858398**, DOI 10.1111/1467-9868.00294 - Youssef Marzouk and Dongbin Xiu,
*A stochastic collocation approach to Bayesian inference in inverse problems*, Commun. Comput. Phys.**6**(2009), no. 4, 826–847. MR**2672325**, DOI 10.4208/cicp.2009.v6.p826 - Youssef M. Marzouk, Habib N. Najm, and Larry A. Rahn,
*Stochastic spectral methods for efficient Bayesian solution of inverse problems*, J. Comput. Phys.**224**(2007), no. 2, 560–586. MR**2330284**, DOI 10.1016/j.jcp.2006.10.010 - Bertil Matérn,
*Spatial variation*, 2nd ed., Lecture Notes in Statistics, vol. 36, Springer-Verlag, Berlin, 1986. With a Swedish summary. MR**867886**, DOI 10.1007/978-1-4615-7892-5 - J. Mercer,
*Functions of positive and negative type, and their connection with the theory of integral equations*, Philosophical Transactions of the Royal Society of London, Series A**209**(1909), 415–446. - N. Metropolis, A. W. Rosenbluth, M. N. Rosenbluth, A. H. Teller, and E. Teller,
*Equation of state calculations by fast computing machines*, J. Chemical Physics**21**(1953), 1087. - Francis J. Narcowich, Joseph D. Ward, and Holger Wendland,
*Sobolev bounds on functions with scattered zeros, with applications to radial basis function surface fitting*, Math. Comp.**74**(2005), no. 250, 743–763. MR**2114646**, DOI 10.1090/S0025-5718-04-01708-9 - Francis J. Narcowich, Joseph D. Ward, and Holger Wendland,
*Sobolev error estimates and a Bernstein inequality for scattered data interpolation via radial basis functions*, Constr. Approx.**24**(2006), no. 2, 175–186. MR**2239119**, DOI 10.1007/s00365-005-0624-7 - Harald Niederreiter,
*Random number generation and quasi-Monte Carlo methods*, CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 63, Society for Industrial and Applied Mathematics (SIAM), Philadelphia, PA, 1992. MR**1172997**, DOI 10.1137/1.9781611970081 - A. O’Hagan,
*Bayesian analysis of computer code outputs: a tutorial*, Reliability Engineering & System Safety**91**(2006), no. 10, 1290–1300. - Giuseppe Da Prato and Jerzy Zabczyk,
*Stochastic equations in infinite dimensions*, Encyclopedia of Mathematics and its Applications, vol. 44, Cambridge University Press, Cambridge, 1992. MR**1207136**, DOI 10.1017/CBO9780511666223 - Carl Edward Rasmussen and Christopher K. I. Williams,
*Gaussian processes for machine learning*, Adaptive Computation and Machine Learning, MIT Press, Cambridge, MA, 2006. MR**2514435** - Patrick Rebeschini and Ramon van Handel,
*Can local particle filters beat the curse of dimensionality?*, Ann. Appl. Probab.**25**(2015), no. 5, 2809–2866. MR**3375889**, DOI 10.1214/14-AAP1061 - Christian P. Robert and George Casella,
*Monte Carlo statistical methods*, Springer Texts in Statistics, Springer-Verlag, New York, 1999. MR**1707311**, DOI 10.1007/978-1-4757-3071-5 - Walter Rudin,
*Principles of mathematical analysis*, 2nd ed., McGraw-Hill Book Co., New York, 1964. MR**0166310** - Jerome Sacks, William J. Welch, Toby J. Mitchell, and Henry P. Wynn,
*Design and analysis of computer experiments*, Statist. Sci.**4**(1989), no. 4, 409–435. With comments and a rejoinder by the authors. MR**1041765** - M. Scheuerer, R. Schaback, and M. Schlather,
*Interpolation of spatial data—a stochastic or a deterministic problem?*, European J. Appl. Math.**24**(2013), no. 4, 601–629. MR**3082868**, DOI 10.1017/S0956792513000016 - Cl. Schillings and Ch. Schwab,
*Sparsity in Bayesian inversion of parametric operator equations*, Inverse Problems**30**(2014), no. 6, 065007, 30. MR**3224127**, DOI 10.1088/0266-5611/30/6/065007 - M. Sinsbeck and W. Nowak,
*Sequential design of computer experiments for the solution of Bayesian inverse problems with process emulators*, SIAM/ASA Journal on Uncertainty Quantification, to appear - Michael L. Stein,
*Interpolation of spatial data*, Springer Series in Statistics, Springer-Verlag, New York, 1999. Some theory for Kriging. MR**1697409**, DOI 10.1007/978-1-4612-1494-6 - A. M. Stuart,
*Inverse problems: a Bayesian perspective*, Acta Numer.**19**(2010), 451–559. MR**2652785**, DOI 10.1017/S0962492910000061 - Wolfgang Walter,
*Ordinary differential equations*, Graduate Texts in Mathematics, vol. 182, Springer-Verlag, New York, 1998. Translated from the sixth German (1996) edition by Russell Thompson; Readings in Mathematics. MR**1629775**, DOI 10.1007/978-1-4612-0601-9 - Holger Wendland,
*Scattered data approximation*, Cambridge Monographs on Applied and Computational Mathematics, vol. 17, Cambridge University Press, Cambridge, 2005. MR**2131724** - Zong Min Wu and Robert Schaback,
*Local error estimates for radial basis function interpolation of scattered data*, IMA J. Numer. Anal.**13**(1993), no. 1, 13–27. MR**1199027**, DOI 10.1093/imanum/13.1.13 - Dongbin Xiu and George Em Karniadakis,
*Modeling uncertainty in flow simulations via generalized polynomial chaos*, J. Comput. Phys.**187**(2003), no. 1, 137–167. MR**1977783**, DOI 10.1016/S0021-9991(03)00092-5

## Additional Information

**Andrew M. Stuart**- Affiliation: Mathematics Institute, Zeeman Building, University of Warwick, Coventry, CV4 7AL, England
- Address at time of publication: Computing and Mathematical Sciences, Caltech, Pasadena, California 91125
- Email: astuart@caltech.edu
**Aretha L. Teckentrup**- Affiliation: Mathematics Institute, Zeeman Building, University of Warwick, Coventry, CV4 7AL, England
- Address at time of publication: School of Mathematics, James Clerk Maxwell Building, University of Edinburgh, EH9 3FD, Edinburgh, Scotland
- Email: a.teckentrup@ed.ac.uk
- Received by editor(s): March 7, 2016
- Received by editor(s) in revised form: September 26, 2016
- Published electronically: August 3, 2017
- © Copyright 2017 American Mathematical Society
- Journal: Math. Comp.
**87**(2018), 721-753 - MSC (2010): Primary 60G15, 62G08, 65D05, 65D30, 65J22
- DOI: https://doi.org/10.1090/mcom/3244
- MathSciNet review: 3739215