Abstract
We demonstrate how to perform direct simulation from the posterior distribution of a class of multiple changepoint models where the number of changepoints is unknown. The class of models assumes independence between the posterior distribution of the parameters associated with segments of data between successive changepoints. This approach is based on the use of recursions, and is related to work on product partition models. The computational complexity of the approach is quadratic in the number of observations, but an approximate version, which introduces negligible error, and whose computational cost is roughly linear in the number of observations, is also possible. Our approach can be useful, for example within an MCMC algorithm, even when the independence assumptions do not hold. We demonstrate our approach on coal-mining disaster data and on well-log data. Our method can cope with a range of models, and exact simulation from the posterior distribution is possible in a matter of minutes.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albert J. H. and Chib S. 1993. Bayes inference via Gibbs sampling of autoregressive time series subject to Markov mean and variance shifts. Journal of Business and Economic Statistics 11: 1–15.
Barry D. and Hartigan J. A. 1992. Product partition models for change point problems. The Annals of Statistics 20: 260–279.
Barry D. and Hartigan J. A. 1993. A Bayesian analysis for change point problems. Journal of the American Statistical Society 88: 309–319.
Braun J. V. and Muller H. G. 1998. Statistical methods for DNA sequence segmentation. Statistical Science 13: 142–162.
Braun J. V., Braun R. K., and Muller H. G. 2000. Multiple changepoint fitting via quasilikelihood, with application to DNA sequence segmentation. Biometrika 87: 301–314.
Brooks S. P., Giudici P., and Roberts G. O. 2003. Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions. Journal of the Royal Statistical Society, series B 65: 3–39.
Carlin B. P., Gelfand A. E., and Smith A. F. M. 1992. Hierarchical Bayesian analysis of changepoint problems. Applied Statistics 41: 389–405.
Carpenter J., Clifford P., and Fearnhead P. 1999. An improved particle filter for non-linear problems. IEE proceedings-Radar, Sonar and Navigation 146: 2–7.
Chen J. and Gupta A. K. 1997. Testing and locating changepoints with application to stock prices. Journal of the American Statistical Association 92: 739–747.
Chib S. 1995. Marginal likelihood from the Gibbs output. Journal of the American Statistical Association 90: 1313–1321.
Chib S. 1996. Calculating posterior distributions and modal estimates in Markov mixture models. Journal of Econometrics 75: 79–98.
Chib S. 1998. Estimation and comparison of multiple change-point models. Journal of Econometrics 86: 221–241.
Fearnhead P. 2005a. Direct simulation for discrete mixture distributions. Statistics and Computing 15: 125–133.
Fearnhead P. 2005b. Exact Bayesian curve fitting and signal segmentation. IEEE Transactions on Signal Processing 53: 2160–2166.
Fearnhead P. and Clifford P. 2003. Online inference for hidden Markov models. Journal of the Royal Statistical Society, Series B 65: 887–899.
Fearnhead P. and Liu Z. 2005. Online inference for multiple changepoint problems. Submitted Available from http://www.maths.lancs.ac.uk/~fearnhea/publications.
Fearnhead P. and Meligkotsidou L. 2004. Exact filtering for partially-observed continuous-time Markov models. Journal of the Royal Statistical Society, series B 66: 771–789.
Green P. 1995. Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732.
Green P. J. 2003. Transdimensional Markov chain Monte Carlo. In: Highly Structured Stochastic Systems (eds. Green P. J., Hjort N. L., and Richardson S.), Oxford University Press.
Hartigan J. A. 1990. Partition models. Communications in Statistics 19: 2745–2756.
Harvey A. C. 1989. Forecasting, stuctural time series and the Kalman filter. Cambridge University Press, Cambridge, UK.
Jarrett R. G. 1979. A note on the intervals between coal-mining disasters. Biometrika 66: 191–3.
Johnson T. D., Elashoff R. M., and Harkema S. J. 2003. A Bayesian change-point analysis of electromyographic data: detecting muscle activation patterns and associated applications. Biostatistics 4: 143–164.
Lavielle M. and Lebarbier E. 2001. An application of MCMC methods for the multiple change-points problem. Signal Processing 81: 39–53.
Liu J. S. 2001. Monte Carlo strategies in scientific computing. New York: Springer.
Liu J. S. and Lawrence C. E. 1999. Bayesian inference on biopolymer models. Bioinformatics 15: 38–52.
Lund R. and Reeves J. 2002. Detection of undocumented changepoints: A revision of the two-phase regression model. Journal of Climate 15: 2547–2554.
ó Ruanaidh J. J. K. and Fitzgerald W. J. 1996. Numerical Bayesion Methods Applied to Signal Processing. New York: Springer.
Pievatolo A. and Green P. J. 1998. Boundary detection through dynamic polygons. Journal of the Royal Statistical Society, Series B 60: 609–626.
Propp J. G. and Wilson D. B. 1996. Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures and Algorithms 9: 223–252.
Punskaya E., Andrieu C., Doucet A., and Fitzgerald W. J. 2002. Bayesian curve fitting using MCMC with applications to signal segmentation. IEEE Transactions on Signal Processing 50: 747–758.
Raftery A. E. and Akman V. E. 1986. Bayesian analysis of a Poisson process with a change-point. Biometrika 73: 85–89.
Ritov Y., Raz A., and Bergman H. 2002. Detection of onset of neuronal activity by allowing for heterogeneity in the change points. Journal of Neuroscience Methods 122: 25–42.
Scott S. L. 2002. Bayesian methods for hidden Markov models: Recursive computing in the 21st century. Journal of the American Statistical Association 97: 337–351.
Stephens D. A. 1994. Bayesian retrospective multiple-changepoint identification. Applied Statistics 43: 159–178.
Worsley K. J. 1979. On the likelihood ratio test for a shift in location of normal populations. Journal of the American Statistical Association 74: 363–367.
Yang T. Y. and Kuo L. 2001. Bayesian binary segmentation procedure for a Poisson process with multiple changepoints. Journal of Computational and Graphical Statistics 10: 772–785.
Yao Y. 1984. Estimation of a noisy discrete-time step function: Bayes and empirical Bayes approaches. The Annals of Statistics 12: 1434–1447.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Fearnhead, P. Exact and efficient Bayesian inference for multiple changepoint problems. Stat Comput 16, 203–213 (2006). https://doi.org/10.1007/s11222-006-8450-8
Received:
Accepted:
Issue Date:
DOI: https://doi.org/10.1007/s11222-006-8450-8