Consistency of the BIC order estimator
Authors:
Imre Csiszár and Paul C. Shields
Journal:
Electron. Res. Announc. Amer. Math. Soc. 5 (1999), 123-127
MSC (1991):
Primary 62F12, 62M05; Secondary 62F13, 60J10
DOI:
https://doi.org/10.1090/S1079-6762-99-00070-0
Published electronically:
October 19, 1999
MathSciNet review:
1715427
Full-text PDF Free Access
Abstract |
References |
Similar Articles |
Additional Information
Abstract: We announce two results on the problem of estimating the order of a Markov chain from observation of a sample path. First is that the Bayesian Information Criterion (BIC) leads to an almost surely consistent estimator. Second is that the Bayesian minimum description length estimator, of which the BIC estimator is an approximation, fails to be consistent for the uniformly distributed i.i.d. process. A key tool is a strong ratio-typicality result for empirical $k$-block distributions. Complete proofs are given in the authors’ article to appear in The Annals of Statistics.
- Andrew Barron, Jorma Rissanen, and Bin Yu, The minimum description length principle in coding and modeling, IEEE Trans. Inform. Theory 44 (1998), no. 6, 2743–2760. Information theory: 1948–1998. MR 1658898, DOI https://doi.org/10.1109/18.720554
- Imre Csiszár and János Körner, Information theory, Probability and Mathematical Statistics, Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers], New York-London, 1981. Coding theorems for discrete memoryless systems. MR 666545
- I. Csiszár and P. Shields, The consistency of the BIC order estimator, Ann. Statis., submitted.
- P. Diaconis and D. A. Freedman, Nonparametric binary regression: a Bayesian approach, Ann. Statist. 21 (1993), no. 4, 2108–2137. MR 1245784, DOI https://doi.org/10.1214/aos/1176349413
- Recent advances in mathematical theory of systems, control, networks and signal processing. I, Mita Press, Tokyo, 1992. Papers from the Ninth International Symposium on the Mathematical Theory of Networks and Systems (MTNS-91) held in Kobe, June 17–21, 1991; Edited by H. Kimura and S. Kodama. MR 1197885
- P. Flajolet, P. Kirschenhofer, and R. F. Tichy, Deviations from uniformity in random strings, Probab. Theory Related Fields 80 (1988), no. 1, 139–150. MR 970475, DOI https://doi.org/10.1007/BF00348756
- Dominique M. A. Haughton, On the choice of a model to fit data from an exponential family, Ann. Statist. 16 (1988), no. 1, 342–355. MR 924875, DOI https://doi.org/10.1214/aos/1176350709
- J. Kieffer, Strongly consistent code-based identification and order estimation for constrained finite-state model classes, IEEE Trans. Inform. Th. 39 (1993), 803–902.
- Raphail E. Krichevsky and Victor K. Trofimov, The performance of universal encoding, IEEE Trans. Inform. Theory 27 (1981), no. 2, 199–207. MR 633417, DOI https://doi.org/10.1109/TIT.1981.1056331
- Katalin Marton and Paul C. Shields, Entropy and the consistent estimation of joint distributions, Ann. Probab. 22 (1994), no. 2, 960–977. MR 1288138
- Jorma Rissanen, Stochastic complexity in statistical inquiry, World Scientific Series in Computer Science, vol. 15, World Scientific Publishing Co., Inc., Teaneck, NJ, 1989. MR 1082556
- Gideon Schwarz, Estimating the dimension of a model, Ann. Statist. 6 (1978), no. 2, 461–464. MR 468014
- A. Barron, J. Rissanen, and B. Yu, The minimum description length principle in coding and modeling, IEEE Trans. Inform. Th. 44 (1998), 2743–2760.
- I. Csiszár and J. Körner, Information Theory. Coding theorems for discrete memoryless systems, Akadémiai Kiadó, Budapest, 1981.
- I. Csiszár and P. Shields, The consistency of the BIC order estimator, Ann. Statis., submitted.
- P. Diaconis and D. Freedman, Nonparametric binary regression: a Bayesian approach, Ann. Statist. 21 (1993), 2108–2137.
- L. Finesso, Estimation of the order of a finite Markov chain, in Recent Advances in the Mathematical Theory of Systems, Control, and Network Signals, Proc. MTNS-91, H. Kimura and S. Kodama, Eds., Mita Press, 1992, pp. 643–645.
- P. Flajolet, P. Kirschenhofer, and R. F. Tichy, Deviations from uniformity in random strings, Probab. Th. Rel. Fields 80 (1988), 139–150.
- D. Haughton, On the choice of a model to fit data from an exponential family, Ann. Statist. 16 (1988), 342–355.
- J. Kieffer, Strongly consistent code-based identification and order estimation for constrained finite-state model classes, IEEE Trans. Inform. Th. 39 (1993), 803–902.
- R. E. Krichevsky and V. K. Trofimov, The performance of universal encoding, IEEE Trans. Inform. Th. 27 (1981), 199–207.
- K. Marton and P. Shields, Entropy and the consistent estimation of joint distributions, Ann. Probab. 22 (1994), 960–977. (Correction, Ann. Probab. 24 (1996), 541–545.) ; MR 97c:94004
- J. Rissanen, Stochastic complexity in statistical inquiry, World Scientific, Singapore, 1989.
- G. Schwarz, Estimating the dimension of a model, Ann. Statist. 6 (1978), 461–464.
Similar Articles
Retrieve articles in Electronic Research Announcements of the American Mathematical Society
with MSC (1991):
62F12,
62M05,
62F13,
60J10
Retrieve articles in all journals
with MSC (1991):
62F12,
62M05,
62F13,
60J10
Additional Information
Imre Csiszár
Affiliation:
A. Rényi Institute of Mathematics, Hungarian Academy of Sciences, POB 127, 1364 Budapest, Hungary
Email:
csiszar@math-inst.hu
Paul C. Shields
Affiliation:
Mathematics Department, The University of Toledo, Toledo, OH 43606
Email:
paul.shields@utoledo.edu
Keywords:
Bayesian information criterion,
order estimation,
ratio-typicality,
Markov chains
Received by editor(s):
February 25, 1999
Published electronically:
October 19, 1999
Additional Notes:
First author supported in part by a joint NSF-Hungarian Academy grant 92
Second author supported in part by a joint NSF-Hungarian Academy grant INT-9515485
Communicated by:
Yitzhak Katznelson
Article copyright:
© Copyright 1999
American Mathematical Society
