Consistency of the BIC order estimator

Authors:
Imre Csiszár and Paul C. Shields

Journal:
Electron. Res. Announc. Amer. Math. Soc. **5** (1999), 123-127

MSC (1991):
Primary 62F12, 62M05; Secondary 62F13, 60J10

Published electronically:
October 19, 1999

MathSciNet review:
1715427

Full-text PDF Free Access

Abstract | References | Similar Articles | Additional Information

Abstract: We announce two results on the problem of estimating the order of a Markov chain from observation of a sample path. First is that the Bayesian Information Criterion (BIC) leads to an almost surely consistent estimator. Second is that the Bayesian minimum description length estimator, of which the BIC estimator is an approximation, fails to be consistent for the uniformly distributed i.i.d.process. A key tool is a strong ratio-typicality result for empirical -block distributions. Complete proofs are given in the authors' article to appear in The Annals of Statistics.

**1.**Andrew Barron, Jorma Rissanen, and Bin Yu,*The minimum description length principle in coding and modeling*, IEEE Trans. Inform. Theory**44**(1998), no. 6, 2743–2760. Information theory: 1948–1998. MR**1658898**, 10.1109/18.720554**2.**Imre Csiszár and János Körner,*Information theory*, Probability and Mathematical Statistics, Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers], New York-London, 1981. Coding theorems for discrete memoryless systems. MR**666545****3.**I. Csiszár and P. Shields,*The consistency of the BIC order estimator,*Ann. Statis., submitted.**4.**P. Diaconis and D. A. Freedman,*Nonparametric binary regression: a Bayesian approach*, Ann. Statist.**21**(1993), no. 4, 2108–2137. MR**1245784**, 10.1214/aos/1176349413**5.***Recent advances in mathematical theory of systems, control, networks and signal processing. I*, Mita Press, Tokyo, 1992. Papers from the Ninth International Symposium on the Mathematical Theory of Networks and Systems (MTNS-91) held in Kobe, June 17–21, 1991; Edited by H. Kimura and S. Kodama. MR**1197885****6.**P. Flajolet, P. Kirschenhofer, and R. F. Tichy,*Deviations from uniformity in random strings*, Probab. Theory Related Fields**80**(1988), no. 1, 139–150. MR**970475**, 10.1007/BF00348756**7.**Dominique M. A. Haughton,*On the choice of a model to fit data from an exponential family*, Ann. Statist.**16**(1988), no. 1, 342–355. MR**924875**, 10.1214/aos/1176350709**8.**J. Kieffer,*Strongly consistent code-based identification and order estimation for constrained finite-state model classes,*IEEE Trans. Inform. Th. 39 (1993), 803-902.**9.**Raphail E. Krichevsky and Victor K. Trofimov,*The performance of universal encoding*, IEEE Trans. Inform. Theory**27**(1981), no. 2, 199–207. MR**633417**, 10.1109/TIT.1981.1056331**10.**Katalin Marton and Paul C. Shields,*Entropy and the consistent estimation of joint distributions*, Ann. Probab.**22**(1994), no. 2, 960–977. MR**1288138**

Katalin Marton and Paul C. Shields,*Correction: “Entropy and the consistent estimation of joint distributions” [Ann. Probab. 22 (1994), no. 2, 960–977; MR1288138 (95g:94004)]*, Ann. Probab.**24**(1996), no. 1, 541–545. MR**1387651****11.**Jorma Rissanen,*Stochastic complexity in statistical inquiry*, World Scientific Series in Computer Science, vol. 15, World Scientific Publishing Co., Inc., Teaneck, NJ, 1989. MR**1082556****12.**Gideon Schwarz,*Estimating the dimension of a model*, Ann. Statist.**6**(1978), no. 2, 461–464. MR**0468014**

Retrieve articles in *Electronic Research Announcements of the American Mathematical Society*
with MSC (1991):
62F12,
62M05,
62F13,
60J10

Retrieve articles in all journals with MSC (1991): 62F12, 62M05, 62F13, 60J10

Additional Information

**Imre Csiszár**

Affiliation:
A. Rényi Institute of Mathematics, Hungarian Academy of Sciences, POB 127, 1364 Budapest, Hungary

Email:
csiszar@math-inst.hu

**Paul C. Shields**

Affiliation:
Mathematics Department, The University of Toledo, Toledo, OH 43606

Email:
paul.shields@utoledo.edu

DOI:
http://dx.doi.org/10.1090/S1079-6762-99-00070-0

Keywords:
Bayesian information criterion,
order estimation,
ratio-typicality,
Markov chains

Received by editor(s):
February 25, 1999

Published electronically:
October 19, 1999

Additional Notes:
First author supported in part by a joint NSF-Hungarian Academy grant 92

Second author supported in part by a joint NSF-Hungarian Academy grant INT-9515485

Communicated by:
Yitzhak Katznelson

Article copyright:
© Copyright 1999
American Mathematical Society