Consistency of the BIC order estimator

Authors:
Imre Csiszár and Paul C. Shields

Journal:
Electron. Res. Announc. Amer. Math. Soc. **5** (1999), 123-127

MSC (1991):
Primary 62F12, 62M05; Secondary 62F13, 60J10

DOI:
https://doi.org/10.1090/S1079-6762-99-00070-0

Published electronically:
October 19, 1999

MathSciNet review:
1715427

Full-text PDF Free Access

Abstract | References | Similar Articles | Additional Information

Abstract: We announce two results on the problem of estimating the order of a Markov chain from observation of a sample path. First is that the Bayesian Information Criterion (BIC) leads to an almost surely consistent estimator. Second is that the Bayesian minimum description length estimator, of which the BIC estimator is an approximation, fails to be consistent for the uniformly distributed i.i.d.process. A key tool is a strong ratio-typicality result for empirical -block distributions. Complete proofs are given in the authors' article to appear in The Annals of Statistics.

**1.**A. Barron, J. Rissanen, and B. Yu,*The minimum description length principle in coding and modeling,*IEEE Trans. Inform. Th. 44 (1998), 2743-2760. MR**99h:94032****2.**I. Csiszár and J. Körner,*Information Theory. Coding theorems for discrete memoryless systems*, Akadémiai Kiadó, Budapest, 1981. MR**84e:94007****3.**I. Csiszár and P. Shields,*The consistency of the BIC order estimator,*Ann. Statis., submitted.**4.**P. Diaconis and D. Freedman,*Nonparametric binary regression: a Bayesian approach,*Ann. Statist. 21 (1993), 2108-2137. MR**94i:62001****5.**L. Finesso,*Estimation of the order of a finite Markov chain,*in*Recent Advances in the Mathematical Theory of Systems, Control, and Network Signals, Proc. MTNS-91,*H. Kimura and S. Kodama, Eds., Mita Press, 1992, pp. 643-645. MR**93g:93005****6.**P. Flajolet, P. Kirschenhofer, and R. F. Tichy,*Deviations from uniformity in random strings,*Probab. Th. Rel. Fields 80 (1988), 139-150. MR**90a:11087****7.**D. Haughton,*On the choice of a model to fit data from an exponential family,*Ann. Statist. 16 (1988), 342-355. MR**89e:62036****8.**J. Kieffer,*Strongly consistent code-based identification and order estimation for constrained finite-state model classes,*IEEE Trans. Inform. Th. 39 (1993), 803-902.**9.**R. E. Krichevsky and V. K. Trofimov,*The performance of universal encoding,*IEEE Trans. Inform. Th. 27 (1981), 199-207. MR**83e:94030****10.**K. Marton and P. Shields,*Entropy and the consistent estimation of joint distributions,*Ann. Probab. 22 (1994), 960-977. (Correction, Ann. Probab. 24 (1996), 541-545.) MR**95g:94004**;MR**97c:94004****11.**J. Rissanen,*Stochastic complexity in statistical inquiry*, World Scientific, Singapore, 1989. MR**92f:68076****12.**G. Schwarz,*Estimating the dimension of a model,*Ann. Statist. 6 (1978), 461-464. MR**57:7855**

Retrieve articles in *Electronic Research Announcements of the American Mathematical Society*
with MSC (1991):
62F12,
62M05,
62F13,
60J10

Retrieve articles in all journals with MSC (1991): 62F12, 62M05, 62F13, 60J10

Additional Information

**Imre Csiszár**

Affiliation:
A. Rényi Institute of Mathematics, Hungarian Academy of Sciences, POB 127, 1364 Budapest, Hungary

Email:
csiszar@math-inst.hu

**Paul C. Shields**

Affiliation:
Mathematics Department, The University of Toledo, Toledo, OH 43606

Email:
paul.shields@utoledo.edu

DOI:
https://doi.org/10.1090/S1079-6762-99-00070-0

Keywords:
Bayesian information criterion,
order estimation,
ratio-typicality,
Markov chains

Received by editor(s):
February 25, 1999

Published electronically:
October 19, 1999

Additional Notes:
First author supported in part by a joint NSF-Hungarian Academy grant 92

Second author supported in part by a joint NSF-Hungarian Academy grant INT-9515485

Communicated by:
Yitzhak Katznelson

Article copyright:
© Copyright 1999
American Mathematical Society