Consistency of the BIC order estimator
Authors:
Imre Csiszár and Paul C. Shields
Journal:
Electron. Res. Announc. Amer. Math. Soc. 5 (1999), 123127
MSC (1991):
Primary 62F12, 62M05; Secondary 62F13, 60J10
Published electronically:
October 19, 1999
MathSciNet review:
1715427
Fulltext PDF Free Access
Abstract 
References 
Similar Articles 
Additional Information
Abstract: We announce two results on the problem of estimating the order of a Markov chain from observation of a sample path. First is that the Bayesian Information Criterion (BIC) leads to an almost surely consistent estimator. Second is that the Bayesian minimum description length estimator, of which the BIC estimator is an approximation, fails to be consistent for the uniformly distributed i.i.d.process. A key tool is a strong ratiotypicality result for empirical block distributions. Complete proofs are given in the authors' article to appear in The Annals of Statistics.
 1.
Andrew
Barron, Jorma
Rissanen, and Bin
Yu, The minimum description length principle in coding and
modeling, IEEE Trans. Inform. Theory 44 (1998),
no. 6, 2743–2760. Information theory: 1948–1998. MR 1658898
(99h:94032), http://dx.doi.org/10.1109/18.720554
 2.
Imre
Csiszár and János
Körner, Information theory, Probability and Mathematical
Statistics, Academic Press, Inc. [Harcourt Brace Jovanovich, Publishers],
New YorkLondon, 1981. Coding theorems for discrete memoryless systems. MR 666545
(84e:94007)
 3.
I. Csiszár and P. Shields, The consistency of the BIC order estimator, Ann. Statis., submitted.
 4.
P.
Diaconis and D.
A. Freedman, Nonparametric binary regression: a Bayesian
approach, Ann. Statist. 21 (1993), no. 4,
2108–2137. MR 1245784
(94i:62001), http://dx.doi.org/10.1214/aos/1176349413
 5.
Recent advances in mathematical theory of systems, control, networks
and signal processing. I, Mita Press, Tokyo, 1992. Papers from the
Ninth International Symposium on the Mathematical Theory of Networks and
Systems (MTNS91) held in Kobe, June 17–21, 1991; Edited by H. Kimura
and S. Kodama. MR 1197885
(93g:93005)
 6.
P.
Flajolet, P.
Kirschenhofer, and R.
F. Tichy, Deviations from uniformity in random strings,
Probab. Theory Related Fields 80 (1988), no. 1,
139–150. MR
970475 (90a:11087), http://dx.doi.org/10.1007/BF00348756
 7.
Dominique
M. A. Haughton, On the choice of a model to fit data from an
exponential family, Ann. Statist. 16 (1988),
no. 1, 342–355. MR 924875
(89e:62036), http://dx.doi.org/10.1214/aos/1176350709
 8.
J. Kieffer, Strongly consistent codebased identification and order estimation for constrained finitestate model classes, IEEE Trans. Inform. Th. 39 (1993), 803902.
 9.
Raphail
E. Krichevsky and Victor
K. Trofimov, The performance of universal encoding, IEEE
Trans. Inform. Theory 27 (1981), no. 2,
199–207. MR
633417 (83e:94030), http://dx.doi.org/10.1109/TIT.1981.1056331
 10.
Katalin
Marton and Paul
C. Shields, Entropy and the consistent estimation of joint
distributions, Ann. Probab. 22 (1994), no. 2,
960–977. MR 1288138
(95g:94004)
Katalin
Marton and Paul
C. Shields, Correction: “Entropy and the consistent
estimation of joint distributions” [Ann.\ Probab.\ {22} (1994), no.\
2, 960–977; MR1288138 (95g:94004)], Ann. Probab.
24 (1996), no. 1, 541–545. MR 1387651
(97c:94004)
 11.
Jorma
Rissanen, Stochastic complexity in statistical inquiry, World
Scientific Series in Computer Science, vol. 15, World Scientific
Publishing Co., Inc., Teaneck, NJ, 1989. MR 1082556
(92f:68076)
 12.
Gideon
Schwarz, Estimating the dimension of a model, Ann. Statist.
6 (1978), no. 2, 461–464. MR 0468014
(57 #7855)
 1.
 A. Barron, J. Rissanen, and B. Yu, The minimum description length principle in coding and modeling, IEEE Trans. Inform. Th. 44 (1998), 27432760. MR 99h:94032
 2.
 I. Csiszár and J. Körner, Information Theory. Coding theorems for discrete memoryless systems, Akadémiai Kiadó, Budapest, 1981. MR 84e:94007
 3.
 I. Csiszár and P. Shields, The consistency of the BIC order estimator, Ann. Statis., submitted.
 4.
 P. Diaconis and D. Freedman, Nonparametric binary regression: a Bayesian approach, Ann. Statist. 21 (1993), 21082137. MR 94i:62001
 5.
 L. Finesso, Estimation of the order of a finite Markov chain, in Recent Advances in the Mathematical Theory of Systems, Control, and Network Signals, Proc. MTNS91, H. Kimura and S. Kodama, Eds., Mita Press, 1992, pp. 643645. MR 93g:93005
 6.
 P. Flajolet, P. Kirschenhofer, and R. F. Tichy, Deviations from uniformity in random strings, Probab. Th. Rel. Fields 80 (1988), 139150. MR 90a:11087
 7.
 D. Haughton, On the choice of a model to fit data from an exponential family, Ann. Statist. 16 (1988), 342355. MR 89e:62036
 8.
 J. Kieffer, Strongly consistent codebased identification and order estimation for constrained finitestate model classes, IEEE Trans. Inform. Th. 39 (1993), 803902.
 9.
 R. E. Krichevsky and V. K. Trofimov, The performance of universal encoding, IEEE Trans. Inform. Th. 27 (1981), 199207. MR 83e:94030
 10.
 K. Marton and P. Shields, Entropy and the consistent estimation of joint distributions, Ann. Probab. 22 (1994), 960977. (Correction, Ann. Probab. 24 (1996), 541545.) MR 95g:94004;MR 97c:94004
 11.
 J. Rissanen, Stochastic complexity in statistical inquiry, World Scientific, Singapore, 1989. MR 92f:68076
 12.
 G. Schwarz, Estimating the dimension of a model, Ann. Statist. 6 (1978), 461464. MR 57:7855
Similar Articles
Retrieve articles in Electronic Research Announcements of the American Mathematical Society
with MSC (1991):
62F12,
62M05,
62F13,
60J10
Retrieve articles in all journals
with MSC (1991):
62F12,
62M05,
62F13,
60J10
Additional Information
Imre Csiszár
Affiliation:
A. Rényi Institute of Mathematics, Hungarian Academy of Sciences, POB 127, 1364 Budapest, Hungary
Email:
csiszar@mathinst.hu
Paul C. Shields
Affiliation:
Mathematics Department, The University of Toledo, Toledo, OH 43606
Email:
paul.shields@utoledo.edu
DOI:
http://dx.doi.org/10.1090/S1079676299000700
PII:
S 10796762(99)000700
Keywords:
Bayesian information criterion,
order estimation,
ratiotypicality,
Markov chains
Received by editor(s):
February 25, 1999
Published electronically:
October 19, 1999
Additional Notes:
First author supported in part by a joint NSFHungarian Academy grant 92
Second author supported in part by a joint NSFHungarian Academy grant INT9515485
Communicated by:
Yitzhak Katznelson
Article copyright:
© Copyright 1999
American Mathematical Society
