Summary
We shall present here a general study of minimum contrast estimators in a nonparametric setting (although our results are also valid in the classical parametric case) for independent observations. These estimators include many of the most popular estimators in various situations such as maximum likelihood estimators, least squares and other estimators of the regression function, estimators for mixture models or deconvolution... The main theorem relates the rate of convergence of those estimators to the entropy structure of the space of parameters. Optimal rates depending on entropy conditions are already known, at least for some of the models involved, and they agree with what we get for minimum contrast estimators as long as the entropy counts are not too large. But, under some circumstances (“large” entropies or changes in the entropy structure due to local perturbations), the resulting the rates are only suboptimal. Counterexamples are constructed which show that the phenomenon is real for non-parametric maximum likelihood or regression. This proves that, under purely metric assumptions, our theorem is optimal and that minimum contrast estimators happen to be suboptimal.
Article PDF
Similar content being viewed by others
References
Andersen, N.T., Giné, E., Ossiander, M., Zinn, J.: The central limit theorem and the law of iterated logarithm for empirical processes under local conditions. Probab. Theory Relat. Fields77, 271–305 (1988)
Bahadur, R.R.: Examples of inconsistency of maximum likelihood estimates. Sankhyã, Ser. A20, 207–210 (1958)
Barlow, R.E., Bartholomew, D.J., Bremner, J.M., Brunk, H.D.: Statistical inference under order restrictions. New York: Wiley 1972
Bass, R.F.: Law of the iterated logarithm for set-indexed partial sum processes with finite variance. Z. Warscheinlichkeitstheor. Verw. Geb.70, 591–608 (1985)
Birgé, L.: Approximation dans les espaces métriques et théorie de l'estimation. Z. Wahrscheinlichkeitstheor. Verw. Geb.65, 181–237 (1983)
Brigé, L.: Stabilité et instabilité du risque minimax pour des variables indépendantes équidistribuées. Ann. Inst. Henri Poincaré, Sect. B20, 201–223 (1984)
Birgé, L.: On estimating a density using Hellinger distance and some other strange facts. Probab. Theory Relat. Fields71, 271–291 (1986)
Birgé, L.: The Grenander estimator: a nonasymptotic approach. Ann. Stat.17, 1532–1549 (1989)
Birgé, L., Massart, P.: TheL 2 entropy with bracketing. Manuscript (1993)
Birman, M.S., Solomjak, M.Z.: Piecewise-polynomial approximation of functions of the classesW p . Mat. Sb.73, 295–317 (1967)
Donoho, D.L.: Statistical estimation and optimal recovery. Technical report. University of California. Berkeley (1989)
Dudley, R.M.: A course on empirical processes. In: Ecole d'Eté de Probabilités de Saint-Flour XII-1982. Berlin Heidelberg New York: Springer 1984
Efroimovitch, S.Yu., Pinsker, M.S.: Estimation of square-integrable density on the basis of a sequence of observations. Probl. Inf. Transm.17, 182–196 (1981)
Grenander, U.: On the theory of mortality measurement. II. Skand. Aktuarietidskr.39, 125–153 (1956)
Grenander, U.: Abstract inference. New-York: Wiley 1980
Groeneboom, P.: Estimating a monotone density. In: LeCam, L.M., Olshen, R.A. (eds.) Proc. of the Berkeley Conference in Honor of Jerzy Neyman and Jack Kiefer, vol. 2, pp. 539–555. Monterey, CA: Wadsworth 1985
Huber, P.J.: The behavior of maximum likelihood estimates under non-standard conditions. In: Proc. 5th Berkeley Symp. Math. Stat. Probab., vol. 1, pp. 221–233. Berkeley, CA: University of California Press 1967
Ibragimov, I.A., Has'minskii, R.Z.: On estimate of the density function. Zap. Nauchn. Semin. Leningr. Otd. Mat. Inst. Steklova98, 61–85 (1980)
Ibragimov, I.A., Has'minskii, R.Z.: Estimation of distribution density. J. Sov. Math.21, 40–57 (1983)
Kolmogorov, A.N., Tikhomirov, V.M.: ε-entropy and ε-capacity of sets in function spaces. Transl., II. Ser., Am. Math. Soc.17, 277–364 (1961)
Le Cam, L.M.: Convergence of estimates under dimensionality restrictions. Ann. Stat.1, 38–53 (1973)
Le Cam, L.M.: Asymptotic methods in statistical decision theory. Berlin Heidelberg New-York: Springer 1986
Lorentz, G.G.: Approximation of functions. New-York: Holt, Rinehart, Winston 1966
Massart, P.: The tight constant in the D.K.W. inequality. Ann. Probab.18, 1269–1283 (1990)
Nemirovskii, A.S., Polyak, B.T., Tsybakov, A.B.: Signal processing by the nonparametric maximum-likelihood method. Probl. Inf. Transm.20, 177–192 (1984)
Nemirovskii, A.S., Polyak, B.T., Tsybakov, A.B.: Rate of convergence of nonparametric estimates of maximum-likelihood type. Probl. Inf. Transm.21, 258–272 (1985)
Ossiander, M.: A central limit theorem under metric entropy withL 2 bracketing. Ann. Probab.15, 897–919 (1987)
Pfanzagl, J.: On the measurability and consistency of minimum contrast estimates. Metrika14, 249–272 (1969)
Pinsker, M.S.: Optimal filtration of square-integrable signals in gaussian noise. Probl. Inf. Transm.16, 120–133 (1980)
Prakasa Rao, B.L.S.: Estimation of a unimodal density. Sankhya, Ser. A31, 23–36 (1983)
Reiss, R.-D.: Sharp rates of convergence of maximum likelihood estimators in nonparametric models. Z. Wahrscheinlichkeitstheor. Verw. Geb.65, 473–482 (1984)
Severini, T.A., Wong, W.H.: Convergence rates of maximum likelihood and related estimates in general parameter spaces. (Preprint 1987)
Shorack, G.R., Wellner, J.A.: Empirical processes with applications to statistics. New-York: Wiley 1986
Stone, C.J.: Optimal rates of convergence for nonparametric regression. Ann. Stat.10, 1040–1053 (1982)
Van de Geer, S.: Estimating a regression function. Ann. Stat.18, 907–924 (1990a)
Van de Geer, S.: Hellinger-consistency of certain nonparametric maximum likelihood estimates. no 614, Utrecht University (Preprint 1990b)