Distributional hyperspace-convergence of Argmin-sets in convex $M$-estimation
Author:
Dietmar Ferger
Journal:
Theor. Probability and Math. Statist. 109 (2023), 3-35
MSC (2020):
Primary 60F05, 62E10; Secondary 60B05, 60B10
DOI:
https://doi.org/10.1090/tpms/1195
Published electronically:
October 3, 2023
Full-text PDF
Abstract |
References |
Similar Articles |
Additional Information
Abstract: In $M$-estimation we consider the sets of all minimizing points of convex empirical criterion functions. These sets are random closed sets. We derive distributional convergence in the hyperspace of all closed subsets of the real line endowed with the Fell-topology. As a special case single minimizing points converge in distribution in the classical sense. In contrast to the literature so far, unusual rates of convergence and non-normal limits emerge, which go far beyond the square-root asymptotic normality. Moreover, our theory can be applied to the sets of zero-estimators.
References
- N. Albrecht, Least squares estimation for binary decision trees, PhD thesis, Technische Universität Dresden, 2020.
- P. K. Andersen and R. D. Gill, Cox’s regression model for counting processes: a large sample study, Ann. Statist. 10 (1982), no. 4, 1100–1120. MR 673646, DOI 10.1214/aos/1176345976
- Peter J. Bickel, Chris A. J. Klaassen, Ya’acov Ritov, and John A. Wellner, Efficient and adaptive estimation for semiparametric models, Springer-Verlag, New York, 1998. Reprint of the 1993 original. MR 1623559
- Patrick Billingsley, Convergence of probability measures, John Wiley & Sons, Inc., New York-London-Sydney, 1968. MR 233396
- Dietmar Ferger, A continuous mapping theorem for the argmin-set functional with applications to convex stochastic processes, Kybernetika (Prague) 57 (2021), no. 3, 426–445. MR 4299457, DOI 10.14736/kyb-2021-3-0426
- Bert Fristedt and Lawrence Gray, A modern approach to probability theory, Probability and its Applications, Birkhäuser Boston, Inc., Boston, MA, 1997. MR 1422917, DOI 10.1007/978-1-4899-2837-5
- Peter Gänssler and Winfried Stute, Wahrscheinlichkeitstheorie, Springer-Verlag, Berlin-New York, 1977 (German). MR 501219, DOI 10.1007/978-3-642-66749-7
- Shelby J. Haberman, Concavity and estimation, Ann. Statist. 17 (1989), no. 4, 1631–1661. MR 1026303, DOI 10.1214/aos/1176347385
- Nils Lid Hjort, Bayes estimators and asymptotic efficiency in parametric counting process models, Scand. J. Statist. 13 (1986), no. 2, 63–85. MR 867355
- N.L. Hjort and D. Pollard, Asymptotic for minimizers of convex processes, Preprint, Dept. of Statistics, Yale University, 1993. http://arxiv.org/abs/1107.3806v1.
- Peter J. Huber, Robust estimation of a location parameter, Ann. Math. Statist. 35 (1964), 73–101. MR 161415, DOI 10.1214/aoms/1177703732
- Peter J. Huber, Robust statistics, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1981. MR 606374, DOI 10.1002/0471725250
- Jana Jurečková, Asymptotic relations of $M$-estimates and $R$-estimates in linear regression model, Ann. Statist. 5 (1977), no. 3, 464–472. MR 433698
- Jana Jurečková, Estimation in a linear model based on regression rank scores, J. Nonparametr. Statist. 1 (1992), no. 3, 197–203. MR 1241522, DOI 10.1080/10485259208832521
- Keith Knight, What are the limiting distributions of quantile estimators?, Statistical data analysis based on the $L_1$-norm and related methods (Neuchâtel, 2002) Stat. Ind. Technol., Birkhäuser, Basel, 2002, pp. 47–65. MR 2001304
- Friedrich Liese and Klaus-J. Miescke, Statistical decision theory, Springer Series in Statistics, Springer, New York, 2008. Estimation, testing, and selection. MR 2421720
- Ilya Molchanov, Theory of random sets, Probability Theory and Stochastic Modelling, vol. 87, Springer-Verlag, London, 2017. Second edition of [ MR2132405]. MR 3751326, DOI 10.1007/978-1-4471-7349-6
- Hung T. Nguyen, An introduction to random sets, Chapman & Hall/CRC, Boca Raton, FL, 2006. MR 2220761, DOI 10.1201/9781420010619
- Constantin P. Niculescu and Lars-Erik Persson, Convex functions and their applications, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, vol. 23, Springer, New York, 2006. A contemporary approach. MR 2178902, DOI 10.1007/0-387-31077-0
- Wojciech Niemiro, Asymptotics for $M$-estimators defined by convex minimization, Ann. Statist. 20 (1992), no. 3, 1514–1533. MR 1186263, DOI 10.1214/aos/1176348782
- Tommy Norberg, Convergence and existence of random set distributions, Ann. Probab. 12 (1984), no. 3, 726–732. MR 744229
- H. Nyquist, The optimal $L_{p}$ norm estimator in linear regression models, Comm. Statist. A—Theory Methods 12 (1983), no. 21, 2511–2524. MR 715180, DOI 10.1080/03610928308828618
- Johann Pfanzagl, Parametric statistical theory, De Gruyter Textbook, Walter de Gruyter & Co., Berlin, 1994. With the assistance of R. Hamböker. MR 1291393, DOI 10.1515/9783110889765
- David Pollard, Empirical processes: theory and applications, NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 2, Institute of Mathematical Statistics, Hayward, CA; American Statistical Association, Alexandria, VA, 1990. MR 1089429, DOI 10.1214/cbms/1462061091
- David Pollard, Asymptotics for least absolute deviation regression estimators, Econometric Theory 7 (1991), no. 2, 186–199. MR 1128411, DOI 10.1017/S0266466600004394
- R. Tyrrell Rockafellar, Convex analysis, Princeton Mathematical Series, No. 28, Princeton University Press, Princeton, NJ, 1970. MR 274683, DOI 10.1515/9781400873173
- R. Tyrrell Rockafellar and Roger J.-B. Wets, Variational analysis, Grundlehren der mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences], vol. 317, Springer-Verlag, Berlin, 1998. MR 1491362, DOI 10.1007/978-3-642-02431-3
- Robert J. Serfling, Approximation theorems of mathematical statistics, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, Inc., New York, 1980. MR 595165, DOI 10.1002/9780470316481
- N. V. Smirnov, Limit distributions for the terms of a variational series, Amer. Math. Soc. Translation 1952 (1952), no. 67, 64. MR 47277
References
- N. Albrecht, Least squares estimation for binary decision trees, PhD thesis, Technische Universität Dresden, 2020.
- P. K. Andersen and R. D. Gill, Cox’s regression model for counting processes: a large sample study, Ann. Statist. 10 (1982), no. 4, 1100–1120. MR 0673646
- P. J. Bickel, C. A. Klaassen, Y. Ritov and J. A. Wellner, Efficient and adaptive inference in semiparametric models, Springer, New York, 1998. MR 1623559
- P. Billingsley, Convergence of probability measures, John Wiley & Sons, New York, 1968. MR 0233396
- D. Ferger, A continuous mapping theorem for the Argmin-set functional with applications to convex stochastic processes, Kybernetika 57 (2021), no. 3, 426–445. MR 4299457
- B. Fristedt and L. Gray, A modern approach to probability theory, Probability and its Applications, Birkhäuser, Boston, 1997. MR 1422917
- P. Gänssler and W. Stute, Wahrscheinlichkeitstheorie, Springer-Verlag, Berlin-New York, 1977. MR 0501219
- S. J. Habermann, Concavity and estimation, Ann. Statist. 17 (1989), no. 4, 1631–1661. MR 1026303
- N. L. Hjort, Bayes estimators and asymptotic efficiency in parametric counting process models, Scand. J. Statist. 13 (1986), no. 2, 63–85. MR 0867355
- N.L. Hjort and D. Pollard, Asymptotic for minimizers of convex processes, Preprint, Dept. of Statistics, Yale University, 1993. http://arxiv.org/abs/1107.3806v1.
- P. Huber, Robust estimation of a location parameter, Ann. Math. Statist. 35 (1964), 73–101. MR 0161415
- P. Huber, Robust statistics, John Wiley & Sons, New York, 1981. MR 0606374
- J. Jurečková, Asymptotic relations of M-estimates and R-estimates in linear regression model, Ann. Statist. 5 (1977), no. 3, 464–472. MR 0433698
- J. Jurečková, Estimation in a linear model based on regression rank scores, J. Nonparametr. Statist. 1 (1992), no. 3, 197–203. MR 1241522
- K. Knight, What are the limiting distributions of quantile estimators? Statistical data analysis based on the $L_1$-norm and related methods (Neuchâtel, 2002) (Y. Dodge, ed.), Stat. Ind. Technol, Birkhäuser, Basel, 2002, pp. 47–65. MR 2001304
- F. Liese and K-J. Mieschke, Statistical decision theory, Springer Series in Statistics, Springer, New York, 2008. MR 2421720
- I. Molchanov, Theory of random sets, Probability Theory and Stochastic Modelling, vol. 87, Springer, London, 2017. MR 3751326
- H. T. Nguyen, An introduction to random sets, Chapman & Hall/CRC, Boca Raton, FL, 2006. MR 2220761
- C. P. Niculescu and L.-E. Persson, Convex functions and their applications: A contemporary approach, CMS Books in Mathematics/Ouvrages de Mathématiques de la SMC, vol. 23, Springer, New York, 2006. MR 2178902
- W. Niemiro, Asymptotic for M-estimators defined by convex minimization, Ann. Statist. 20 (1992), no. 3, 1514–1533. MR 1186263
- T. Norberg, Convergence and existence of random set distributions, Ann. Probab. 12 (1984), no. 3, 726–732. MR 0744229
- H. Nyquist, The optimal $L_p$ norm estimator in linear regression models, Comm. Statist. A—Theory Methods 12 (1983), no. 21, 2511–2524. MR 0715180
- J. Pfanzagl, Parametric statistical theory, De Gruyter Textbook, De Gruyter, Berlin, 1994. MR 1291393
- D. Pollard, Empirical processes: Theory and applications, NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 2, Inst. Math. Statist., Hayward, CA, 1990. MR 1089429
- D. Pollard, Asymptotics for least absolute deviation regression estimators, Econometric Theory 7 (1991), no. 2, 186–199. MR 1128411
- R. T. Rockafellar, Convex analysis, Princeton Mathematical Series, vol. 28, Princeton Univ. Press, Princeton, NJ, 1970. MR 0274683
- R. T. Rockafellar and R. J.-B. Wets, Variational analysis, Grundlehren der mathematischen Wissenschaften, vol. 317, Springer, Berlin, 1998. MR 1491362
- R. J. Serfling, Approximation theorems of mathematical statistics, Wiley Series in Probability and Mathematical Statistics, John Wiley & Sons, New York, 1980. MR 0595165
- N. V. Smirnov, Limit distributions for the terms of a variational series, Amer. Math. Soc. Translation, vol. 67, 1952. MR 0047277
Similar Articles
Retrieve articles in Theory of Probability and Mathematical Statistics
with MSC (2020):
60F05,
62E10,
60B05,
60B10
Retrieve articles in all journals
with MSC (2020):
60F05,
62E10,
60B05,
60B10
Additional Information
Dietmar Ferger
Affiliation:
Fakultät Mathematik, Technische Universität Dresden, D-01069 Dresden, Germany
Email:
dietmar.ferger@tu-dresden.de
Keywords:
M-estimation,
convex empirical processes,
Argmin-sets,
random closed sets,
Fell-topology
Received by editor(s):
April 12, 2022
Accepted for publication:
November 21, 2022
Published electronically:
October 3, 2023
Article copyright:
© Copyright 2023
Taras Shevchenko National University of Kyiv
