Asymptotic properties of the residual bootstrap for Lasso estimators
HTML articles powered by AMS MathViewer
- by A. Chatterjee and S. N. Lahiri PDF
- Proc. Amer. Math. Soc. 138 (2010), 4497-4509 Request permission
Abstract:
In this article, we derive the asymptotic distribution of the bootstrapped Lasso estimator of the regression parameter in a multiple linear regression model. It is shown that under some mild regularity conditions on the design vectors and the regularization parameter, the bootstrap approximation converges weakly to a random measure. The convergence result rigorously establishes a previously known heuristic formula for the limit distribution of the bootstrapped Lasso estimator. It is also shown that when one or more components of the regression parameter vector are zero, the bootstrap may fail to be consistent.References
- Krishna B. Athreya and Soumendra N. Lahiri, Measure theory and probability theory, Springer Texts in Statistics, Springer, New York, 2006. MR 2247694
- R. N. Bhattacharya and R. Ranga Rao, Normal approximation and asymptotic expansions, Robert E. Krieger Publishing Co., Inc., Melbourne, FL, 1986. Reprint of the 1976 original. MR 855460
- D. A. Freedman, Bootstrapping regression models, Ann. Statist. 9 (1981), no. 6, 1218–1228. MR 630104, DOI 10.1214/aos/1176345638
- Olav Kallenberg, Random measures, 4th ed., Akademie-Verlag, Berlin; Academic Press, Inc., London, 1986. MR 854102
- JeanKyung Kim and David Pollard, Cube root asymptotics, Ann. Statist. 18 (1990), no. 1, 191–219. MR 1041391, DOI 10.1214/aos/1176347498
- Keith Knight and Wenjiang Fu, Asymptotics for lasso-type estimators, Ann. Statist. 28 (2000), no. 5, 1356–1378. MR 1805787, DOI 10.1214/aos/1015957397
- Hannes Leeb and Benedikt M. Pötscher, Sparse estimators and the oracle property, or the return of Hodges’ estimator, J. Econometrics 142 (2008), no. 1, 201–211. MR 2394290, DOI 10.1016/j.jeconom.2007.05.017
- Benedikt M. Pötscher and Hannes Leeb, On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding, J. Multivariate Anal. 100 (2009), no. 9, 2065–2082. MR 2543087, DOI 10.1016/j.jmva.2009.06.010
- Robert Tibshirani, Regression shrinkage and selection via the lasso, J. Roy. Statist. Soc. Ser. B 58 (1996), no. 1, 267–288. MR 1379242, DOI 10.1111/j.2517-6161.1996.tb02080.x
- Aad W. van der Vaart and Jon A. Wellner, Weak convergence and empirical processes, Springer Series in Statistics, Springer-Verlag, New York, 1996. With applications to statistics. MR 1385671, DOI 10.1007/978-1-4757-2545-2
- M. J. Wainwright, Sharp thresholds for high-dimensional and noisy recovery of sparsity. arXiv:math/0605740v1 (2006).
- Peng Zhao and Bin Yu, On model selection consistency of Lasso, J. Mach. Learn. Res. 7 (2006), 2541–2563. MR 2274449
- Hui Zou, The adaptive lasso and its oracle properties, J. Amer. Statist. Assoc. 101 (2006), no. 476, 1418–1429. MR 2279469, DOI 10.1198/016214506000000735
Additional Information
- A. Chatterjee
- Affiliation: Department of Statistics, Texas A&M University, College Station, Texas 77843-3143
- Email: cha@stat.tamu.edu
- S. N. Lahiri
- Affiliation: Department of Statistics, Texas A&M University, College Station, Texas 77843-3143
- MR Author ID: 310114
- Email: snlahiri@stat.tamu.edu
- Received by editor(s): January 22, 2009
- Received by editor(s) in revised form: December 23, 2009, and March 2, 2010
- Published electronically: July 9, 2010
- Additional Notes: This research was partially supported by NSF grant DMS-0707139.
- Communicated by: Edward C. Waymire
- © Copyright 2010 American Mathematical Society
- Journal: Proc. Amer. Math. Soc. 138 (2010), 4497-4509
- MSC (2010): Primary 62J07; Secondary 62G09, 62E20
- DOI: https://doi.org/10.1090/S0002-9939-2010-10474-4
- MathSciNet review: 2680074