Remote Access Proceedings of the American Mathematical Society
Green Open Access

Proceedings of the American Mathematical Society

ISSN 1088-6826(online) ISSN 0002-9939(print)

 
 

 

Asymptotic properties of the residual bootstrap for Lasso estimators


Authors: A. Chatterjee and S. N. Lahiri
Journal: Proc. Amer. Math. Soc. 138 (2010), 4497-4509
MSC (2010): Primary 62J07; Secondary 62G09, 62E20
DOI: https://doi.org/10.1090/S0002-9939-2010-10474-4
Published electronically: July 9, 2010
MathSciNet review: 2680074
Full-text PDF Free Access

Abstract | References | Similar Articles | Additional Information

Abstract: In this article, we derive the asymptotic distribution of the bootstrapped Lasso estimator of the regression parameter in a multiple linear regression model. It is shown that under some mild regularity conditions on the design vectors and the regularization parameter, the bootstrap approximation converges weakly to a random measure. The convergence result rigorously establishes a previously known heuristic formula for the limit distribution of the bootstrapped Lasso estimator. It is also shown that when one or more components of the regression parameter vector are zero, the bootstrap may fail to be consistent.


References [Enhancements On Off] (What's this?)

  • [AL06] Krishna B. Athreya and Soumendra N. Lahiri.
    Measure theory and probability theory.
    Springer Texts in Statistics. Springer, New York, 2006. MR 2247694
  • [BRR86] R. N. Bhattacharya and R. Ranga Rao.
    Normal approximation and asymptotic expansions.
    Robert E. Krieger Publishing Co., Inc., Melbourne, FL, 1986.
    Reprint of the 1976 original. MR 0855460
  • [Fre81] D. A. Freedman.
    Bootstrapping regression models.
    Ann. Statist., 9(6):1218-1228, 1981. MR 0630104
  • [Kal86] Olav Kallenberg.
    Random measures.
    Akademie-Verlag, Berlin, fourth edition, 1986. MR 0854102
  • [KP90] JeanKyung Kim and David Pollard.
    Cube root asymptotics.
    Ann. Statist., 18(1):191-219, 1990. MR 1041391
  • [KF00] Keith Knight and Wenjiang Fu.
    Asymptotics for lasso-type estimators.
    Ann. Statist., 28(5):1356-1378, 2000. MR 1805787
  • [LP08] Hannes Leeb and Benedikt M. Pötscher.
    Sparse estimators and the oracle property, or the return of Hodges' estimator.
    J. Econometrics, 142(1):201-211, 2008. MR 2394290
  • [PL09] Benedikt M. Pötscher and Hannes Leeb.
    On the distribution of penalized maximum likelihood estimators: the LASSO, SCAD, and thresholding.
    J. Multivariate Anal., 100(9):2065-2082, 2009. MR 2543087
  • [Tib96] Robert Tibshirani.
    Regression shrinkage and selection via the lasso.
    J. Roy. Statist. Soc. Ser. B, 58(1):267-288, 1996. MR 1379242
  • [vdVW96] Aad W. van der Vaart and Jon A. Wellner.
    Weak convergence and empirical processes.
    With applications to statistics.
    Springer Series in Statistics. Springer-Verlag, New York, 1996. MR 1385671
  • [Wai06] M. J. Wainwright,
    Sharp thresholds for high-dimensional and noisy recovery of sparsity.

    arXiv:math/0605740v1 (2006).
  • [ZY06] Peng Zhao and Bin Yu.
    On model selection consistency of Lasso.
    J. Mach. Learn. Res., 7:2541-2563, 2006. MR 2274449
  • [Zou06] Hui Zou.
    The adaptive lasso and its oracle properties.
    J. Amer. Statist. Assoc., 101(476):1418-1429, 2006. MR 2279469

Similar Articles

Retrieve articles in Proceedings of the American Mathematical Society with MSC (2010): 62J07, 62G09, 62E20

Retrieve articles in all journals with MSC (2010): 62J07, 62G09, 62E20


Additional Information

A. Chatterjee
Affiliation: Department of Statistics, Texas A&M University, College Station, Texas 77843-3143
Email: cha@stat.tamu.edu

S. N. Lahiri
Affiliation: Department of Statistics, Texas A&M University, College Station, Texas 77843-3143
Email: snlahiri@stat.tamu.edu

DOI: https://doi.org/10.1090/S0002-9939-2010-10474-4
Keywords: Consistency, bootstrap, penalized regression, random measure
Received by editor(s): January 22, 2009
Received by editor(s) in revised form: December 23, 2009, and March 2, 2010
Published electronically: July 9, 2010
Additional Notes: This research was partially supported by NSF grant DMS-0707139.
Communicated by: Edward C. Waymire
Article copyright: © Copyright 2010 American Mathematical Society

American Mathematical Society