Abstract
We consider the problem of reconstructing an unknown function f on a domain X from samples of f at n randomly chosen points with respect to a given measure ρ X . Given a sequence of linear spaces (V m ) m>0 with dim(V m )=m≤n, we study the least squares approximations from the spaces V m . It is well known that such approximations can be inaccurate when m is too close to n, even when the samples are noiseless. Our main result provides a criterion on m that describes the needed amount of regularization to ensure that the least squares method is stable and that its accuracy, measured in L 2(X,ρ X ), is comparable to the best approximation error of f by elements from V m . We illustrate this criterion for various approximation schemes, such as trigonometric polynomials, with ρ X being the uniform measure, and algebraic polynomials, with ρ X being either the uniform or Chebyshev measure. For such examples we also prove similar stability results using deterministic samples that are equispaced with respect to these measures.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Change history
20 September 2018
The inequality on line 8 of page 826 is stated in the wrong direction
Notes
While such a basis is generally not accessible when ρ X is unknown, we require it only for the analysis. The actual computation of the estimator can be made using any known basis of V m , since the solution w is independent of the basis used in computing it.
References
R. Ahlswede, A. Winter, Strong converse for identification via quantum channels, IEEE Trans. Inf. Theory 48, 569–579 (2002).
Y. Baraud, Model selection for regression on a random design, ESAIM Probab. Stat. 6, 127–146 (2002).
L. Birgé, P. Massart, Minimum contrast estimators on sieves: exponential bounds and rates of convergence, Bernoulli 4, 329–375 (1998).
A. Cohen, R. DeVore, C. Schwab, Analytic regularity and polynomial approximation of parametric elliptic PDE’s, J. Anal. Appl. 9, 11–47 (2011).
W. Gander, W. Gautschi, Adaptive quadrature—revisited, BIT Numer. Math. 40, 84–101 (2000).
L. Györfi, M. Kohler, A. Krzyzak, H. Walk, A Distribution-Free Theory of Nonparametric Regression (Springer, Berlin, 2002).
G. Migliorati, F. Nobile, E. von Schweriny, R. Tempone, Analysis of the point collocation method. Preprint, MOX, Politecnico di Milano (2011).
J. Tropp, User friendly tail bounds for sums of random matrices, Found. Comput. Math. 12, 389–434 (2012).
Acknowledgements
This research has been partially supported by the ANR Defi08 “ECHANGE” and by the US NSF grant DMS-1004718. Portions of this work were completed while M.A.D. and D.L. were visitors at University Pierre et Marie Curie.
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Felipe Cucker.
Rights and permissions
About this article
Cite this article
Cohen, A., Davenport, M.A. & Leviatan, D. On the Stability and Accuracy of Least Squares Approximations. Found Comput Math 13, 819–834 (2013). https://doi.org/10.1007/s10208-013-9142-3
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10208-013-9142-3