Abstract
The learning approach of empirical risk minimization (ERM) is taken for the regression problem in the least square framework. A standard assumption for the error analysis in the literature is the uniform boundedness of the output sampling process. In this paper we abandon this boundedness assumption and conduct error analysis for the ERM learning algorithm with unbounded sampling processes satisfying an increment condition for the moments of the output. The key novelty of our analysis is a covering number argument for estimating the sample error.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cucker, F., Smale, S.: On the mathematical foundations of learning. Bull. Amer. Math. Soc., 39, 1–49 (2002)
Capponnetto, A., De Vito, E.: Optimal rates for the regularized least-squares algorithm. Found. Comput. Math., 7, 331–368 (2007)
Wang, C., Zhou, D. X.: Least square regularized regression with unbounded sampling. J. Complex., accepted, doi: 10.1016/j.jco.2010.10.002
Smale, S., Zhou, D. X.: Estimating the approximation error in learing theory. Anal. Appl., 1, 17–41 (2003)
Bennett, G.: Probability inequalities for the sum of independent random variables. J. Amer. Stat. Assoc., 57, 33–45 (1962)
Cucker, F., Zhou, D. X.: Learning Theory: An Approximation Theory Viewpoint, Cambridge University Press, Cambridge, 2007
De Vito, E., Caponnetto, A., Rosasco, L.: Model selection for regularized least-squares algorithm in learning theory. Found. Comput. Math., 5, 59–85 (2005)
Smale, S., Zhou, D. X.: Learning theory estimates via integral operators and their approximations. Constr. Approx., 26, 153–172 (2007)
Wu, Q., Ying, Y., Zhou, D. X.: Learning rates of least-square regularized regression. Found. Comput. Math., 6, 171–192 (2006)
Wu, Q., Ying, Y., Zhou, D. X.: Multi-kernel regularized classifiers. J. Complexity, 23, 108–134 (2007)
Wu, Q., Zhou, D. X.: Learning with sample dependent hypothesis spaces. Comput. Math. Appl., 56, 2896–2907 (2008)
Zhang, T.: Leave-one-out bounds for kernel methods. Neural Comut., 15, 1397–1437 (2003)
Zhou, D. X.: The covering number in learning thoery. J. Complexity, 18, 739–767 (2002)
Zhou, D. X.: Capacity of reproducing kernel spaces in learning theory. IEEE Trans. Inform. Theory, 49, 1743–1752 (2003)
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported by the Research Grants Council of Hong Kong (Grant No. 103508)
Electronic supplementary material
Rights and permissions
About this article
Cite this article
Wang, C., Guo, Z.C. ERM learning with unbounded sampling. Acta. Math. Sin.-English Ser. 28, 97–104 (2012). https://doi.org/10.1007/s10114-012-9739-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10114-012-9739-5