Abstract
Sliced Inverse Regression (SIR) is an effective method for dimension reduction in high-dimensional regression problems. The original method, however, requires the inversion of the predictors covariance matrix. In case of collinearity between these predictors or small sample sizes compared to the dimension, the inversion is not possible and a regularization technique has to be used. Our approach is based on a Fisher Lecture given by R.D. Cook where it is shown that SIR axes can be interpreted as solutions of an inverse regression problem. We propose to introduce a Gaussian prior distribution on the unknown parameters of the inverse regression problem in order to regularize their estimation. We show that some existing SIR regularizations can enter our framework, which permits a global understanding of these methods. Three new priors are proposed leading to new regularizations of the SIR method. A comparison on simulated data as well as an application to the estimation of Mars surface physical properties from hyperspectral images are provided.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amato, U., Antoniadis, A., De Feiss, I.: Dimension reduction in functional regression with applications. Comput. Stat. Data Anal. 50, 2422–2446 (2006)
Antoniadis, A., Grégoire, G., McKeague, I.W.: Bayesian estimation in single-index models. Stat. Sin. 14, 1147–1164 (2004)
Aragon, Y., Saracco, J.: Sliced inverse regression: an appraisal of small sample alternatives to slicing. Comput. Stat. 12, 109–130 (1997)
Bernard-Michel, C., Gardes, L., Girard, S.: Estimation of Mars surface physical properties from hyperspectral images using Sliced Inverse Regression. Technical Report, INRIA. http://hal.inria.fr/inria-00187444 (2007)
Bernard-Michel, C., Gardes, L., Girard, S.: A note on Sliced Inverse Regression with regularizations. Biometrics (2008, to appear)
Chiaromonte, F., Martinelli, J.: Dimension reduction strategies for analyzing global gene expression data with a response. Math. Biosci. 176, 123–144 (2002)
Cook, R.D., Weisberg, S.: Discussion of “Sliced Inverse Regression for dimension reduction” by K.C. Li. J. Am. Stat. Assoc. 86, 328–332 (1991)
Cook, R.D.: Regression Graphics. Ideas for Studying Regressions Through Graphics. Wiley Series in Probability and Statistics. Wiley, New York (1998)
Cook, R.D.: Testing predictor contributions in sufficient dimension reduction. Ann. Stat. 32, 1062–1092 (2004)
Cook, R.D.: Fisher lecture: Dimension reduction in regression. Stat. Sci. 22(1), 1–26 (2007)
Cook, R.D., Ni, L.: Sufficient dimension reduction via inverse regression: A minimum discrepancy approach. J. Am. Stat. Assoc. 100, 410–428 (2005)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press, Cambridge (2000)
Douté, S., Schmitt, B., Langevin, Y., Bibring, J.-P., Altieri, F., Bellucci, G., Gondet, B., Poulet, F.: South pole of Mars: Nature and composition of the icy terrains from Mars Express OMEGA observations. Planet. Space Sci. 55(1–2), 113–133 (2007)
Draper, N.R., Smith, R.: Applied Regression Analysis, 3rd edn. Wiley, New York (1998)
Ferré, L.: Determining the dimension in Sliced Inverse Regression and related methods. J. Am. Stat. Assoc. 93, 132–140 (1998)
Ferré, L., Yao, A.F.: Smoothed functional inverse regression. Stat. Sin. 15, 665–683 (2005)
Friedman, J.H.: Regularized discriminant analysis. J. Am. Stat. Assoc. 84, 165–175 (1989)
Gannoun, A., Saracco, J.: An asymptotic theory for SIR α method. Stat. Sin. 13, 297–310 (2003)
Heiberger, R.: Generation of random orthogonal matrices. J. R. Stat. Soc. Ser. C 27, 199–206 (1978)
Hsing, T., Carroll, R.J.: An asymptotic theory for Sliced Inverse Regression. Ann. Stat. 20(2), 1040–1061 (1992)
Li, K.C.: Sliced Inverse Regression for dimension reduction. J. Am. Stat. Assoc. 86, 316–327 (1991)
Li, L., Li, H.: Dimension reduction methods for micro-arrays with application to censored survival data. Bioinformatics 20(18), 3406–3412 (2004)
Li, L., Yin, X.: Sliced Inverse Regression with regularizations. Biometrics 64(1), 124–131 (2008)
Saracco, J.: An asymptotic theory for Sliced Inverse Regression. Commun. Stat. Theory Methods 26, 2141–2171 (1997)
Saracco, J.: Asymptotics for pooled marginal slicing estimator based on SIR α approach. J. Multivar. Anal. 96(1), 117–135 (2005)
Schmidt, F., Douté, S., Schmitt, B.: Wavanglet: an efficient supervised classifier for hyperspectral images. Geosci. Remote Sens. IEEE Trans. 45(5), 1374–1385 (2007)
Schott, J.R.: Determining the dimensionality in Sliced Inverse Regression. J. Am. Stat. Assoc. 89, 141–148 (1994)
Scrucca, L.: Regularized Sliced Inverse Regression with applications in classification. In: Zani, S., Cerioli, A., Riani, M., Vichi, M. (eds.) Data Analysis, Classification and the Forward Search, pp. 59–66. Springer, Berlin (2006)
Scrucca, L.: Class prediction and gene selection for DNA microarrays using regularized Sliced Inverse Regression. Comput. Stat. Data Anal. 52, 438–451 (2007)
Vogel, C.R.: Computational Methods for Inverse Problems. SIAM, Philadelphia (2002)
Zhong, W., Zeng, P., Ma, P., Liu, J.S., Zhu, Y.: RSIR: Regularized Sliced Inverse Regression for motif discovery. Bioinformatics 21(22), 4169–4175 (2005)
Zhu, L.X., Ng, K.W.: Asymptotics of Sliced Inverse Regression. Stat. Sin. 5, 727–736 (1995)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Bernard-Michel, C., Gardes, L. & Girard, S. Gaussian Regularized Sliced Inverse Regression. Stat Comput 19, 85–98 (2009). https://doi.org/10.1007/s11222-008-9073-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11222-008-9073-z