Hostname: page-component-8448b6f56d-wq2xx Total loading time: 0 Render date: 2024-04-19T00:24:40.505Z Has data issue: false hasContentIssue false
  • English
  • Français

Model selection and estimation of a component in additive regression

Published online by Cambridge University Press:  28 November 2013

Xavier Gendre
Xavier Gendre*
Affiliation:
Institut de Mathématiques de Toulouse, Équipe de Statistique et Probabilités, Université Paul Sabatier, 31000 Toulouse, France. xavier.gendre@math.univ-toulouse.fr
Get access

Abstract

Let Y ∈ ℝn be a random vector with mean s and covariance matrix σ2PntPn where Pn is some known n × n-matrix. We construct a statistical procedure to estimate s as well as under moment condition on Y or Gaussian hypothesis. Both cases are developed for known or unknown σ2. Our approach is free from any prior assumption on s and is based on non-asymptotic model selection methods. Given some linear spaces collection {Sm, m ∈ ℳ}, we consider, for any m ∈ ℳ, the least-squares estimator ŝm of s in Sm. Considering a penalty function that is not linear in the dimensions of the Sm’s, we select some m̂ ∈ ℳ in order to get an estimator ŝm̂ with a quadratic risk as close as possible to the minimal one among the risks of the ŝm’s. Non-asymptotic oracle-type inequalities and minimax convergence rates are proved for ŝm̂. A special attention is given to the estimation of a non-parametric component in additive models. Finally, we carry out a simulation study in order to illustrate the performances of our estimators in practice.

Keywords

Model selectionnonparametric regressionpenalized criterionoracle inequalitycorrelated dataadditive regressionminimax rate
Type
Research Article
Information
ESAIM: Probability and Statistics , Volume 18 , 2014 , pp. 77 - 116
Copyright
© EDP Sciences, SMAI, 2013

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akaike, H., Statistical predictor identification. Ann. Inst. Stat. Math. 22 (1970) 203–217. Google Scholar
S. Arlot, Choosing a penalty for model selection in heteroscedastic regression. Preprint arXiv:0812.3141v2 (2010).
Arlot, S. and Massart, P., Data-driven calibration of penalties for least-squares regression. J. Machine Learn. Research 10 (2009) 245–279. Google Scholar
Baraud, Y., Model selection for regression on a fixed design. Probab. Theory Related Fields 117 (2000) 467–493. Google Scholar
Baraud, Y., Model selection for regression on a random design. ESAIM: Probab. Statist. 6 (2002) 127–146. Google Scholar
Baraud, Y., Comte, F. and Viennet, G., Adaptive estimation in autoregression or β-mixing regression via model selection. Ann. Stat. 29 (2001) 839–875. Google Scholar
L. Birgé and P. Massart, From model selection to adaptive estimation. Festschrift for Lucien Lecam: Research Papers in Probab. Stat. (1997) 55–87.
Birgé, L. and Massart, P., Minimum contrast estimators on sieves: exponential bounds and rates of convergence. Bernoulli 4 (1998) 329–375. Google Scholar
Birgé, L. and Massart, P., An adaptive compression algorithm in Besov spaces. Constr. Approx. 16 (2000) 1–36. Google Scholar
Birgé, L. and Massart, P.. Gaussian model selection. J. Europ. Math. Soc. 3 (2001) 203–268. Google Scholar
Birgé, L. and Massart, P., Minimal penalties for gaussian model selection. Probab. Theory Related Fields 138 (2007) 33–73. Google Scholar
Breiman, L. and Friedman, J.H., Estimating optimal transformations for multiple regression and correlations (with discussion). J. Amer. Stat. Assoc. 80 (1985) 580–619. Google Scholar
Brunel, E. and Comte, F., Adaptive nonparametric regression estimation in presence of right censoring. Math. Methods Stat. 15 (2006) 233–255. Google Scholar
E. Brunel and F. Comte, Model selection for additive regression models in the presence of censoring, chapt. 1 in “Mathematical Methods in Survival Analysis, Reliability and Quality of Life”, Wiley (2008) 17–31.
Buja, A., Hastie, T.J. and Tibshirani, R.J., Linear smoothers and additive models (with discussion). Ann. Stat. 17 (1989) 453–555. Google Scholar
Comte, F. and Rozenholc, Y., Adaptive estimation of mean and volatility functions in (auto-)regressive models. Stoch. Process. Appl. 97 (2002) 111–145. Google Scholar
Gendre, X., Simultaneous estimation of the mean and the variance in heteroscedastic gaussian regression. Electron. J. Stat. 2 (2008) 1345–1372. Google Scholar
W. Härdle, M. Müller, S. Sperlich and A. Werwatz. Nonparametric and Semiparametric Models. Springer (2004).
T.J. Hastie and R.J. Tibshirani, Generalized additive models. Chapman and Hall (1990).
R.A. Horn and C.R. Johnson, Matrix analysis. Cambridge University Press (1990).
Laurent, B., Loubes, J.M. and Marteau, C., Testing inverse problems: a direct or an indirect problem? J. Stat. Plann. Inference 141 (2011) 1849–1861. Google Scholar
Laurent, B. and Massart, P., Adaptive estimation of a quadratic functional by model selection. Ann. Stat. 28 (2000) 1302–1338. Google Scholar
Leontief, W., Introduction to a theory of the internal structure of functional relationships. Econometrica 15 (1947) 361–373. Google Scholar
Linton, O. and Nielsen, J.P., A kernel method of estimating structured nonparametric regression based on marginal integration. Biometrika 82 (1995) 93–101. Google Scholar
Mallows, C.L., Some comments on c p. Technometrics 15 (1973) 661–675,. Google Scholar
Mammen, E., Linton, O. and Nielsen, J.P., The existence and asymptotic properties of a backfitting projection algorithm under weak conditions. Ann. Stat. 27 (1999) 1443–1490. Google Scholar
P. Massart, Concentration inequalities and model selection, in vol. 1896 of Lect. Notes Math. Lectures from the 33rd Summer School on Probability Theory held in Saint-Flour, July 6-23 (2003). Springer, Berlin (2007).
A.D.R. McQuarrie and C.L. Tsai, Regression and times series model selection. River Edge, NJ (1998).
Meier, L., S., van de Geer and P., Bühlmann, High-dimensional additive modeling. Ann. Stat. 37 (2009) 3779–3821. Google Scholar
Opsomer, J. and Ruppert, D., Fitting a bivariate additive model by local polynomial regression. Ann. Stat. 25 (1997) 186–211. Google Scholar
V.V. Petrov, Limit theorems of probability theory: sequences of independent random variables. Oxford Studies Probab. 4 (1995).
Ravikumar, P.D., Liu, H., Lafferty, J.D. and Wasserman, L.A., Sparse additive models. J. Royal Statist. Soc. 71 (2009) 1009–1030. Google Scholar
S. Robin, F. Rodolphe and S. Schbath, DNA, Words and Models. Cambridge University Press (2005).
Ruppert, D. and Wand, M.P., Multivariate locally weighted least squares regression. Ann. Stat. 22 (1994) 1346–1370. Google Scholar
H. Scheffé, The analysis of variance. Wiley-Interscience (1959).
Severance-Lossin, E. and Sperlich, S., Estimation of derivatives for additive separable models. Statististics 33 (1999) 241–265. Google Scholar
Stone, C.J., Additive regression and other nonparametric models. Ann. Stat. 14 (1985) 590–606. Google Scholar
Tjøstheim, D. and Auestad, B., Nonparametric identification of nonlinear time series: Selecting significant lags. J. Amer. Stat. Assoc. 89 (1994) 1410–1430. Google Scholar
von Bahr, B. and Esseen, C.G., Inequalities for the rth absolute moment of a sum of random variables 1 ≤ r ≤ 2 . Ann. Math. Stat. 36 (1965) 299–303. Google Scholar