Skip to main content Accessibility help
×
Home

INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS

Published online by Cambridge University Press:  04 September 2018


Xinyu Zhang
Affiliation:
Chinese Academy of Sciences Qingdao University
Chu-An Liu
Affiliation:
Academia Sinica
Corresponding
E-mail address:

Abstract

This article considers the problem of inference for nested least squares averaging estimators. We study the asymptotic behavior of the Mallows model averaging estimator (MMA; Hansen, 2007) and the jackknife model averaging estimator (JMA; Hansen and Racine, 2012) under the standard asymptotics with fixed parameters setup. We find that both MMA and JMA estimators asymptotically assign zero weight to the under-fitted models, and MMA and JMA weights of just-fitted and over-fitted models are asymptotically random. Building on the asymptotic behavior of model weights, we derive the asymptotic distributions of MMA and JMA estimators and propose a simulation-based confidence interval for the least squares averaging estimator. Monte Carlo simulations show that the coverage probabilities of proposed confidence intervals achieve the nominal level.


Type
ARTICLES
Copyright
Copyright © Cambridge University Press 2018 

Access options

Get access to the full version of this content by using one of the access options below.

Footnotes

We thank three anonymous referees, the co-editor Liangjun Su, and the editor Peter C.B. Phillips for many constructive comments and suggestions. We also thank conference participants of SETA 2016, AMES 2016, and CFE 2017 for their discussions and suggestions. Xinyu Zhang gratefully acknowledges the research support from National Natural Science Foundation of China (Grant numbers 71522004, 11471324 and 71631008). Chu-An Liu gratefully acknowledges the research support from the Ministry of Science and Technology of Taiwan (MOST 104-2410-H-001-092-MY2). All errors remain the authors’.


References

Andrews, D.W.K. (1991) Asymptotic optimality of generalized CL, cross-validation, and generalized cross-validation in regression with heteroskedastic errors. Journal of Econometrics 47, 359377.10.1016/0304-4076(91)90107-OCrossRefGoogle Scholar
Breiman, L. (1996) Bagging predictors. Machine Learning 24, 123140.10.1007/BF00058655CrossRefGoogle Scholar
Buckland, S.T., Burnham, K.P., & Augustin, N.H. (1997) Model selection: An integral part of inference. Biometrics 53, 603618.10.2307/2533961CrossRefGoogle Scholar
Camponovo, L. (2015) On the validity of the pairs bootstrap for lasso estimators. Biometrika 102(4), 981987.10.1093/biomet/asv039CrossRefGoogle Scholar
Chatterjee, A. & Lahiri, S.N. (2011) Bootstrapping lasso estimators. Journal of the American Statistical Association 106(494), 608625.10.1198/jasa.2011.tm10159CrossRefGoogle Scholar
Chatterjee, A. & Lahiri, S.N. (2013) Rates of convergence of the adaptive lasso estimators to the oracle distribution and higher order refinements by the bootstrap. The Annals of Statistics 41(3), 12321259.10.1214/13-AOS1106CrossRefGoogle Scholar
Claeskens, G. & Hjort, N.L. (2008) Model Selection and Model Averaging. Cambridge University Press.10.1017/CBO9780511790485CrossRefGoogle Scholar
DiTraglia, F. (2016) Using invalid instruments on purpose: Focused moment selection and averaging for GMM. Journal of Econometrics 195, 187208.10.1016/j.jeconom.2016.07.006CrossRefGoogle Scholar
Hansen, B.E. (2007) Least squares model averaging. Econometrica 75, 11751189.10.1111/j.1468-0262.2007.00785.xCrossRefGoogle Scholar
Hansen, B.E. (2008) Least-squares forecast averaging. Journal of Econometrics 146(2), 342350.10.1016/j.jeconom.2008.08.022CrossRefGoogle Scholar
Hansen, B.E. (2014) Model averaging, asymptotic risk, and regressor groups. Quantitative Economics 5(3), 495530.10.3982/QE332CrossRefGoogle Scholar
Hansen, B.E. (2018) Econometrics. Unpublished manuscript, University of Wisconsin.Google Scholar
Hansen, B.E. & Racine, J. (2012) Jacknife model averaging. Journal of Econometrics 167, 3846.10.1016/j.jeconom.2011.06.019CrossRefGoogle Scholar
Hansen, P., Lunde, A., & Nason, J. (2011) The model confidence set. Econometrica 79, 453497.Google Scholar
Hjort, N.L. & Claeskens, G. (2003a) Frequentist model average estimators. Journal of the American Statistical Association 98, 879899.10.1198/016214503000000828CrossRefGoogle Scholar
Hjort, N.L. & Claeskens, G. (2003b) Rejoinder to the focused information criterion and frequentist model average estimators. Journal of the American Statistical Association 98(464), 938945.10.1198/016214503000000882CrossRefGoogle Scholar
Hoeting, J.A., Madigan, D., Raftery, A.E., & Volinsky, C.T. (1999) Bayesian model averaging: A tutorial. Statistical Science 14, 382417.Google Scholar
Inoue, A. & Kilian, L. (2008) How useful is bagging in forecasting economic time series? A case study of US consumer price inflation. Journal of the American Statistical Association 103, 511522.10.1198/016214507000000473CrossRefGoogle Scholar
Kabaila, P. (1995) The effect of model selection on confidence regions and prediction regions. Econometric Theory 11(3), 537549.10.1017/S0266466600009403CrossRefGoogle Scholar
Kabaila, P. (1998) Valid confidence intervals in regression after variable selection. Econometric Theory 14(4), 463482.10.1017/S0266466698144031CrossRefGoogle Scholar
Kim, J. & Pollard, D. (1990) Cube root asymptotics. The Annals of Statistics 18, 191219.10.1214/aos/1176347498CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2003) The finite-sample distribution of post-model-selection estimators and uniform versus non-uniform approximations. Econometric Theory 19(1), 100142.10.1017/S0266466603191050CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2005) Model selection and inference: Facts and fiction. Econometric Theory 21(1), 2159.10.1017/S0266466605050036CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2006) Can one estimate the conditional distribution of post-model-selection estimators? The Annals of Statistics 34(5), 25542591.10.1214/009053606000000821CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2008) Can one estimate the unconditional distribution of post-model-selection estimators? Econometric Theory 24(02), 338376.10.1017/S0266466608080158CrossRefGoogle Scholar
Leeb, H. & Pötscher, B. (2017) Testing in the presence of nuisance parameters: Some comments on tests post-model-selection and random critical values. In Ahmed, S.E. (ed.), Big and Complex Data Analysis: Methodologies and Applications, pp. 6982. Springer International Publishing.10.1007/978-3-319-41573-4_4CrossRefGoogle Scholar
Li, K.-C. (1987) Asymptotic optimality for Cp, CL, cross-validation and generalized cross-validation: Discrete index set. The Annals of Statistics 15, 958975.10.1214/aos/1176350486CrossRefGoogle Scholar
Liang, H., Zou, G., Wan, A.T.K., & Zhang, X. (2011) Optimal weight choice for frequentist model average estimators. Journal of the American Statistical Association 106, 10531066.10.1198/jasa.2011.tm09478CrossRefGoogle Scholar
Liu, C.-A. (2015) Distribution theory of the least squares averaging estimator. Journal of Econometrics 186, 142159.10.1016/j.jeconom.2014.07.002CrossRefGoogle Scholar
Liu, Q. & Okui, R. (2013) Heteroscedasticity-robust Cp model averaging. Econometrics Journal 16, 462473.10.1111/ectj.12009CrossRefGoogle Scholar
Lu, X. (2015) A covariate selection criterion for estimation of treatment effects. Journal of Business and Economic Statistics 33, 506522.10.1080/07350015.2014.982755CrossRefGoogle Scholar
Lu, X. & Su, L. (2015) Jackknife model averaging for quantile regressions. Journal of Econometrics 188(1), 4058.10.1016/j.jeconom.2014.11.005CrossRefGoogle Scholar
Magnus, J., Powell, O., & Prüfer, P. (2010) A comparison of two model averaging techniques with an application to growth empirics. Journal of Econometrics 154(2), 139153.10.1016/j.jeconom.2009.07.004CrossRefGoogle Scholar
Moral-Benito, E. (2015) Model averaging in economics: An overview. Journal of Economic Surveys 29(1), 4675.10.1111/joes.12044CrossRefGoogle Scholar
Pötscher, B. (1991) Effects of model selection on inference. Econometric Theory 7(2), 163185.10.1017/S0266466600004382CrossRefGoogle Scholar
Pötscher, B. (2006) The distribution of model averaging estimators and an impossibility result regarding its estimation. Lecture Notes-Monograph Series 52, 113129.10.1214/074921706000000987CrossRefGoogle Scholar
Pötscher, B. & Leeb, H. (2009) On the distribution of penalized maximum likelihood estimators: The lasso, scad, and thresholding. Journal of Multivariate Analysis 100(9), 20652082.10.1016/j.jmva.2009.06.010CrossRefGoogle Scholar
Raftery, A.E. & Zheng, Y. (2003) Discussion: Performance of Bayesian model averaging. Journal of the American Statistical Association 98(464), 931938.10.1198/016214503000000891CrossRefGoogle Scholar
Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58, 267288.Google Scholar
Van der Vaart, A. & Wellner, J. (1996) Weak Convergence and Empirical Processes. Springer Verlag.10.1007/978-1-4757-2545-2CrossRefGoogle Scholar
Wan, A.T.K., Zhang, X., & Zou, G. (2010) Least squares model averaging by Mallows criterion. Journal of Econometrics 156, 277283.10.1016/j.jeconom.2009.10.030CrossRefGoogle Scholar
Yang, Y. (2000) Combining different procedures for adaptive regression. Journal of Multivariate Analysis 74(1), 135161.10.1006/jmva.1999.1884CrossRefGoogle Scholar
Yang, Y. (2001) Adaptive regression by mixing. Journal of the American Statistical Association 96, 574588.10.1198/016214501753168262CrossRefGoogle Scholar
Yuan, Z. & Yang, Y. (2005) Combining linear regression models: When and how? Journal of the American Statistical Association 100, 12021214.10.1198/016214505000000088CrossRefGoogle Scholar
Zhang, X. & Liang, H. (2011) Focused information criterion and model averaging for generalized additive partial linear models. The Annals of Statistics 39, 174200.10.1214/10-AOS832CrossRefGoogle Scholar
Zhang, X., Wan, A.T., & Zhou, S.Z. (2012) Focused information criteria, model selection, and model averaging in a Tobit model with a nonzero threshold. Journal of Business and Economic Statistics 30, 132142.10.1198/jbes.2011.10075CrossRefGoogle Scholar
Zhang, X., Wan, A.T.K., & Zou, G. (2013) Model averaging by jackknife criterion in models with dependent data. Journal of Econometrics 174, 8294.10.1016/j.jeconom.2013.01.004CrossRefGoogle Scholar
Zhang, X., Zou, G., & Liang, H. (2014) Model averaging and weight choice in linear mixed-effects models. Biometrika 101, 205218.10.1093/biomet/ast052CrossRefGoogle Scholar
Zou, H. (2006) The adaptive lasso and its oracle properties. Journal of the American Statistical Association 101, 14181429.10.1198/016214506000000735CrossRefGoogle Scholar

Full text views

Full text views reflects PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.

Total number of HTML views: 0
Total number of PDF views: 425 *
View data table for this chart

* Views captured on Cambridge Core between 04th September 2018 - 3rd December 2020. This data will be updated every 24 hours.

Hostname: page-component-b4dcdd7-zcwv2 Total loading time: 0.327 Render date: 2020-12-03T16:44:29.862Z Query parameters: { "hasAccess": "0", "openAccess": "0", "isLogged": "0", "lang": "en" } Feature Flags last update: Thu Dec 03 2020 15:58:41 GMT+0000 (Coordinated Universal Time) Feature Flags: { "metrics": true, "metricsAbstractViews": false, "peerReview": true, "crossMark": true, "comments": true, "relatedCommentaries": true, "subject": true, "clr": false, "languageSwitch": true }

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

INFERENCE AFTER MODEL AVERAGING IN LINEAR REGRESSION MODELS
Available formats
×
×

Reply to: Submit a response


Your details


Conflicting interests

Do you have any conflicting interests? *