Hostname: page-component-77c89778f8-vpsfw Total loading time: 0 Render date: 2024-07-21T09:17:59.371Z Has data issue: false hasContentIssue false
  • English
  • Français

SHRINKAGE ESTIMATION FOR NEARLY SINGULAR DESIGNS

Published online by Cambridge University Press:  30 November 2007

Keith Knight
Keith Knight
Affiliation:
University of Toronto
Get access Rights & Permissions [Opens in a new window]

Abstract

Shrinkage estimation procedures such as ridge regression and the lasso have been proposed for stabilizing estimation in linear models when high collinearity exists in the design. In this paper, we consider asymptotic properties of shrinkage estimators in the case of “nearly singular” designs.I thank Hannes Leeb and Benedikt Pötscher and also the referees for their valuable comments. This research was supported by a grant from the Natural Sciences and Engineering Research Council of Canada.

Type
Research Article
Information
Econometric Theory , Volume 24 , Issue 2 , April 2008 , pp. 323 - 337
Copyright
© 2008 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Barnabani, M. (2002) Wald-Based Approach with Singular Information Matrix. Working paper 2002/13, Department of Statistics, University of Florence.
Caner, M. (2004) Nearly Singular Design in GMM and Generalized Empirical Likelihood Estimators. Working paper, Department of Economics, University of Pittsburgh.
Caner, M. (2006) Lasso Type GMM Estimator. Working paper, Department of Economics, University of Pittsburgh.
Chernozhukov, V. (2005) Extremal quantile regression. Annals of Statistics 33, 806–839.Google Scholar
Chernozhukov, V. & H. Hong (2004) Likelihood estimation and inference in a class of nonregular econometric models. Econometrica 72, 1445–1480.Google Scholar
Fan, J. & R. Li (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360.Google Scholar
Frank, I.E. & J.H. Friedman (1993) A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109–148.Google Scholar
Fu, W.J. (1998) Penalized regressions: The bridge versus the lasso. Journal of Computational and Graphical Statistics 7, 397–416.Google Scholar
Gabaix, X. & R. Ibragimov (2006) Log(rank − 1/2): A Simple Way to Improve the OLS Estimation of Tail Exponents. Working paper, Harvard Institute of Economic Research.
Geyer, C.J. (1994) On the asymptotics of constrained M-estimation. Annals of Statistics 22, 1993–2010.Google Scholar
Geyer, C.J. (1996) On the Asymptotics of Convex Stochastic Optimization. Technical report, Department of Statistics, University of Minnesota.
Hoerl, A.E. & R.W. Kennard (1970) Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 12, 55–67.Google Scholar
Knight, K. (1999) Epi-convergence in Distribution and Stochastic Equi-semicontinuity. Unpublished manuscript, Department of Statistics, University of Toronto.
Knight, K. & W. Fu (2000) Asymptotics for lasso-type estimators. Annals of Statistics 28, 1356–1378.Google Scholar
Leeb, H. & B. Pötscher (2006) Performance limits for estimators of the risk or distribution of shrinkage-type estimators, and some general lower risk-bound results. Econometric Theory 22, 69–97.Google Scholar
Lorber, A., L.E. Wanger, & B.R. Kowalski (1987) A theoretical foundation for the PLS algorithm. Journal of Chemometrics 1, 19–31.Google Scholar
Pflug, G.C. (1995) Asymptotic stochastic programs. Mathematics of Operations Research 20, 769–789.Google Scholar
Phillips, P.C.B. (2001) Regression with Slowly Varying Regressors. Cowles Foundation Discussion paper 1310, Yale University.
Radchenko, P. (2004) Reweighting the Lasso. Unpublished manuscript, Department of Statistics, University of Chicago.
Rotnitzky, A., D.R. Cox, M. Bottai, & J. Robins (2000) Likelihood-based inference with singular information matrix. Bernoulli 6, 243–284.Google Scholar
Srivastava, M.S. (1971) On fixed width confidence bounds for regression parameters. Annals of Mathematical Statistics 42, 1403–1411.Google Scholar
Stock, J.H., J.H. Wright, & M. Yogo (2002) A survey of weak instruments and weak identification in generalized method of moments. Journal of Business & Economic Statistics 20, 518–529.Google Scholar
Stone, M. & R.J. Brooks (1990) Continuum regression: Cross-validated sequentially constructed prediction embracing ordinary least squares, partial least squares and principal components regression (with discussion). Journal of the Royal Statistical Society, Series B 52, 237–269; corrigendum (1992) 54, 906–907.Google Scholar
Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 58, 267–288.Google Scholar
van der Vaart, A.W. & J.A. Wellner (1996) Weak Convergence and Empirical Processes with Applications to Statistics. Springer.
Wold, H. (1984) PLS regression. In N.L. Johnson & S. Kotz (eds.), Encyclopedia of Statistical Sciences, vol. 6, pp. 581–591. Wiley.