Hostname: page-component-7bb8b95d7b-cx56b Total loading time: 0 Render date: 2024-09-19T22:32:05.938Z Has data issue: false hasContentIssue false
  • English
  • Français

PERFORMANCE LIMITS FOR ESTIMATORS OF THE RISK OR DISTRIBUTION OF SHRINKAGE-TYPE ESTIMATORS, AND SOME GENERAL LOWER RISK-BOUND RESULTS

Published online by Cambridge University Press:  12 December 2005

Hannes Leeb  and
Benedikt M. Pötscher
Hannes Leeb
Affiliation:
Yale University
Benedikt M. Pötscher
Affiliation:
University of Vienna
Get access Rights & Permissions [Opens in a new window]

Abstract

We consider the problem of estimating measures of precision of shrinkage-type estimators such as their risk or distribution. The notion of shrinkage-type estimators here refers to estimators such as the James–Stein estimator and Lasso-type estimators, in addition to “thresholding” estimators such as, e.g., Hodges' so-called superefficient estimator. Although the precision measures of such estimators typically can be estimated consistently, we show that they cannot be estimated uniformly consistently (even locally). This follows as a corollary to (locally) uniform lower bounds on the performance of estimators of the precision measures that we obtain in the paper. These lower bounds are typically quite large (e.g., they approach ½ or 1 depending on the situation considered). The analysis is based on some general lower risk bounds and related general results on the (non)existence of uniformly consistent estimators also obtained in the paper.A preliminary draft of the results in Section 3 of this paper was written in 1999.

Type
Research Article
Information
Econometric Theory , Volume 22 , Issue 1 , February 2006 , pp. 69 - 97
Copyright
© 2006 Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

REFERENCES

Adkins, L.C. (1990) Small sample performance of jackknife confidence intervals for the James-Stein estimator. Communications in Statistics: Simulation and Computation 19, 401–418.CrossRefGoogle Scholar
Adkins, L.C. (1992) Finite sample moments of a bootstrap estimator of the James-Stein rule. Econometric Reviews 11, 173–193.CrossRefGoogle Scholar
Bauer, P., B.M. Pötscher, & P. Hackl (1988) Model selection by multiple test procedures. Statistics 19, 39–44.CrossRefGoogle Scholar
Chen, J. (1997) A general lower bound of minimax risk for absolute-error loss. Canadian Journal of Statistics 25, 545–558.CrossRefGoogle Scholar
Chen, S.S., D.L. Donoho, & M.A. Saunders (1998) Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing 20, 33–61.CrossRefGoogle Scholar
Donoho, D.L. & I.M. Johnstone (1994) Ideal spatial adaptation by wavelet shrinkage. Biometrika 81, 425–455.CrossRefGoogle Scholar
Donoho, D.L. & R.C. Liu (1987) Geometrizing rates of convergence I. Technical report, Department of Statistics, University of California, Berkeley.
Donoho, D.L. & R.C. Liu (1991) Geometrizing rates of convergence II. Annals of Statistics 19, 633–667.CrossRefGoogle Scholar
Fan, J. & R. Li (2001) Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association 96, 1348–1360.CrossRefGoogle Scholar
Frank, I.E. & J. Friedman (1993) A statistical view of some chemometrics regression tools (with discussion). Technometrics 35, 109–148.CrossRefGoogle Scholar
George, E.I. (1986) Minimax multiple shrinkage estimation. Annals of Statistics 14, 188–205.CrossRefGoogle Scholar
Geyer, C.J. (1996) On the Asymptotics of Convex Stochastic Optimization. Manuscript, University of Minnesota.
Huber, C. (1997) Lower bounds for function estimation. In D. Pollard, E. Torgersen, & G.L. Yang (eds.), Festschrift for Lucien Le Cam: Research Papers in Probability and Statistics, pp. 245–258. Springer.
Jennrich, R.I. & S.D. Oman (1986) How much does Stein estimation help in multiple linear regression? Technometrics 28, 113–121.Google Scholar
Judge, G.G. & M.E. Bock (1978) The Statistical Implications of Pre-Test and Stein-Rule Estimators in Econometrics. North-Holland/Elsevier.
Knight, K. & W. Fu (2000) Asymptotics of Lasso-type estimators. Annals of Statistics 28, 1356–1378.Google Scholar
Kubokawa, T. & M.S. Srivastava (2002) Estimating risk and the mean squared error matrix in Stein estimation. Journal of Multivariate Analysis 82, 39–64.CrossRefGoogle Scholar
Le Cam, L. (1973) Convergence of estimates under dimensionality restrictions. Annals of Statistics 1, 38–53.CrossRefGoogle Scholar
Le Cam, L. & L. Schwartz (1960) A necessary and sufficient condition for the existence of consistent estimators. Annals of Mathematical Statistics 31, 140–150.Google Scholar
Leeb, H. & B.M. Pötscher (2003) Can One Estimate the Conditional Distribution of Post-Model-Selection Estimators? Manuscript, Department of Statistics, University of Vienna.
Leeb, H. & B.M. Pötscher (2005) Model selection and inference: Facts and fiction. Econometric Theory 21, 21–59.CrossRefGoogle Scholar
Lu, K.L. & J.O. Berger (1989) Estimation of normal means: Frequentist estimation of loss. Annals of Statistics 17, 890–906.CrossRefGoogle Scholar
Pfanzagl, J. (1998) The nonexistence of confidence sets for discontinuous functionals. Journal of Statistical Planning and Inference 75, 9–20.CrossRefGoogle Scholar
Pötscher, B.M. (1991) Effects of model selection on inference. Econometric Theory 7, 163–185.CrossRefGoogle Scholar
Pötscher, B.M. (2002) Lower risk bounds and properties of confidence sets for ill-posed estimation problems with applications to spectral density and persistence estimation, unit roots, and estimation of long memory parameters. Econometrica 70, 1035–1065.CrossRefGoogle Scholar
Sen, P.K. (1986) Whither jackknifing in Stein-rule estimation? Communications in Statistics: Theory and Methods A15, 2245–2266.Google Scholar
Stein, C.M. (1981) Estimation of the mean of a multivariate normal distribution. Annals of Statistics 9, 1135–1151.CrossRefGoogle Scholar
Tibshirani, R. (1996) Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society, Series B 88, 267–288.Google Scholar
van der Vaart, A.W. (1998) Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics. Cambridge University Press.
Venter, J.H. & S.J. Steel (1990) Estimating risk reduction in Stein estimation. Canadian Journal of Statistics 18, 221–232.CrossRefGoogle Scholar
Wan, A.T.K. & G. Zou (2004) On unbiased and improved loss estimation for the mean of a multivariate normal distribution with unknown variance. Journal of Statistical Planning and Inference 119, 17–22.CrossRefGoogle Scholar
Yatracos, Y.G. (1985) On the existence of uniformly consistent estimators. Proceedings of the American Mathematical Society 94, 479–486.CrossRefGoogle Scholar
Yu, B. (1997) Assouad, Fano, and Le Cam. In D. Pollard, E. Torgersen, & G.L. Yang (eds.), Festschrift for Lucien Le Cam: Research Papers in Probability and Statistics, pp. 423–435. Springer.