Skip to main content Accessibility help
×
Home
Hostname: page-component-7ccbd9845f-zxw8g Total loading time: 0.336 Render date: 2023-01-28T13:53:27.935Z Has data issue: true Feature Flags: { "useRatesEcommerce": false } hasContentIssue true

ADAPTIVE BAYESIAN ESTIMATION OF CONDITIONAL DENSITIES

Published online by Cambridge University Press:  13 July 2016

Andriy Norets*
Affiliation:
Brown University
Debdeep Pati
Affiliation:
Florida State University
*
*Address correspondence to Andriy Norets, Associate Professor, Department of Economics, Brown University, Providence, RI 02912; e-mail: andriy_norets@brown.edu.

Abstract

We consider a nonparametric Bayesian model for conditional densities. The model is a finite mixture of normal distributions with covariate dependent multinomial logit mixing probabilities. A prior for the number of mixture components is specified on positive integers. The marginal distribution of covariates is not modeled. We study asymptotic frequentist behavior of the posterior in this model. Specifically, we show that when the true conditional density has a certain smoothness level, then the posterior contraction rate around the truth is equal up to a log factor to the frequentist minimax rate of estimation. An extension to the case when the covariate space is unbounded is also established. As our result holds without a priori knowledge of the smoothness level of the true density, the established posterior contraction rates are adaptive. Moreover, we show that the rate is not affected by inclusion of irrelevant covariates in the model. In Monte Carlo simulations, a version of the model compares favorably to a cross-validated kernel conditional density estimator.

Type
ARTICLES
Copyright
Copyright © Cambridge University Press 2016 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank the editor, the co-editor, and referees for helpful comments. Dr. Pati acknowledges support for this project from the Office of Naval Research (ONR BAA 14-0001) and NSF DMS-1613156.

References

REFERENCES

Barron, A., Schervish, M.J., & Wasserman, L. (1999) The consistency of posterior distributions in nonparametric problems. The Annals of Statistics 27(2), 536561.Google Scholar
Bhattacharya, A., Pati, D., & Dunson, D. (2014) Anisotropic function estimation using multi-bandwidth Gaussian processes. The Annals of Statistics 42(1), 352381.CrossRefGoogle ScholarPubMed
Chung, Y. & Dunson, D.B. (2009) Nonparametric Bayes conditional distribution modeling with variable selection. Journal of the American Statistical Association 104(488), 16461660.CrossRefGoogle ScholarPubMed
De Iorio, M., Muller, P., Rosner, G.L., & MacEachern, S.N. (2004) An ANOVA model for dependent random measures. Journal of the American Statistical Association 99(465), 205215.CrossRefGoogle Scholar
De Jonge, R. & van Zanten, J.H. (2010) Adaptive nonparametric Bayesian inference using location-scale mixture priors. The Annals of Statistics 38(6), 33003320.CrossRefGoogle Scholar
Dunson, D.B. & Park, J.H. (2008) Kernel stick-breaking processes. Biometrika 95(2), 307323.CrossRefGoogle ScholarPubMed
Dunson, D.B., Pillai, N., & Park, J.H. (2007) Bayesian density regression. Journal of the Royal Statistical Society Series B (Statistical Methodology) 69(2), 163183.CrossRefGoogle Scholar
Efromovich, S. (2007) Conditional density estimation in a regression setting. The Annals of Statistics 35(6), 25042535.CrossRefGoogle Scholar
Geweke, J. & Keane, M. (2007) Smoothly mixing regressions. Journal of Econometrics 138, 252290.CrossRefGoogle Scholar
Ghosal, S., Ghosh, J.K., & Ramamoorthi, R.V. (1999) Posterior consistency of Dirichlet mixtures in density estimation. The Annals of Statistics 27(1), 143158.Google Scholar
Ghosal, S., Ghosh, J.K., & van der Vaart, A.W. (2000) Convergence rates of posterior distributions. The Annals of Statistics 28(2), 500531.CrossRefGoogle Scholar
Ghosal, S. & van der Vaart, A.W. (2001) Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities. The Annals of Statistics 29(5), 12331263.Google Scholar
Ghosal, S. & van der Vaart, A.W. (2007) Posterior convergence rates of Dirichlet mixtures at smooth densities. The Annals of Statistics 35(2), 697723.CrossRefGoogle Scholar
Griffin, J.E. & Steel, M.F.J. (2006) Order-based dependent Dirichlet processes. Journal of the American Statistical Association 101(473), 179194.CrossRefGoogle Scholar
Hall, P., Racine, J., & Li, Q. (2004) Cross-validation and the estimation of conditional probability densities. Journal of the American Statistical Association 99(468), 10151026.CrossRefGoogle Scholar
Hayfield, T. & Racine, J.S. (2008) Nonparametric econometrics: The np package. Journal of Statistical Software 27(5), 132.CrossRefGoogle Scholar
Huang, T.M. (2004) Convergence rates for posterior distributions and adaptive estimation. The Annals of Statistics 32(4), 15561593.CrossRefGoogle Scholar
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., & Hinton, G.E. (1991) Adaptive mixtures of local experts. Neural Computation 3(1), 7987.CrossRefGoogle Scholar
Jordan, M. & Xu, L. (1995) Convergence results for the em approach to mixtures of experts architectures. Neural Networks 8(9), 14091431.CrossRefGoogle Scholar
Keane, M. & Stavrunova, O. (2011) A smooth mixture of tobits model for healthcare expenditure. Health Economics 20(9), 11261153.CrossRefGoogle ScholarPubMed
Kruijer, W., Rousseau, J., & van der Vaart, A. (2010) Adaptive Bayesian density estimation with location-scale mixtures. Electronic Journal of Statistics 4, 12251257.CrossRefGoogle Scholar
Li, F., Villani, M., & Kohn, R. (2010) Flexible modeling of conditional distributions using smooth mixtures of asymmetric student t densities. Journal of Statistical Planning and Inference 140(12), 36383654.CrossRefGoogle Scholar
Li, Q. & Racine, J.S. (2007) Nonparametric Econometrics: Theory and Practice. Princeton University Press.Google Scholar
MacEachern, S.N. (1999) Dependent nonparametric processes. Proceedings of the Section on Bayesian Statistical Science, pp. 5055. American Statistical Association.Google Scholar
Norets, A. (2010) Approximation of conditional densities by smooth mixtures of regressions. The Annals of Statistics 38(3), 17331766.CrossRefGoogle Scholar
Norets, A. (2015) Optimal retrospective sampling for a class of variable dimension models. Unpublished manuscript, Brown University.
Norets, A. & Pelenis, J. (2012) Bayesian modeling of joint and conditional distributions. Journal of Econometrics 168, 332346.CrossRefGoogle Scholar
Norets, A. & Pelenis, J. (2014) Posterior consistency in conditional density estimation by covariate dependent mixtures. Econometric Theory 30, 606646.CrossRefGoogle Scholar
Pati, D., Dunson, D.B., & Tokdar, S.T. (2013) Posterior consistency in conditional distribution estimation. Journal of Multivariate Analysis 116, 456472.CrossRefGoogle ScholarPubMed
Peng, F., Jacobs, R.A., & Tanner, M.A. (1996) Bayesian inference in mixtures-of-experts and hierarchical mixtures-of-experts models with an application to speech recognition. Journal of the American Statistical Association 91(435), 953960.CrossRefGoogle Scholar
Rousseau, J. (2010) Rates of convergence for the posterior distributions of mixtures of betas and adaptive nonparametric estimation of the density. The Annals of Statistics 38(1), 146180.CrossRefGoogle Scholar
Scricciolo, C. (2006) Convergence rates for Bayesian density estimation of infinite-dimensional exponential families. Annals of Statatistics 34(6), 28972920.CrossRefGoogle Scholar
Shen, W. & Ghosal, S. (2016) Adaptive Bayesian density regression for high-dimensional data. Bernoulli 22(1), 396420.CrossRefGoogle Scholar
Shen, W., Tokdar, S.T., & Ghosal, S. (2013) Adaptive Bayesian multivariate density estimation with Dirichlet mixtures. Biometrika 100(3), 623640.CrossRefGoogle Scholar
Tokdar, S., Zhu, Y., & Ghosh, J. (2010) Bayesian density regression with logistic Gaussian process and subspace projection. Bayesian Analysis 5(2), 319344.CrossRefGoogle Scholar
van der Vaart, A.W. & van Zanten, J.H. (2009) Adaptive Bayesian estimation using a Gaussian random field with inverse gamma bandwidth. The Annals of Statistics 37(5B), 26552675.CrossRefGoogle Scholar
Villani, M., Kohn, R., & Giordani, P. (2009) Regression density estimation using smooth adaptive Gaussian mixtures. Journal of Econometrics 153(2), 155173.CrossRefGoogle Scholar
Villani, M., Kohn, R., & Nott, D.J. (2012) Generalized smooth finite mixtures. Journal of Econometrics 171(2), 121133.CrossRefGoogle Scholar
Wade, S., Dunson, D.B., Petrone, S., & Trippa, L. (2014) Improving prediction from Dirichlet process mixtures via enrichment. The Journal of Machine Learning Research 15(1), 10411071.Google Scholar
Wood, S., Jiang, W., & Tanner, M. (2002) Bayesian mixture of splines for spatially adaptive nonparametric regression. Biometrika 89(3), 513528.CrossRefGoogle Scholar
Yang, Y. & Tokdar, S.T. (2015) Minimax-optimal nonparametric regression in high dimensions. The Annals of Statistics 43(2), 652674.CrossRefGoogle Scholar
14
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

ADAPTIVE BAYESIAN ESTIMATION OF CONDITIONAL DENSITIES
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

ADAPTIVE BAYESIAN ESTIMATION OF CONDITIONAL DENSITIES
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

ADAPTIVE BAYESIAN ESTIMATION OF CONDITIONAL DENSITIES
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *