Hostname: page-component-76fb5796d-22dnz Total loading time: 0 Render date: 2024-04-26T08:25:00.525Z Has data issue: false hasContentIssue false
  • English
  • Français

Weak Convergence Rates of Population Versus Single-Chain Stochastic Approximation MCMC Algorithms

Part of: Markov processes Probabilistic methods, simulation and stochastic differential equations

Published online by Cambridge University Press:  22 February 2016

Qifan Song ,
Mingqi Wu  and
Faming Liang
Qifan Song*
Affiliation:
Texas A&M University
Mingqi Wu*
Affiliation:
Shell Global Solutions (US) Inc.
Faming Liang*
Affiliation:
Texas A&M University
*
∗ Current address: Department of Statistics, Purdue University, West Lafayette, IN 47907, USA.
∗∗ Postal address: Shell Technology Center Houston, 3333 Highway 6 South, Houston, TX 77082, USA.
∗∗∗ Current address: Department of Biostatistics, University of Florida, Gainesville, FL 321611, USA. Email address: faliang@ufl.edu
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In this paper we establish the theory of weak convergence (toward a normal distribution) for both single-chain and population stochastic approximation Markov chain Monte Carlo (MCMC) algorithms (SAMCMC algorithms). Based on the theory, we give an explicit ratio of convergence rates for the population SAMCMC algorithm and the single-chain SAMCMC algorithm. Our results provide a theoretic guarantee that the population SAMCMC algorithms are asymptotically more efficient than the single-chain SAMCMC algorithms when the gain factor sequence decreases slower than O(1 / t), where t indexes the number of iterations. This is of interest for practical applications.

Keywords

Asymptotic normalityMarkov chain Monte Carlostochastic approximationMetropolis-Hastings algorithm

MSC classification

Primary: 60J22: Computational methods in Markov chains
Secondary: 65C05: Monte Carlo methods
Type
General Applied Probability
Information
Advances in Applied Probability , Volume 46 , Issue 4 , December 2014 , pp. 1059 - 1083
Copyright
© Applied Probability Trust 

References

Aldous, D., Lovász, L. and Winkler, P. (1997). Mixing times for uniformly ergodic Markov chains. Stoch. Process. Appl. 71, 165–185.Google Scholar
Andrieu, C. and Moulines, É. (2006). On the ergodicity properties of some adaptive MCMC algorithms. Ann. Appl. Prob. 16, 1462–1505.Google Scholar
Andrieu, C., Moulines, É. and Priouret, P. (2005). Stability of stochastic approximation under verifiable conditions. SIAM J. Control Optimization 44, 283–312.Google Scholar
Atchadé, Y. and Fort, G. (2010). Limit theorems for some adaptive MCMC algorithms with subgeometric kernels. Bernoulli 16, 116–154.Google Scholar
Atchadé, Y., Fort, G., Moulines, E. and Priouret, P. (2011) Adaptive Markov chain Monte Carlo: theory and methods. In Bayesian Time Series Models, Cambridge University Press, pp. 32–51.Google Scholar
Benveniste, A., Métivier, M. and Priouret, P. (1990). Adaptive Algorithms and Stochastic Approximations. Springer, Berlin.Google Scholar
Casella, G. and Berger, R. L. (2002). Statistical Inference, 2nd edn. Thomson Learning, Pacific Grove, CA.Google Scholar
Chen, H.-F. (2002). Stochastic Approximation and Its Applications. Kluwer, Dordrecht.Google ScholarPubMed
Cheon, S. and Liang, F. (2009). Bayesian phylogeny analysis via stochastic approximation Monte Carlo. Mol. Phylogenet. Evol. 53, 394–403.CrossRefGoogle ScholarPubMed
Geman, S. and Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741.Google Scholar
Geyer, C. J. (1991). Markov chain Monte Carlo maximum likelihood. In Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface, Interface Foundation, Fairfax Station, VA, pp. 153–163.Google Scholar
Gilks, W. R., Roberts, G. O., and George, E. I. (1994). Adaptive direction sampling. J. R. Statist. Soc. Ser. D (The Statistician) 43, 179–189.Google Scholar
Gu, M. G. and Kong, F. H. (1998). A stochastic approximation algorithm with Markov chain Monte-Carlo method for incomplete data estimation problems. Proc. Nat. Acad. Sci. USA 95, 7270–7274.Google Scholar
Haario, H., Saksman, E. and Tamminen, J. (2001). An adaptive Metropolis algorithm. Bernoulli 7, 223–242.Google Scholar
Hall, P. and Heyde, C. C. (1980). Martingale Limit Theory and Its Applications. Academic Press, New York.Google Scholar
Hastings, W. K. (1970). Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57, 97–109.Google Scholar
Liang, F. (2007). Continuous contour Monte Carlo for marginal density estimation with an application to a spatial statistical model. J. Comput. Graph. Statist. 16, 608–632.Google Scholar
Liang, F. (2009). Improving SAMC using smoothing methods: theory and applications to Bayesian model selection problems. Ann. Statist. 37, 2626–2654.Google Scholar
Liang, F. (2010). Trajectory averaging for stochastic approximation MCMC algorithms. Ann. Statist. 38, 2823–2856.Google Scholar
Liang, F., and Wong, W. H. (2000). Evolutionary Monte Carlo: applications to Cp model sampling and change point problem. Statistica Sinica 10, 317–342.Google Scholar
Liang, F., and Wong, W. H. (2001). Real-parameter evolutionary Monte Carlo with applications to Bayesian mixture models. J. Amer. Statist. Assoc. 96, 653–666.Google Scholar
Liang, F. and Zhang, J. (2009). Learning Bayesian networks for discrete data. Comput. Statist. Data. Anal. 53, 865–876.Google Scholar
Liang, F., Liu, C. and Carroll, R. J. (2007). Stochastic approximation in Monte Carlo computation. J. Amer. Statist. Assoc. 102, 305–320.Google Scholar
Liu, J. S., Liang, F. and Wong, W. H. (2000). The multiple-try method and local optimization in Metropolis sampling. J. Amer. Statist. Assoc. 95, 121–134.Google Scholar
Marinari, E. and Parisi, G. (1992). Simulated tempering: a new Monte Carlo scheme. Europhys. Lett. 19, 451–458.Google Scholar
Metropolis, N. et al. (1953). Equation of state calculations by fast computing machines. J. Chem. Phys. 21, 1087–1092.Google Scholar
Meyn, S. and Tweedie, R. L. (2009). Markov Chains and Stochastic Stability, 2nd edn. Cambridge University Press.CrossRefGoogle Scholar
Nummelin, E. (1984), General Irreducible Markov Chains and Nonnegative Operators. Cambridge University Press.Google Scholar
Pelletier, M. (1998). Weak convergence rates for stochastic approximation with application to multiple targets and simulated annealing. Ann. Appl. Prob. 8, 10–44.Google Scholar
Robbins, H. and Monro, S. (1951). A stochastic approximation method. Ann. Math. Statist. 22, 400–407.Google Scholar
Roberts, G. O. and Rosenthal, J. S. (2009). Examples of adaptive MCMC. J. Comput. Graph. Statist. 18, 349–367.Google Scholar
Roberts, G. O. and Tweedie, R. L. (1996). Geometric convergence and central limit theorems for multidimensional Hastings and Metropolis algorithms. Biometrika 83, 95–110.Google Scholar
Song, Q., Wu, M. and Liang, F. (2013). Supplementary material for ‘Weak convergence rates of population versus single-chain stochastic approximation MCMC algorithms’. Available at http://www.stat.tamu.edu/∼fliang.Google Scholar
Tadić, V. (1997). On the convergence of stochastic iterative algorithms and their applications to machine learning. Technical report, Mihajlo Pupin Institute, Serbia, Yugoslavia. A short version of this paper was published in Proc. 36th Conf. Decision & Control, San Diego, CA, pp. 2281–2286.Google Scholar
Wang, F. and Landau, D. P. (2001). Efficient, multiple-range random walk algorithm to calculate the density of states. Phys. Rev. Lett. 86, 2050–2053.Google Scholar
Wong, W. H. and Liang, F. (1997). Dynamic weighting in Monte Carlo and optimization. Proc. Nat. Acad. Sci. USA 94, 14220–14224.Google Scholar
Younes, L. (1989). Parametric inference for imperfectly observed Gibbsian fields. Prob. Theory Relat. Fields 82, 625–645.Google Scholar
Ziedan, I. E. (1972). Explicit solution of the Lyapunov-matrix equation. IEEE Trans. Automatic Control 17, 379–381.Google Scholar