Hostname: page-component-848d4c4894-xm8r8 Total loading time: 0 Render date: 2024-06-28T19:51:41.623Z Has data issue: false hasContentIssue false
  • English
  • Français

Monte Carlo fusion

Part of: Decision theory Probabilistic methods, simulation and stochastic differential equations

Published online by Cambridge University Press:  12 July 2019

Hongsheng Dai ,
Murray Pollock  and
Gareth Roberts
Hongsheng Dai*
Affiliation:
University of Essex
Murray Pollock*
Affiliation:
University of Warwick
Gareth Roberts*
Affiliation:
University of Warwick
*
*Postal address: Department of Mathematical Sciences, University of Essex, Wivenhoe Park, Colchester, CO4 3SQ, UK. Email address: hdaia@essex.ac.uk
**Postal address: Department of Statistics, University of Warwick, Gibbet Hill Road, Coventry, CV4 7ES, UK.
**Postal address: Department of Statistics, University of Warwick, Gibbet Hill Road, Coventry, CV4 7ES, UK.
Get access Rights & Permissions [Opens in a new window]

Abstract

In this paper we propose a new theory and methodology to tackle the problem of unifying Monte Carlo samples from distributed densities into a single Monte Carlo draw from the target density. This surprisingly challenging problem arises in many settings (for instance, expert elicitation, multiview learning, distributed ‘big data’ problems, etc.), but to date the framework and methodology proposed in this paper (Monte Carlo fusion) is the first general approach which avoids any form of approximation error in obtaining the unified inference. In this paper we focus on the key theoretical underpinnings of this new methodology, and simple (direct) Monte Carlo interpretations of the theory. There is considerable scope to tailor the theory introduced in this paper to particular application settings (such as the big data setting), construct efficient parallelised schemes, understand the approximation and computational efficiencies of other such unification paradigms, and explore new theoretical and methodological directions.

Keywords

Fork-and-joinfusionLangevin diffusionMonte Carlo

MSC classification

Primary: 65C05: Monte Carlo methods 65C60: Computational problems in statistics
Secondary: 62C10: Bayesian problems; characterization of Bayes procedures 65C30: Stochastic differential and integral equations
Type
Research Papers
Information
Journal of Applied Probability , Volume 56 , Issue 1 , March 2019 , pp. 174 - 191
Copyright
© Applied Probability Trust 2019 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agarwal, A. and Duchi, J. C. (2012). Distributed delayed stochastic optimization. In 51st IEEE Conference on Decision and Control, pp. 5451–5452.Google Scholar
Berger, O. J. (1980). Statistical Decision Theory and Bayesian Analysis. Springer, New York.10.1007/978-1-4757-1727-3CrossRefGoogle Scholar
Beskos, A. and Roberts, G. O. (2005). Exact simulation of diffusions. Ann. Appl. Prob. 15, 2422–2444.10.1214/105051605000000485CrossRefGoogle Scholar
Beskos, A., Papaspiliopoulos, O. and Roberts, G.O. (2006b), Retrospective exact simulation of diffusion sample paths with applications, Bernoulli 12, 1077–1098.10.3150/bj/1165269151CrossRefGoogle Scholar
Beskos, A., Papaspiliopoulos, O. and Roberts, G. O. (2008). A factorisation of diffusion measure and finite sample path constructions. Methodology Comput. Appl. Prob. 10, 85–104.10.1007/s11009-007-9060-4CrossRefGoogle Scholar
Beskos, A., Papaspiliopoulos, O., Roberts, G. O. and Fearnhead, P. (2006a). Exact and computationally efficient likelihood-based estimation for discretely observed diffusion processes (with discussion). J. R. Statist. Soc. B 68, 333–382.10.1111/j.1467-9868.2006.00552.xCrossRefGoogle Scholar
Chen, N. and Huang, Z. (2013). Localisation and exact simulation of Brownian motion-driven stochastic differential equations. Math. Operat. Res. 38, 591–616.10.1287/moor.2013.0585CrossRefGoogle Scholar
Dacunha-Castelle, D. and Florens-Zmirou, D. (1986). Estimation of the coefficients of a diffusion from discrete observations. Stochastics 19, 263–284.10.1080/17442508608833428CrossRefGoogle Scholar
Dai, H. (2014). Exact simulation for diffusion bridges: an adaptive approach. J. Appl. Prob. 51, 346–358.10.1239/jap/1402578629CrossRefGoogle Scholar
Dai, H. (2017). A new rejection sampling method without using hat function. Bernoulli 23, 2434–2465.10.3150/16-BEJ814CrossRefGoogle Scholar
Dai, H., Pollock, M. and Roberts, G. (2019). Monte Carlo fusion. Supplementary material. Available at http://doi.org/10.1017/jpr.2019.12 http://doi.org/10.1017/jpr.2019.12.CrossRefGoogle Scholar
Fleiss, J. L. (1993). Statistical basis of meta-analysis. Statist. Methods Med. Res. 2, 121–145.10.1177/096228029300200202CrossRefGoogle ScholarPubMed
Genest, C. and Zidek, J. V. (1986). Combining probability distributions: a critique and an annotated bibliography. Statist. Sci. 1, 114–148.10.1214/ss/1177013825CrossRefGoogle Scholar
Hansen, N. R. (2003). Geometric ergodicity of discrete-time approximations to multivariate diffusions. Bernoulli 9, 725–743.10.3150/bj/1066223276CrossRefGoogle Scholar
Li, C., Srivastava, S. and Dunson, D. B. (2017). Simple, scalable and accurate posterior interval estimation. Biometrika 104, 665–680.CrossRefGoogle Scholar
Li, Y., Yang, M. and Zhang, Z. (2015). Multi-view representation learning: A survey from shallow methods to deep methods. J. Latex Class Files 14, 20pp.Google Scholar
Masuda, H. (2004). On multidimensional Ornstein-Uhlenbeck processes driven by a general Lévy process. Bernoulli 10, 97–120.10.3150/bj/1077544605CrossRefGoogle Scholar
Minsker, S., Srivastava, S., Lin, L. and Dunson, D. B. (2014). Scalable and robust Bayesian inference via the median posterior. In Proc. 31st Internat. Conference on Machine Learning, pp. 1656–1664.Google Scholar
Neiswanger, W., Wang, C. and Xing, E. P. (2014). Asymptotically exact, embarrassingly parallel MCMC. In Proc. 13th Conference on Uncertainty In Artificial Intelligence, pp. 623–632.Google Scholar
Pollock, M. (2013). Some Monte Carlo methods for jump diffusions. Doctoral Thesis, University of Warwick.Google Scholar
Pollock, M., Johansen, A. M. and Roberts, G. O. (2016b). On the exact and ε-strong simulation of (jump) diffusions. Bernoulli 22, 794–856.10.3150/14-BEJ676CrossRefGoogle Scholar
Pollock, M, Fearnhead, P., Johansen, A. M. and Roberts, G. O. (2016a). The scalable Langevin exact algorithm: bayesian inference for big data. Submitted to J. R. Statist. Soc. B.Google Scholar
Scott, S. L. et al. (2016). Bayes and big data: the consensus Monte Carlo algorithm. Internat. J. Manag. Sci. Eng. Manag. 11, 78–88.Google Scholar
Smith, T. C., Spiegelhalter, D. J. and Thomas, A. (1995). Bayesian approaches to random-effects meta-analysis: a comparative study Statist. Med. 14, 2685–2699.10.1002/sim.4780142408CrossRefGoogle ScholarPubMed
Srivastava, S., Cevher, V., Dinh, Q. and Dunson, D. (2016). WASP: scalable Bayes via barycenters of subset posteriors. In Proc. 18th Internat. Conference on Artificial Intelligence and Statistics, pp. 912–920.Google Scholar
Stamatakis, A. and Aberer, A. J. (2013). Novel parallelization schemes for large-scale likelihood-based phylogenetic inference. In Proc. 2013 IEEE 27th Internat. Symposium on Parallel and Distributed Processing, pp. 1195-1204.10.1109/IPDPS.2013.70CrossRefGoogle Scholar
Tan, A., Doss, H. and Hobert, J. P. (2015). Honest importance sampling with multiple Markov chains. J. Comput. Graph. Statist. 24, 792–826.10.1080/10618600.2014.929523CrossRefGoogle ScholarPubMed
Wang, X. and Dunson, D. B. (2014). Parallelizing MCMC via Weierstrass sampler. Preprint. Available at https://arxiv.org/abs/1312.4605v2 https://arxiv.org/abs/1312.4605v2.Google Scholar
Zhao, J., Xie, X., Xu, X. and Sun, S. (2017). Multi-view learning overview: recent progress and new challenges. Inf. Fusion 38, 43–54.10.1016/j.inffus.2017.02.007CrossRefGoogle Scholar
Supplementary material: PDF

Dai supplementary material

Supplementary material

Download Dai supplementary material(PDF)
PDF 238.6 KB