Hostname: page-component-7479d7b7d-c9gpj Total loading time: 0 Render date: 2024-07-10T19:39:59.293Z Has data issue: false hasContentIssue false
  • English
  • Français

Faster MCMC for Gaussian latent position network models

Published online by Cambridge University Press:  22 February 2022

Neil A. Spencer Open the ORCID record for Neil A. Spencer [Opens in a new window] ,
Brian W. Junker  and
Tracy M. Sweet Open the ORCID record for Tracy M. Sweet [Opens in a new window]
Neil A. Spencer*
Affiliation:
Harvard University, Boston, MA 02115, USA
Brian W. Junker
Affiliation:
Carnegie Mellon University, Pittsburgh, PA 15213, USA
Tracy M. Sweet
Affiliation:
University of Maryland College Park, College Park, MD 20742, USA
*
*Corresponding author. Email: nspencer@hpsh.harvard.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Latent position network models are a versatile tool in network science; applications include clustering entities, controlling for causal confounders, and defining priors over unobserved graphs. Estimating each node’s latent position is typically framed as a Bayesian inference problem, with Metropolis within Gibbs being the most popular tool for approximating the posterior distribution. However, it is well-known that Metropolis within Gibbs is inefficient for large networks; the acceptance ratios are expensive to compute, and the resultant posterior draws are highly correlated. In this article, we propose an alternative Markov chain Monte Carlo strategy—defined using a combination of split Hamiltonian Monte Carlo and Firefly Monte Carlo—that leverages the posterior distribution’s functional form for more efficient posterior computation. We demonstrate that these strategies outperform Metropolis within Gibbs and other algorithms on synthetic networks, as well as on real information-sharing networks of teachers and staff in a school district.

Keywords

Hamiltonian Monte Carlonetwork datafirefly Monte Carlolatent space modellongitudinal network dataBayesian computation
Type
Research Article
Information
Network Science , Volume 10 , Issue 1 , March 2022 , pp. 20 - 45
Copyright
© The Author(s), 2022. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Action Editor: Stanley Wasserman

References

Airoldi, E. M., Blei, D. M., Fienberg, S. E., & Xing, E. P. (2008). Mixed membership stochastic blockmodels. Journal of Machine Learning Research, 9(Sep), 1981–2014.Google ScholarPubMed
Aliverti, E., & Durante, D. (2019). Spatial modeling of brain connectivity data via latent distance models with nodes clustering. Statistical Analysis and Data Mining: The ASA Data Science Journal, 12(3), 185–196.CrossRefGoogle Scholar
Bales, B., Pourzanjani, A., Vehtari, A., & Petzold, L. (2019). Selecting the metric in Hamiltonian Monte Carlo. arXiv preprint arXiv:1905.11916.Google Scholar
Betancourt, M. (2016). Identifying the optimal integration time in Hamiltonian Monte Carlo. arXiv preprint arXiv:1601.00225.Google Scholar
Betancourt, M. (2017). A conceptual introduction to Hamiltonian Monte Carlo. arXiv preprint arXiv:1701.02434.Google Scholar
Bloem-Reddy, B., & Cunningham, J. (2016). Slice sampling on Hamiltonian trajectories. In International Conference on Machine Learning (pp. 3050–3058).Google Scholar
Borgs, C., Chayes, J., Cohn, H., & Zhao, Y. (2014). An $l^p$ theory of sparse graph convergence i: Limits, sparse random graph models, and power law distributions. Transactions of the American Mathematical Society.Google Scholar
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., $\ldots$ Riddell, A. (2017). Stan: A probabilistic programming language. Journal of Statistical Software, 76(1).CrossRefGoogle Scholar
Carrington, P. J., Scott, J., & Wasserman, S. (2005). Models and Methods in Social Network Analysis, vol. 28. Cambridge university press.CrossRefGoogle Scholar
Chao, W.-L., Solomon, J., Michels, D., & Sha, F. (2015). Exponential integration for Hamiltonian Monte Carlo. In International Conference on Machine Learning (pp. 1142–1151).Google Scholar
Chen, L., Vogelstein, J. T., Lyzinski, V., & Priebe, C. E. (2016). A joint graph inference case study: the c. elegans chemical and electrical connectomes. In Worm, vol. 5, (p. e1142041). Taylor & Francis.CrossRefGoogle Scholar
Chiu, G. S., & Westveld, A. H. (2011). A unifying approach for food webs, phylogeny, social networks, and statistics. Proceedings of the National Academy of Sciences, 108(38), 15881–15886.CrossRefGoogle Scholar
Clauset, A., Moore, C., & Newman, M. E. (2008). Hierarchical structure and the prediction of missing links in networks. Nature, 453(7191), 98.CrossRefGoogle ScholarPubMed
Crane, H. (2018). Probabilistic Foundations of Statistical Network Analysis. Chapman and Hall/CRC.CrossRefGoogle Scholar
Dabbs, B., Adhikari, S., & Sweet, T. (2020). Conditionally independent dyads (CID) network models: a latent variable approach to statistical social network analysis. Social Networks, Revision Under Review.CrossRefGoogle Scholar
Doucet, A., & Johansen, A. M. (2009). A tutorial on particle filtering and smoothing: Fifteen years later. Handbook of Nonlinear Filtering, 12(656–704), 3.Google Scholar
Duane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987). Hybrid Monte Carlo. Physics letters B, 195(2), 216–222.CrossRefGoogle Scholar
ErdÖs, P., & RÉnyi, A. (1960). On the evolution of random graphs. Publications of the Mathematical Institute of the Hungarian Academy of Sciences, 5(1), 17–60.Google Scholar
Fosdick, B. K., McCormick, T. H., Murphy, T. B., Ng, T. L. J., & Westling, T. (2018). Multiresolution network models. Journal of Computational and Graphical Statistics (pp. 1–12).Google ScholarPubMed
Gamerman, D., & Lopes, H. F. (2006). Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference. CRC Press.CrossRefGoogle Scholar
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013). Bayesian Data Analysis. CRC Press.CrossRefGoogle Scholar
Girolami, M., & Calderhead, B. (2011). Riemann manifold Langevin and Hamiltonian Monte Carlo methods. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 73(2), 123–214.CrossRefGoogle Scholar
Goldenberg, A., Zheng, A. X., Fienberg, S. E., Airoldi, E. M., (2010). A survey of statistical network models. Foundations and Trends in Machine Learning, 2(2), 129–233.CrossRefGoogle Scholar
Hahn, P. R., He, J., & Lopes, H. F. (2019). Efficient sampling for gaussian linear regression with arbitrary priors. Journal of Computational and Graphical Statistics, 28(1), 142–154.CrossRefGoogle Scholar
Handcock, M. S., Raftery, A. E., & Tantrum, J. M. (2007). Model-based clustering for social networks. Journal of the Royal Statistical Society: Series A (Statistics in Society), 170(2), 301–354.CrossRefGoogle Scholar
Hecker, M., Lambeck, S., Toepfer, S., Van Someren, E., & Guthke, R. (2009). Gene regulatory network inference: Data integration in dynamic models—a review. Biosystems, 96(1), 86–103.CrossRefGoogle ScholarPubMed
Hoff, P. D., Raftery, A. E., & Handcock, M. S. (2002). Latent space approaches to social network analysis. Journal of the American Statistical Association, 97(460), 1090–1098.CrossRefGoogle Scholar
Hoffman, M. D., & Gelman, A. (2014). The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research, 15(1), 1593–1623.Google Scholar
Ji, P., & Jin, J. (2016). Coauthorship and citation networks for statisticians. The Annals of Applied Statistics, 10(4), 1779–1812.Google Scholar
Kim, B., Lee, K. H., Xue, L., & Niu, X. (2018). A review of dynamic network models with latent variables. Statistics Surveys, 12, 105.CrossRefGoogle ScholarPubMed
Krivitsky, P. N., & Handcock, M. S. (2008). Fitting position latent cluster models for social networks with latentnet. Journal of Statistical Software, 24.CrossRefGoogle Scholar
Krivitsky, P. N., Handcock, M. S., Raftery, A. E., & Hoff, P. D. (2009). Representing degree distributions, clustering, and homophily in social networks with latent cluster random effects models. Social Networks, 31(3), 204–213.CrossRefGoogle ScholarPubMed
Leimkuhler, B., & Reich, S. (2004). Simulating Hamiltonian Dynamics, vol. 14. Cambridge University Press.Google Scholar
Linderman, S., Adams, R. P., & Pillow, J. W. (2016). Bayesian latent structure discovery from multi-neuron recordings. In Advances in Neural Information Processing Systems (pp. 2002–2010).Google Scholar
Maclaurin, D., & Adams, R. P. (2015). Firefly Monte Carlo: Exact MCMC with subsets of data. In International Joint Conference on Artificial Intelligence.Google Scholar
Mannseth, J., Kleppe, T. S., & Skaug, H. J. (2016). On the application of higher order symplectic integrators in Hamiltonian Monte Carlo. arXiv preprint arXiv:1608.07048.Google Scholar
McFowland III, E., & Shalizi, C. R. (2021). Estimating causal peer influence in homophilous social networks by inferring latent locations. Journal of the American Statistical Association (pp. 1–12).CrossRefGoogle Scholar
Murray, I., Adams, R., & MacKay, D. (2010). Elliptical slice sampling. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics (pp. 541–548).Google Scholar
Neal, R. M. (2011). MCMC using Hamiltonian dynamics. In S. Brooks, A. Gelman, G. L. Jones, & X.-L. Meng (Eds.), Handbook of Markov chain Monte Carlo, chap. 5. New York: CRC Press, pp. 113–162.Google Scholar
Newman, M. E. (2002). Spread of epidemic disease on networks. Physical Review E, 66(1), 016128.CrossRefGoogle ScholarPubMed
Pakman, A., & Paninski, L. (2014). Exact Hamiltonian Monte Carlo for truncated multivariate gaussians. Journal of Computational and Graphical Statistics, 23(2), 518–542.CrossRefGoogle Scholar
Papaspiliopoulos, O., Roberts, G. O., & SkÖld, M. (2007). A general framework for the parametrization of hierarchical models. Statistical Science, (pp. 59–73).Google Scholar
Raftery, A. E., Niu, X., Hoff, P. D., & Yeung, K. Y. (2012). Fast inference for the latent space network model using a case-control approximate likelihood. Journal of Computational and Graphical Statistics, 21(4), 901–919.CrossRefGoogle ScholarPubMed
Rastelli, R., Friel, N., & Raftery, A. E. (2016). Properties of latent variable network models. Network Science, 4(4), 407–432.CrossRefGoogle Scholar
Rastelli, R., Maire, F., & Friel, N. (2018). Computationally efficient inference for latent position network models. arXiv preprint arXiv:1804.02274.Google Scholar
Roberts, G. O., Rosenthal, J. S., et al. (2001). Optimal scaling for various metropolis-hastings algorithms. Statistical Science, 16(4), 351–367.CrossRefGoogle Scholar
Salter-Townshend, M., & McCormick, T. H. (2017). Latent space models for multiview network data. The Annals of Applied Statistics, 11(3), 1217.CrossRefGoogle ScholarPubMed
Salter-Townshend, M., & Murphy, T. B. (2013). Variational Bayesian inference for the latent position cluster model for network data. Computational Statistics & Data Analysis, 57(1), 661–671.CrossRefGoogle Scholar
Shahbaba, B., Lan, S., Johnson, W. O., & Neal, R. M. (2014). Split Hamiltonian Monte Carlo. Statistics and Computing, 24(3), 339–349.CrossRefGoogle Scholar
Shortreed, S., Handcock, M. S., & Hoff, P. (2006). Positional estimation within a latent space model for networks. Methodology, 2(1), 24–33.CrossRefGoogle Scholar
Spencer, N. A., & Shalizi, C. R. (2019). Projective, sparse, and learnable latent position network models. arXiv preprint arXiv:1709.09702.Google Scholar
Spillane, J. P., & Hopkins, M. (2013). Organizing for instruction in education systems and school organizations: How the subject matters. Journal of Curriculum Studies, 45(6), 721–747.CrossRefGoogle Scholar
Spillane, J. P., Hopkins, M., & Sweet, T. M. (2018). School district educational infrastructure and change at scale: Teacher peer interactions and their beliefs about mathematics instruction. American Educational Research Journal, 55(3), 532–571.CrossRefGoogle Scholar
Sweet, T., & Adhikari, S. (2020). A latent space network model for social influence. Psychometrika, (pp. 1–24).Google ScholarPubMed
Sweet, T. M., Thomas, A. C., & Junker, B. W. (2013). Hierarchical network models for education research: Hierarchical latent space models. Journal of Educational and Behavioral Statistics, 38(3), 295–318.CrossRefGoogle Scholar
Turnbull, K. (2020). Advancements in Latent Space Network Modelling. Ph.D. thesis, Lancaster University.Google Scholar
Xie, F., & Levinson, D. (2009). Modeling the growth of transportation networks: A comprehensive review. Networks and Spatial Economics, 9(3), 291–307.CrossRefGoogle Scholar
Supplementary material: PDF

Spencer et al. supplementary material

Spencer et al. supplementary material

Download Spencer et al. supplementary material(PDF)
PDF 440.8 KB