Skip to main content Accessibility help
Hostname: page-component-f7d5f74f5-qtg9w Total loading time: 0 Render date: 2023-10-04T22:56:21.163Z Has data issue: false Feature Flags: { "corePageComponentGetUserInfoFromSharedSession": true, "coreDisableEcommerce": false, "coreDisableSocialShare": false, "coreDisableEcommerceForArticlePurchase": false, "coreDisableEcommerceForBookPurchase": false, "coreDisableEcommerceForElementPurchase": false, "coreUseNewShare": true, "useRatesEcommerce": true } hasContentIssue false

29 - Meta-Analysis

Integration of Empirical Findings through Quantitative Modeling

from Part VII - General Analytic Considerations

Published online by Cambridge University Press:  23 March 2020

Aidan G. C. Wright
University of Pittsburgh
Michael N. Hallquist
Pennsylvania State University
Get access


Meta-analysis is a well-established approach to integrating research findings, with a long history in the sciences and in psychology in particular. Its use in summarizing research findings has special significance given increasing concerns about scientific replicability, but it has other important uses as well, such as integrating information across studies to examine models that might otherwise be too difficult to study in a single sample. This chapter discusses different forms and purposes of meta-analyses, typical elements of meta-analyses, and basic statistical and analytic issues that arise, such as choice of meta-analytic model and different sources of variability and bias in estimates. The chapter closes with discussion of emerging issues in meta-analysis and directions for future research.

Publisher: Cambridge University Press
Print publication year: 2020

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)


Berkey, C. S., Hoaglin, D. C., Mosteller, F., & Colditz, G. A. (1995). A Random-Effects Regression Model for Meta-Analysis. Statistics in Medicine, 14(4), 395411.CrossRefGoogle ScholarPubMed
Bickel, P. J., & Blackwell, D. (1967). A Note on Bayes Estimates. Annals of Mathematical Statistics, 38, 19071911.CrossRefGoogle Scholar
Blackwell, D., & Girshick, M. A. (1954). Theory of Games and Statistical Decisions. New York: John Wiley.Google Scholar
Bodnar, O., Link, A., Arendacká, B., Possolo, A., & Elster, C. (2017). Bayesian Estimation in Random Effects Meta-Analysis Using a Non-informative Prior. Statistics in Medicine, 36(2), 378399.CrossRefGoogle ScholarPubMed
Carlsson, R., Schimmack, U., Williams, D. R., & Bürkner, P.-C. (2017). Bayes Factors from Pooled Data Are no Substitute for Bayesian Meta-Analysis: Commentary on Scheibehenne, Jamil, and Wagenmakers (2016). Psychological Science, 28(11), 16941697.CrossRefGoogle Scholar
Carter, E. C., Kofler, L. M., Forster, D. E., & McCullough, M. E. (2015). A Series of Meta-Analytic Tests of the Depletion Effect: Self-Control Does Not Seem to Rely on a Limited Resource. Journal of Experimental Psychology: General, 144, 796815.CrossRefGoogle Scholar
Center for Open Science. (2018). Transparency and Openness Promotion Guidelines. Retrieved from Scholar
Cheung, M. (2015). Meta-Analysis: A Structural Equation Modeling Approach. Chichester: Wiley.Google Scholar
Clarke, B. S. (1999). Asymptotic Normality of the Posterior in Relative Entropy. IEEE Transactions on Information Theory, 45(1), 165176.CrossRefGoogle Scholar
Cochran, W. G. (1954). The Combination of Estimates from Different Experiments. Biometrics, 10(1), 101129.CrossRefGoogle Scholar
Copas, J. B. (1999). What Works? Selectivity Models and Meta-Analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 162, 95109.CrossRefGoogle Scholar
Copas, J. B. (2013). A Likelihood-Based Sensitivity Analysis for Publication Bias in Meta-Analysis. Journal of the Royal Statistical Society: Series C (Applied Statistics), 62, 4766.Google Scholar
Duval, S., & Tweedie, R. (2000). Trim and Fill: A Simple Funnel-Plot-Based Method of Testing and Adjusting for Publication Bias in Meta-Analysis. Biometrics, 56, 455463.CrossRefGoogle ScholarPubMed
Fleiss, J. L., Cohen, J., & Everitt, B. S. (1969). Large Sample Standard Errors of Kappa and Weighted Kappa. Psychological Bulletin, 72(5), 323327.CrossRefGoogle Scholar
Fritz, C. O., Morris, P. E., & Richler, J. J. (2012). Effect Size Estimates: Current Use, Calculations, and Interpretation. Journal of Experimental Psychology: General, 141(1), 218.CrossRefGoogle ScholarPubMed
Furlow, C. F., & Beretvas, S. N. (2005). Meta-Analytic Methods of Pooling Correlation Matrices for Structural Equation Modeling under Different Patterns of Missing Data. Psychological Methods, 10(2), 227254.CrossRefGoogle ScholarPubMed
Glass, G. V. (2015). Meta-Analysis at Middle Age: A Personal History. Research Synthesis Methods, 6(3), 221231.CrossRefGoogle Scholar
Greenland, S., & O’Rourke, K. (2001). On the Bias Produced by Quality Scores in Meta-Analysis, and a Hierarchical View of Proposed Solutions. Biostatistics, 2(4), 463471.CrossRefGoogle Scholar
Gustafson, P., & Clarke, B. (2004). Decomposing Posterior Variance. Journal of Statistical Planning and Inference, 119(2), 311327.CrossRefGoogle Scholar
Hagger, M. S., Chatzisarantis, N. L. D., Alberts, H., Anggono, C. O., Batailler, C., Birt, A. R., … Zwienenberg, M. (2016). A Multilab Preregistered Replication of the ego-Depletion Effect. Perspectives on Psychological Science, 11(4), 546573.CrossRefGoogle ScholarPubMed
Hagger, M. S., Wood, C., Stiff, C., & Chatzisarantis, N. L. D. (2010). Ego Depletion and the Strength Model of Self-Control: A Meta-Analysis. Psychological Bulletin, 136, 495525.CrossRefGoogle ScholarPubMed
Hardy, R. J., & Thompson, S. G. (1996). A Likelihood Approach to Meta-Analysis with Random Effects. Statistics in Medicine, 15(6), 619629.3.0.CO;2-A>CrossRefGoogle ScholarPubMed
Hedges, L. V. (1984). Estimation of Effect Size under Nonrandom Sampling: The Effects of Censoring Studies Yielding Statistically Insignificant Mean Differences. Journal of Educational and Behavioral Statistics, 9, 6185.CrossRefGoogle Scholar
Higgins, J. P. T., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., … Cochrane Statistical Methods Group (2011). The Cochrane Collaboration’s Tool for Assessing Risk of Bias in Randomized Trials. British Medical Journal, 343, d5928.CrossRefGoogle Scholar
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009). A Re-evaluation of Random-Effects Meta-Analysis. Journal of the Royal Statistical Society: Series A (Statistics in Society), 172(1), 137159.CrossRefGoogle ScholarPubMed
Huang-Pollock, C. L., Karalunas, S. L., Tam, H., & Moore, A. N. (2012). Evaluating Vigilance Deficits in ADHD: A Meta-Analysis of CPT Performance. Journal of Abnormal Psychology, 121(2), 360371.CrossRefGoogle ScholarPubMed
Ioannidis, J. P. A. (2005). Differentiating Biases from Genuine Heterogeneity: Distinguishing Artifactual from Substantive Effects. In Rothstein, H., Sutton, A. J., & Borestein, M. (Eds.), Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments (pp. 287302). Chichester: Wiley.Google Scholar
Ioannidis, J. P. A. (2008). Interpretation of Tests of Heterogeneity and Bias in Meta-Analysis. Journal of Evaluation in Clinical Practice, 14(5), 951957.CrossRefGoogle ScholarPubMed
Iyengar, S., & Greenhouse, J. B. (1988). Selection Models and the File Drawer Problem. Statistical Science, 3(1), 109117.CrossRefGoogle Scholar
Jin, Z.-C., Zhou, X.-H., & He, J. (2015). Statistical Methods for Dealing with Publication Bias in Meta-Analysis. Statistics in Medicine, 34(2), 343360.CrossRefGoogle ScholarPubMed
Jüni, P., Witschi, A., Bloch, R., & Egger, M. (1999). The Hazards of Scoring the Quality of Clinical Trials for Meta-Analysis. Journal of the American Medical Association, 282(11), 10541060.CrossRefGoogle ScholarPubMed
Kass, R. E., Eden, U. T., & Brown, E. N. (2014). Analysis of Neural Data. New York: Springer.CrossRefGoogle Scholar
Kraemer, H. C. (2005). A Simple Effect Size Indicator for Two-Group Comparisons? A Comment on Requivalent. Psychological Methods, 10(4), 413419.CrossRefGoogle Scholar
Kraemer, H. C., & Andrews, G. (1982). A Nonparametric Technique for Meta-Analysis Effect Size Calculation. Psychological Bulletin, 91(2), 404412.CrossRefGoogle Scholar
Krueger, R. F., & Markon, K. E. (2006). Reinterpreting Comorbidity: A Model-Based Approach to Understanding and Classifying Psychopathology. Annual Review of Clinical Psychology, 2(1), 111133.CrossRefGoogle ScholarPubMed
Lee, C. H., Cook, S., Lee, J. S., & Han, B. (2016). Comparison of Two Meta-Analysis Methods: Inverse-Variance-Weighted Average and Weighted Sum of z-Scores. Genomics & Informatics, 14(4), 173180.CrossRefGoogle ScholarPubMed
Li, S., Jiang, H., Yang, H., Chen, W., Peng, J., Sun, M., … Zeng, J. (2015). The Dilemma of Heterogeneity Tests in Meta-Analysis: A Challenge from a Simulation Study. PLOS ONE, 10(5), e0127538.CrossRefGoogle ScholarPubMed
Lin, D. Y., & Zeng, D. (2010). On the Relative Efficiency of Using Summary Statistics versus Individual-Level Data in Meta-Analysis. Biometrika, 97(2), 321332.CrossRefGoogle ScholarPubMed
Lumley, T. (2002). Network Meta-Analysis for Indirect Treatment Comparisons. Statistics in Medicine, 21(16), 23132324.CrossRefGoogle ScholarPubMed
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for Publication Bias in Meta-Analysis: An Evaluation of Selection Methods and Some Cautionary Notes. Perspectives on Psychological Science, 11(5), 730749.CrossRefGoogle ScholarPubMed
Munafo, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., … Ioannidis, J. P. A. (2017). A Manifesto for Reproducible Science. Nature Human Behaviour, 1(1), 0021.CrossRefGoogle Scholar
Noorbaloochi, S., & Meeden, G. (1983). Unbiasedness as the Dual of Being Bayes. Journal of the American Statistical Association, 78, 619623.CrossRefGoogle Scholar
Park, S., Serpedin, E., & Qaraqe, K. (2013). Gaussian Assumption: The Least Favorable but the Most Useful. IEEE Signal Processing Magazine, 30(3), 183186.CrossRefGoogle Scholar
Revelle, W., & Zinbarg, R. E. (2009). Coefficients Alpha, Beta, Omega, and the GLB: Comments on Sijtsma. Psychometrika, 74(1), 145154.CrossRefGoogle Scholar
Rosenberg, M. S. (2010). A Generalized Formula for Converting Chi-Square Tests to Effect Sizes for Meta-Analysis. PLoS ONE, 5(4), e10059.CrossRefGoogle ScholarPubMed
Rosenthal, R., & Rubin, D. B. (2003). Requivalent: A Simple Effect Size Indicator. Psychological Methods, 8(4), 492496.CrossRefGoogle Scholar
Ruscio, J. (2008). A Probability-Based Measure of Effect Size: Robustness to Base Rates and Other Factors. Psychological Methods, 13(1), 1930.CrossRefGoogle ScholarPubMed
Scheibehenne, B., Jamil, T., & Wagenmakers, E.-J. (2016). Bayesian Evidence Synthesis Can Reconcile Seemingly Inconsistent Results: The Case of Hotel Towel Reuse. Psychological Science, 27(7), 10431046.CrossRefGoogle ScholarPubMed
Schmidt, F. L., & Hunter, J. E. (2015). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings. Thousand Oaks, CA: Sage.CrossRefGoogle Scholar
Shadish, W. R. (2015). Introduction to the Special Issue on the Origins of Modern Meta-Analysis: Introduction to Special Issue. Research Synthesis Methods, 6(3), 219220.CrossRefGoogle ScholarPubMed
Sharma, L., Markon, K. E., & Clark, L. A. (2014). Toward a Theory of Distinct Types of “Impulsive” Behaviors: A Meta-Analysis of Self-Report and Behavioral Measures. Psychological Bulletin, 140(2), 374408.CrossRefGoogle Scholar
Smith, T. C., Spiegelhalter, D. J., & Thomas, A. (1995). Bayesian Approaches to Random-Effects Meta-Analysis: A Comparative Study. Statistics in Medicine, 14(24), 26852699.CrossRefGoogle ScholarPubMed
Stein, M. S., & Nossek, J. A. (2017). A Pessimistic Approximation for the Fisher Information Measure. IEEE Transactions on Signal Processing, 65(2), 386396.CrossRefGoogle Scholar
Stram, D. O. (1996). Meta-Analysis of Published Data Using a Linear Mixed-Effects Model. Biometrics, 52, 536544.CrossRefGoogle ScholarPubMed
Switzer, F. S., Paese, P. W., & Drasgow, F. (1992). Bootstrap Estimates of Standard Errors in Validity Generalization. Journal of Applied Psychology, 77(2), 123129.CrossRefGoogle Scholar
Van Aert, R. C. M., Wicherts, J. M., & van Assen, M. A. L. M. (2016). Conducting Meta-Analyses Based on p Values: Reservations and Recommendations for Applying p-Uniform and p-Curve. Perspectives on Psychological Science, 11(5), 713729.CrossRefGoogle Scholar
Van Houwelingen, H. C., Zwinderman, K. H., & Stijnen, T. (1993). A Bivariate Approach to Meta-Analysis. Statistics in Medicine, 12(24), 22732284.CrossRefGoogle ScholarPubMed
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., … Salanti, G. (2016). Methods to Estimate the Between-Study Variance and Its Uncertainty in Meta-Analysis. Research Synthesis Methods, 7(1), 5579.CrossRefGoogle ScholarPubMed
Viswesvaran, C., & Ones, D. S. (1995). Theory Testing: Combining Psychometric Meta-Analysis and Structural Equations Modeling. Personnel Psychology, 48(4), 865885.CrossRefGoogle Scholar
Xu, A., & Raginsky, M. (2017). Information-Theoretic Lower Bounds on Bayes Risk in Decentralized Estimation. IEEE Transactions on Information Theory, 63(3), 15801600.CrossRefGoogle Scholar
Zwaan, R., Etz, A., Lucas, R., & Donnellan, M. (2018). Making Replication Mainstream. Behavioral and Brain Sciences, 41, e120.CrossRefGoogle ScholarPubMed

Save book to Kindle

To save this book to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the or variations. ‘’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats