Hostname: page-component-8448b6f56d-jr42d Total loading time: 0 Render date: 2024-04-19T23:00:18.775Z Has data issue: false hasContentIssue false

SOME CONTRIBUTIONS TO MEASURING AND UNDERSTANDING HETEROGENEITY IN META-ANALYSIS

Published online by Cambridge University Press:  15 June 2022

MAXWELL CAIRNS*
Affiliation:
Department of Mathematics and Statistics, La Trobe University, Melbourne, Victoria 3086, Australia
Rights & Permissions [Opens in a new window]

Abstract

Type
Abstracts of Australasian PhD Theses
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of Australian Mathematical Publishing Association Inc.

Between-studies variance or heterogeneity is present in almost every meta-analysis. Therefore, understanding and accounting for the amount of heterogeneity present in an analysis will provide the researcher with important information about the validity of their claims or potentially generate new hypotheses for future research. Current practice is to report a p-value or a measure known as $I^2$ before making such a claim about the magnitude of the heterogeneity present in the analysis. This approach has often been shown to be flawed, both through use of p-values and rampant misinterpretation of the results.

In this thesis, we look at several measures for explaining heterogeneity. We reconsider a measure called the Diamond Ratio [Reference Cumming and Calin-Jageman4,Reference Higgins and Thompson5], which is the ratio of the width of the random-effects confidence interval over the width of the fixed-effect confidence interval of a meta-analysis. We propose new confidence intervals for this simple visual approach of describing the amount of heterogeneity and provide further discussion on how it can be used to describe heterogeneity [Reference Cairns, Cumming, Calin-Jageman and Prendergast1].

As another measure of heterogeneity, we consider the coefficient of variation (CV) [Reference Takkouche, Cadarso-Suarez and Spiegelman6] to describe heterogeneity relative to the effect. We discuss transformations of the CV so that it lies on the interval [0, 1] which may be preferred for interpretation. We then propose new confidences for the CV and its transformations which exhibit strong coverage properties, providing improvements over existing intervals for the CV. We show that the use of such measures, together with confidence intervals alongside $I^2$ , provides a more comprehensive look at the amount of heterogeneity in the meta-analysis [Reference Cairns and Prendergast3]. We then show that these concepts can extend to the context of meta-regression retaining their strong coverage properties [Reference Cairns and Prendergast2]. Finally, we provide a brief informal discussion of the various methods used for describing heterogeneity.

Footnotes

Thesis submitted to La Trobe University in April 2021; degree approved on 7 December 2021; primary supervisor Luke Prendergast, co-supervisor Robert Staudte.

References

Cairns, M., Cumming, G., Calin-Jageman, R. and Prendergast, L. A., ‘The diamond ratio: a visual indicator of the extent of heterogeneity in meta-analysis’, Br. J. Math. Stat. Psychol. 75(2) (2022), 201219.CrossRefGoogle ScholarPubMed
Cairns, M. and Prendergast, L. A., ‘Extending the coefficient of variation for measuring heterogeneity following a meta-regression’, Preprint, 2021, arXiv:2111.09518.Google Scholar
Cairns, M. and Prendergast, L. A., ‘On ratio measures of heterogeneity for meta-analyses’, Res. Synth. Methods 13(1) (2022), 2847.CrossRefGoogle ScholarPubMed
Cumming, G. and Calin-Jageman, R., Introduction to the New Statistics: Estimation, Open Science, and Beyond (Routledge, New York, 2017).Google Scholar
Higgins, J. P. T. and Thompson, S. G., ‘Quantifying heterogeneity in a meta-analysis’, Stat. Med. 21 (2002), 15391558.CrossRefGoogle ScholarPubMed
Takkouche, B., Cadarso-Suarez, C. and Spiegelman, D., ‘Evaluation of old and new tests of heterogeneity in epidemiologic meta-analysis’, Amer. J. Epidemiol. 150(2) (1999), 206215.CrossRefGoogle ScholarPubMed