Hostname: page-component-848d4c4894-nmvwc Total loading time: 0 Render date: 2024-07-06T12:51:21.015Z Has data issue: false hasContentIssue false

Assessing the methodological quality of studies included in systematic reviews: Interpretation of scores

Published online by Cambridge University Press:  14 July 2022

Andrea C. Büchler
Affiliation:
Department of Medical Microbiology and Infectious Diseases, Erasmus MC University Medical Center, Rotterdam, The Netherlands
Anne F. Voor in ‘t holt*
Affiliation:
Department of Medical Microbiology and Infectious Diseases, Erasmus MC University Medical Center, Rotterdam, The Netherlands
*
Author for correspondence: Anne F. Voor in ‘t holt, E-mail: a.voorintholt@erasmusmc.nl
Rights & Permissions [Opens in a new window]

Abstract

Type
Letter to the Editor
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of The Society for Healthcare Epidemiology of America

To the Editor—Assessing the methodological quality of and, thus, risk of bias within studies included in systematic reviews is important to place the conclusions of systematic reviews in context. The choice of appropriate tools to assess the risk of bias depends on the design of the individual study. Reference Ma, Wang, Yang, Huang, Weng and Zeng1 The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement Reference Page, Moher and Bossuyt2 recommends the following: (1) to present quality assessments as scores for each component domain, (2) to restrict the primary analysis to studies judged to be at low risk of bias, (3) to stratify studies according to risk of bias using subgroup analysis or meta-regression, or (4) to adjust the result from each study in an attempt to remove the bias. Despite all available information and guidance, we feel that a step in this process is missing: the interpretation of the scores, that is, the classification of a study as being of low, medium, or high methodological quality. When only reporting scores without interpretation or threshold, it is impossible to select or stratify studies during analyses.

Possible approaches to classify the quality of included studies could be (1) to divide the assessed scores of included studies for each tool in thirds, (2) to divide the highest possible score for each tool in thirds, (3) to come up with your own scoring system, (4) to not interpret scores, or (5) to establish uniform thresholds that would be used by all authors. Choosing the first or third option will result in noncomparability of systematic review on the same or similar topics, especially if the quality assessment of the included studies is not publicly accessible. The second option does not take importance of certain domains into account. The fourth option does not allow for selection or stratification of studies based on quality scores. In systematic reviews recently published in the journal Infection Control and Hospital Epidemiology, assessment of risk of bias was either not reported, Reference Lim, Htun, Ong, Guo and Chow3,Reference Shaaban, Yassine, Bedwani and Abu-Sheasha4 an interpretation was not given and scores were reported for each individual study, Reference Puig-Asensio, Marra, Childs, Kukla, Perencevich and Schweizer5 and/or the threshold was chosen by the authors. Reference Babar, El Kurdi and El Iskandarani6,Reference Krah, Jones, Lake and Hersh7

We feel that scores alone do not give enough guidance to properly estimate the quality of a study. Rather than reporting risk of bias as a separate and independent paragraph of the systematic review, classification of studies as low, medium, or high methodological quality is needed to incorporate the risk of bias in the analyses. We encourage the authors of future systematic reviews with or without meta-analysis to integrate the quality assessment throughout the results section, to perform subgroup analyses excluding studies of low methodological quality, and to see the quality assessment as an important part of the research and not just a mandatory paragraph. Additionally, we ask the authors of the PRISMA statement, Cochrane, and developers of quality assessment tools to add threshold scores for low, medium, or high quality to the agenda because, in our opinion, they are urgently needed.

Acknowledgments

Financial support

No financial support was provided relevant to this article.

Conflict of interest

The authors report no conflicts of interest relevant to this article.

References

Ma, LL, Wang, YY, Yang, ZH, Huang, D, Weng, H, Zeng, XT. Methodological quality (risk of bias) assessment tools for primary and secondary medical studies: what are they and which is better? Mil Med Res 2020;7:7.Google ScholarPubMed
Page, MJ, Moher, D, Bossuyt, PM, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021;372:n160.CrossRefGoogle ScholarPubMed
Lim, DW, Htun, HL, Ong, LS, Guo, H, Chow, A. Systematic review of determinants influencing antibiotic prescribing for uncomplicated acute respiratory tract infections in adult patients at the emergency department. Infect Control Hosp Epidemiol 2022;43:366375.CrossRefGoogle ScholarPubMed
Shaaban, RH, Yassine, OG, Bedwani, RN, Abu-Sheasha, GA. Evaluation of the costing methodology of published studies estimating costs of surgical site infections: a systematic review. Infect Control Hosp Epidemiol 2021. doi: 10.1017/ice.2021.381.CrossRefGoogle Scholar
Puig-Asensio, M, Marra, AR, Childs, CA, Kukla, ME, Perencevich, EN, Schweizer, ML. Effectiveness of chlorhexidine dressings to prevent catheter-related bloodstream infections. Does one size fit all? A systematic literature review and meta-analysis. Infect Control Hosp Epidemiol 2020;41:13881395.CrossRefGoogle ScholarPubMed
Babar, S, El Kurdi, B, El Iskandarani, M, et al. Oral vancomycin prophylaxis for the prevention of Clostridium difficile infection: a systematic review and meta-analysis. Infect Control Hosp Epidemiol 2020;41:13021309.CrossRefGoogle ScholarPubMed
Krah, NM, Jones, TW, Lake, J, Hersh, AL. The impact of antibiotic allergy labels on antibiotic exposure, clinical outcomes, and healthcare costs: a systematic review. Infect Control Hosp Epidemiol 2021;42:530548.CrossRefGoogle ScholarPubMed