Hostname: page-component-8448b6f56d-m8qmq Total loading time: 0 Render date: 2024-04-24T09:39:22.325Z Has data issue: false hasContentIssue false
  • English
  • Français

Peer review of searches for studies for health technology assessments, systematic reviews, and other evidence syntheses

Part of: Special Collection - Methods

Published online by Cambridge University Press:  24 May 2021

Carol Lefebvre Open the ORCID record for Carol Lefebvre [Opens in a new window]  and
Steven Duffy Open the ORCID record for Steven Duffy [Opens in a new window]
Carol Lefebvre*
Affiliation:
Lefebvre Associates Ltd, Oxford, UK
Steven Duffy
Affiliation:
Kleijnen Systematic Reviews Ltd, Unit 6, Escrick Business Park, Riccall Road, Escrick, YorkYO19 6FD, UK
*
Author for correspondence: Carol Lefebvre, E-mail: carol@lefebvreassociates.org
Get access Rights & Permissions [Opens in a new window]

Abstract

Introduction

Peer review of searches is a process whereby both the search strategies and the search process description are reviewed, ideally using an evidence-based checklist.

Rationale

As the search strategy underpins any well-conducted evidence synthesis, its quality could affect the final result. Evidence shows, however, that search strategies are prone to error.

Findings

There is increasing awareness and use of the PRESS Evidence-Based Checklist and peer review of search strategies, at the outset of evidence syntheses, prior to the searches being run, and this is now recommended by a number of evidence synthesis organizations.

Recommendations and conclusions

Searches for evidence syntheses should be peer reviewed by a suitably qualified and experienced librarian or information specialist after being designed, ideally, by another suitably qualified and experienced librarian or information specialist. Peer review of searches should take place at two important stages in the evidence synthesis process; at the outset of the project prior to the searches being run and at the prepublication stage. There is little empirical evidence, however, to support the effectiveness of peer review of searches. Further research is required to assess this. Those wishing to stay up to date with the latest developments in information retrieval, including peer review of searches, should consult the SuRe Info resource (http://www.sure-info.org), which seeks to help information specialists and others by providing easy access to the findings from current information retrieval methods research and thus support more research-based information retrieval practice.

Keywords

Databases as topicInformation storage and retrievalPeer reviewSystematic reviews as topicTechnology assessment, biomedical
Type
Method
Information
International Journal of Technology Assessment in Health Care , Volume 37 , Issue 1 , 2021 , e64
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Lefebvre, C, Glanville, J, Briscoe, S, Littlewood, A, Marshall, C, Metzendorf, M-I, et al. Chapter 4: Searching for and selecting studies. 4.4.8 Peer review of search strategies. In: Higgins, JPT, Thomas, J, Chandler, J, Cumpston, M, Li, T, Page, MJ, et al. , editors. Cochrane Handbook for Systematic Reviews of Interventions Version 6.2 (updated Feb 2021). Cochrane; 2021 [cited 2021 Mar 26]. Available from: www.training.cochrane.org/handbook.Google Scholar
Lefebvre, C, Duffy, S. Peer reviewing search strategies. Summarized Research in Information Retrieval for HTA (SuRe Info). HTAi; 2020 [cited 2020 Dec 14]. Available from: http://vortal.htai.org/index.php?q=node/918.Google Scholar
Mullins, MM, DeLuca, JB, Crepaz, N, Lyles, CM. Reporting quality of search methods in systematic reviews of HIV behavioral interventions (2000-2010): Are the searches clearly explained, systematic and reproducible? Res Synth Methods. 2014;5:116–30.CrossRefGoogle ScholarPubMed
Layton, D. A critical review of search strategies used in recent systematic reviews published in selected prosthodontic and implant-related journals: Are systematic reviews actually systematic? Int J Prosthodont. 2017;30:13–21.CrossRefGoogle ScholarPubMed
Sampson, M, McGowan, J. Errors in search strategies were identified by type and frequency. J Clin Epidemiol. 2006;59:1057–63.CrossRefGoogle ScholarPubMed
Franco, JVA, Garrote, VL, Escobar Liquitay, CM, Vietto, V. Identification of problems in search strategies in Cochrane reviews. Res Synth Methods. 2018;9:408–16.CrossRefGoogle ScholarPubMed
Lefebvre, C, Manheimer, E, Glanville, J. Chapter 6: Searching for studies. In: Higgins, JPT, Green, S, editors. Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 (updated March 2011). The Cochrane Collaboration; 2011 [cited 2020 Dec 14]. Available from: https://training.cochrane.org/cochrane-handbook-systematic-reviews-interventions#previous-versions.Google Scholar
Chandler, J, Churchill, R, Higgins, J, Lasserson, T, Tovey, D. Methodological Expectations of Cochrane Intervention Reviews (MECIR). Standards for the conduct and reporting of new Cochrane Intervention Reviews 2013. Version Dec 2013 [cited 2020 Dec 14]. Available from: https://community.cochrane.org/mecir-manual/key-points-and-introduction/versions-and-changes-mecir.Google Scholar
Sampson, M, McGowan, J, Cogo, E, Grimshaw, J, Moher, D, Lefebvre, C. An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009;62:944–52.CrossRefGoogle ScholarPubMed
McGowan, J, Sampson, M, Lefebvre, C. An evidence based checklist for the Peer Review of Electronic Search Strategies (PRESS EBC). Evid Based Libr Inf Pract. 2010;5:149–54.CrossRefGoogle Scholar
Salvador-Olivan, JA, Marco-Cuenca, G, Arquero-Aviles, R. Errors in search strategies used in systematic reviews and their effects on information retrieval. J Med Libr Assoc. 2019;107:210–21.CrossRefGoogle ScholarPubMed
Grossetta Nardini, HK, Batten, J, Funaro, MC, Garcia-Milian, R, Nyhan, K, Spak, JM, et al. Librarians as methodological peer reviewers for systematic reviews: Results of an online survey. Res Integr Peer Rev. 2019;4: 23.CrossRefGoogle Scholar
Iverson, S, Della Seta, M, Lefebvre, C, Ritchie, M, Traditi, L, Baliozian, K. International health library associations urge the International Committee of Medical Journal Editors (ICMJE) to seek information specialists as peer reviewers for knowledge synthesis publications. JEAHIL. 2020;16:58–61.Google Scholar
McGowan, J, Sampson, M, Salzwedel, DM, Cogo, E, Foerster, V, Lefebvre, C. PRESS – Peer Review of Electronic Search Strategies: 2015 Guideline Explanation and Elaboration (PRESS E&E). Ottawa: CADTH; 2016 [cited 2020 Dec 14]. Available from: https://www.cadth.ca/sites/default/files/pdf/CP0015_PRESS_Update_Report_2016.pdf.Google Scholar
McGowan, J, Sampson, M, Salzwedel, DM, Cogo, E, Foerster, V, Lefebvre, C. PRESS – Peer Review of Electronic Search Strategies: 2015 guideline statement. J Clin Epidemiol. 2016;75:40–6.CrossRefGoogle ScholarPubMed
Cochrane Diagnostic Test Accuracy Review Unit. The Cochrane DTA editorial process. V10. Cochrane; 2016 [cited 2020 Dec 14]. Available from: https://methods.cochrane.org/sdt/sites/methods.cochrane.org.sdt/files/public/uploads/cochrane_dta_editorial_process.pdf.Google Scholar
Cochrane Information Specialists Executive. Peer review of search. White Paper. Cochrane Information Specialists Portal; 2019 [cited 2020 Dec 14]. Available from: https://community.cochrane.org/organizational-info/resources/resources-groups/information-specialists-portal/search-peer-review.Google Scholar
Centre for Reviews and Dissemination. Systematic reviews: CRD's guidance for undertaking reviews in health care. York: University of York; 2009 [cited 2020 Dec 14]. Available from: https://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/SysRev3.htm.Google Scholar
EUnetHTA JA3WP6B2-2 Authoring Team. Process of information retrieval for systematic reviews and health technology assessments on clinical effectiveness. Methodological guidelines. Diemen, The Netherlands: EUnetHTA; 2019 [cited 2020 Dec 14]. Available from: https://www.eunethta.eu/.Google Scholar
Institute of Medicine. Finding what works in health care: Standards for systematic reviews. Washington (DC): The National Academies Press; 2011 [cited 2020 Dec 14]. Available from: https://www.nap.edu/catalog/13059/finding-what-works-in-health-care-standards-for-systematic-reviews.Google Scholar
Institute for Quality and Efficiency in Health Care (IQWiG). IQWiG general methods: Version 6.0. Cologne: Institute for Quality and Efficiency in Health Care; 2020 [cited 2021 Mar 28]. Available from: https://www.iqwig.de/en/about-us/methods/methods-paper/Google Scholar
National Institute for Health and Care Excellence (NICE). Chapter 5: Identifying the evidence: Literature searching and evidence submission. In: Developing NICE guidelines: The manual. Process and methods [PMG20]. London: NICE; 2020 [cited 2020 Dec 14]. Available from: https://www.nice.org.uk/process/pmg20/chapter/identifying-the-evidence-literature-searching-and-evidence-submission.Google Scholar
Rethlefsen, ML, Kirtley, S, Waffenschmidt, S, Ayala, AP, Moher, D, Page, MJ, et al. PRISMA-S: An extension to the PRISMA statement for reporting literature searches in systematic reviews. Syst Rev. 2021;10:39.CrossRefGoogle ScholarPubMed
Page, MJ, McKenzie, JE, Bossuyt, PM, Boutron, I, Hoffmann, TC, Mulrow, CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.CrossRefGoogle ScholarPubMed
Page, MJ, Moher, D, Bossuyt, PM, Boutron, I, Hoffmann, TC, Mulrow, CD, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;372:n160.CrossRefGoogle ScholarPubMed
Agency for Healthcare Research and Quality (AHRQ). Methods guide for effectiveness and comparative effectiveness reviews. AHRQ publication No. 10(14)-EHC063-EF. Rockville (MD): Agency for Healthcare Research and Quality; 2014 [cited 2020 Dec 14]. Available from: https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/cer-methods-guide_overview.pdf.Google Scholar
SuRe Info Project Team. Summarized Research in Information Retrieval for HTA (SuRe Info). HTAi [cited 2020 Dec 14]. Available from: http://www.sure-info.org.Google Scholar
Isojärvi, J, Glanville, J. Evidence-based searching for health technology assessment: Keeping up-to-date with SuRe Info. Int J Technol Assess Health Care. 2021;37(1):e51. doi: 10.1017/S026646232100009X. PMID: 33840393.CrossRefGoogle ScholarPubMed
SuRe Info Project Team. SuRe Info: Authors’ manual. Version 1.1. HTAi; 2017 [cited 2020 Dec 14]. Available from: http://www.sure-info.org.Google Scholar
Relevo, R, Paynter, R. Peer review of search strategies. In: Methods Research Report. (Prepared by the Oregon Evidence-based Practice Center under Contract No. 290-2007-100572) AHRQ Publication No. 12-EHC068-EF. Rockville (MD): Agency for Healthcare Research and Quality; 2012 [cited 2020 Dec 14]. Available from: https://effectivehealthcare.ahrq.gov/sites/default/files/pdf/search-strategies_research.pdf.Google Scholar
Spry, C, Mierzwinski-Urban, M. The impact of the peer review of literature search strategies in support of rapid review reports. Res Synth Methods. 2018;9:521–6.CrossRefGoogle ScholarPubMed
Folb, BL, Klem, ML, Youk, AO, Dahm, JJ, He, M, Ketchum, AM, et al. Continuing education for systematic reviews: A prospective longitudinal assessment of a workshop for librarians. J Med Libr Assoc. 2020;108:36–46.CrossRefGoogle ScholarPubMed
Supplementary material: File

Lefebvre and Duffy supplementary material

Lefebvre and Duffy supplementary material

Download Lefebvre and Duffy supplementary material(File)
File 24.6 KB