Skip to main content Accessibility help


  • Access


      • Send article to Kindle

        To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        The writer’s guide to education scholarship in emergency medicine: Systematic reviews and the scholarship of integration (part 4)
        Available formats

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        The writer’s guide to education scholarship in emergency medicine: Systematic reviews and the scholarship of integration (part 4)
        Available formats

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        The writer’s guide to education scholarship in emergency medicine: Systematic reviews and the scholarship of integration (part 4)
        Available formats
Export citation



Reviews help scholars consolidate evidence and guide their educational practice. However, few papers describe how to effectively publish review papers. We completed a scoping review to develop a set of quality indicators that will assist junior authors to publish reviews and integrative scholarship.


MEDLINE, Embase, ERIC, and Google Scholar were searched for English language articles published between 2012 and January 2016 using the terms review, medical education, how to publish, and emergency medicine. Titles and abstracts were reviewed by two authors and included if they focused on how to publish a review or outlined reporting guidelines of reviews. The articles were reviewed in parallel for calibration, and disagreements were resolved through a consensus.


A full text review of the 25 articles was conducted, and 196 recommendations were extracted from 13 articles. A hand search of the included articles’ reference lists and expert recommendation found an additional eight articles. These recommendations were thematically analysed into a list of seven themes and 32 items. Additionally, seven evaluation tools and reporting guidelines were found to guide researchers in optimizing their reviews for publication.


In emergency medicine education, review articles can help synthesize educational research so that educators can engage in evidence-based scholarly teaching. We hope that this work will act as an introduction to those interested in engaging in integrative scholarship by providing them with a guide to key quality markers and important checklists for improving their research.


In his report for the Carnegie Foundation for the Advancement of Teaching in 1990, Ernest L. Boyer called for a new definition of scholarship, proposing a view that includes discovery, integration, application, and teaching. 1 Review articles fit into Boyer’s scholarship of integration, aiming to synthesize existing evidence and to provide a comprehensive overview of the current understanding of a specific area or focus. 2 They provide readers with an understanding of what research has already been done, what research has yet to be done, and where the gaps in knowledge exist within the totality of evidence on a particular topic.

Reviewing the literature for one’s own edification in medical education

For the early career scholar or for the experienced researcher entering a new field, an overview of the current literature is essential to gain understanding and context. 3 Scholars benefit from completing informal topic reviews of the key concepts within a field. Additionally, because education research lends itself to many different perspectives and research techniques, junior medical education researchers may find it useful to review the literature to gain a better understanding of how various scientists use different methods to approach the same question. Reviewing the methods used in the field of medical education can be quite helpful for junior educators looking to conduct new research. 4

Concepts in medical education span across multiple disciplines, so review search strategies must go beyond the traditional databases (i.e., PubMed) and include others such as PsychInfo, ERIC, Google Scholar, and so forth. 5 A knowledgeable mentor or an expert-recommended foundational paper can serve as a good starting point for these informal reviews. Although topic and methodological reviews are not generally accepted for publication in journals today, successful researchers use these to formulate pertinent research questions and to determine the most appropriate study design for their inquiry.

Reviewing the literature to conduct the scholarship of integration

Traditionally, the health care field has concentrated on methodologically rigorous systematic reviews concentrated on quantifying the effects of interventions or the accuracy of diagnostic tests. 6 However, many other types of knowledge synthesis methods exist, several of which are better suited to the complex qualitative data synthesis often required in medical education. 5 Non-systematic, narrative reviews integrate research to identify new insights, whereas systemic reviews summarize research on a focused topic and evaluate quality of existing bodies of evidence. 3 The least rigorous review articles, commentary papers and summary documents, are usually written by experts in the field and integrate concepts and a large body of literature to provide a short overview for readers.

The heterogeneity of studies within medical education presents unique challenges, particularly with the systematic review. 7 Additional challenges stem from its history of intuition-based practice and the myriad of ideological backgrounds of medical education authors. 5 Historically, these challenges have led to a body of evidence lacking rigor and interest compared to the more traditional clinical trials in medicine.

The primary objective of this publication is to perform a scoping review to collate guidance on publishing review articles in medical education. It is meant for novice educators and hopes to provide a synthesis summary of existing recommendations, but not to deliver an in-depth description of review types and specific methodologies. We hope that it will provide guidance to junior authors writing medical education reviews in emergency medicine (EM).


Search strategy

A scoping review was undertaken to collate existing literature on how to publish a review in EM medical education. All efforts were made to adhere to Arksy and O’Malley’s framework. 9 , 10 A systematic database search was undertaken with an expert librarian using MEDLINE, Embase, ERIC, and Google Scholar. Each search was limited to English-language journal articles and assessed in parallel by two independent authors (KW and AM). Articles were excluded if they were duplicates, deemed redundant, or did not address the focused research question on how to publish a systematic review as determined by a consensus review. A senior author was available to mediate any disagreements if a consensus could not be reached, and authors erred on the side of inclusion when a disagreement ensued.

Initially, MEDLINE was searched using “and/or” combinations of variations of keywords: “medical education,” “review literature as a topic,” “EM,” and “conducting systematic reviews.” A second search of Embase was performed using “and/or” combinations of the following keywords: “medical education,” “review,” “publishing,” “writing,” and “EM.” A third search of ERIC was performed using the key phrases, “how to publish a review” and “medical education.” The differences in search terms were due to the lack of consistent terminology among databases. Search terms needed to be altered in consultation with the librarian based on taxonomy specific to each database and outputs. The first 500 titles sorted by relevance as per the ERIC search engine were reviewed for relevance and duplicates. A final search was undertaken via Google Scholar using the following three phrases: “how to publish a review in medical education,” “how to publish a review in medicine,” and “how to publish a review in EM.” The first 500 titles, as sorted by relevance as per the Google Scholar search engine, were screened. 11 Figure 1 shows that 2,489 titles were screened initially, of which 2,444 were excluded based on irrelevance, language, or publication type.

Figure 1 Flow diagram of the review process.

One reviewer (KW) also performed a supplemental hand search of 15 EM and 12 medical education journals from 2014-2016 to augment the database. These 27 journals were chosen from a list of all MEDLINE-indexed journals in EM and medical education as reviewed by a co-investigator (AH) for current activity and appropriate focus. Journals were hand-searched for author guidelines, recommended quality evaluation tools, and relevant articles pertaining to the publication of systematic reviews. Additional articles were obtained through expert consultation and ancestry searching of relevant article reference lists.

Combining the initial database search, relevant titles were obtained through hand searching and expert recommendations; a total of 73 abstracts were screened in parallel by two authors (KW and AM). From these, 24 full-text articles were read in detail and assessed for relevance. Thirteen articles were deemed appropriate and included in guideline synthesis on how to publish a review in medical education.


Two investigators independently reviewed the final 13 selected full-text articles to extract quality markers, and a master list was created. These quality markers were analysed with a thematic analysis. 12 , 13 Audit trails were created, and a final list was generated based on a consensus review by two primary authors (KW and AM).


Thematic analysis

Thirty-two quality indicators emerged along seven themes: pitfalls to avoid, preparation, introduction, methods, results, discussion, and funding (Table 1).

Table 1 Quality indicators

Author guidelines

Tables 2 and 3 outline specific guidelines for review articles associated with their respective journals obtained through hand searching of relevant EM and medical education journals that publish review articles.

Table 2 Medical education journals that accept review articles

Table 3 EM journals that accept review articles

Quality evaluation tools and reporting guidelines

Seven quality evaluation tools and reporting guidelines for reviews were identified and are outlined in Table 4. These quality checklists serve as a useful resource for the implementation and quality evaluation of review research.

Table 4 Brief overview of several review types and associated resources


We have performed a thorough literature search to develop a list of recommendations and to collate guidelines that will assist authors in publishing reviews in EM medical education.

Our findings demonstrate that while there is an abundance of research dedicated to how to perform a review, very little research exists on how to write or publish a review. Additionally, the vast majority of papers focused on systematic reviews, leaving our paper void of recommendations for other review types. There are several explanations for this finding. Although the health care field is familiar with systematic reviews, the remaining types of review studies are less common, with a corresponding decrease in published research relating to non-systematic reviews. The lack of consistent terminology for the other types of knowledge synthesis methods further complicated our literature search. With such a heterogeneity of review studies available in medical education, a prominence of systematic reviews in the literature, in combination with the lack of consistent terminology for the other types of synthesis methods, resulted in our focus on systematic reviews.

Although the collated recommendations were broad, we hope that the quality indicators will be useful to junior researchers in assisting them to successfully publish a review in medical education. Additionally, several quality checklists and reporting guidelines were found applicable to various types of reviews. We hope these guidelines will be useful to junior scholars during the design, development, and publications stages of their scholarship.


One limitation of the review is that very few articles answered the question of “how to get published,” thus limiting our search. Despite the assistance of two librarians and the vast database selection, our highest yield publications came from expert recommendation. One notable difficulty in studying the review literature was the lack of consistent terminology. Search terms had to be altered with the consultation of a librarian based on taxonomy specific to each database and outputs. We were left with a few very sensitive but non-specific search strategies, producing many irrelevant articles to scan through. This increases the chance that relevant articles were missed and highlights the importance of consistency in the reporting of review articles. Because most papers focused on systematic reviews, recommendations for other review types were lacking.


Review papers permit readers to quickly update their knowledge of a given topic through the synthesis of education research and support educators to use evidence-based practices. This research identified key recommendations to enable those interested in engaging in integrative scholarship by providing them with a guide to key quality markers and important checklists for improving their work in the writing and publishing phases.

Competing interests: None declared.


1. Boyer, E, Moser, D, Ream, TC, Braxton, JM. Scholarship Reconsidered: Priorities of the Professoriate. San Francisco: John Wiley & Sons; 2015.
2. Grady, EC, Roise, A, Barr, D, et al. Defining scholarly activity in graduate medical education. J Grad Med Educ 2012;4(4):558-561, 10.4300/JGME-D-12-00266.1.
3. Harvey, BJ, Lang, ES, Frank, JR. The Research Guide: A Primer for Residents, Other Health Care Trainees, and Practitioners. Ottawa: Royal College of Physicians and Surgeons of Canada; 2011.
4. Boet, S, Sharma, S, Goldman, J, Reeves, S. Medical education research: an overview of methods. Can J Anesth 2012;59:159-170, 10.1007/s12630-011-9635-y.
5. Cook, DA, West, CP. Conducting systematic reviews in medical education: A stepwise approach. Med Educ 2012;46(10):943-952, 10.1111/j.1365-2923.2012.04328.x.
6. Rowe, BH, Alderson, P. The Cochrane Library: A Resource for Clinical Problem Solving in Emergency Medicine. Ann Emerg Med 1999;34(1):6-10.
7. Sherbino, J, van Melle, E, Bandiera, G, et al. Education scholarship in emergency medicine part 1: innovating and improving teaching and learning. CJEM 2014;15(Suppl 1):S1-s5, doi:10.2310/8000.2014.141454.
8. Reed, DA, Cook, DA, Beckman, TJ, Levine, RB, Kern, DE, Wright, SM. Association between funding of Published Medical Education Research. JAMA 2007;298(9):1002-1009, 10.1001/jama.298.9.1002.
9. Arskey, H, O’Malley, L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol 2004;8(1):19-32.
10. Colquhoun, HL, Levac, D, O’Brien, KK, et al. Scoping reviews: Time for clarity in definition, methods, and reporting. J Clin Epidemiol 2014;67(12):1291-1294, 10.1016/j.jclinepi.2014.03.013.
11. Chan, TM, Wallner, C, Swoboda, TK., Leone K a, Kessler C. Assessing interpersonal and communication skills in emergency medicine. Acad Emerg Med 2012;19(12):1390-1402, 10.1111/acem.12030.
12. Frank, JR, Mungroo, R, Ahmad, Y, Wang, M, De Rossi, S, Horsley, T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach 2010;32(8):631-637, 10.3109/0142159X.2010.500898.
13. Paterson, QS, Thoma, B, Milne, WK, Lin, M, Chan, TM. A Systematic Review and Qualitative Analysis to Determine Quality Indicators for Health Professions Education Blogs and Podcasts. J Grad Med Educ 2015;7(4):549-554.
14. Bordage, G. Reasons reviewers reject and accept manuscripts: the strengths and weaknesses in medical education reports. Acad Med 2001;76(9):889.
15. Kelly, KD, Travers, A, Dorgan, M, Slater, L, Rowe, BH. Evaluating the quality of systematic reviews in the emergency medicine literature. Ann Emerg Med 2001;38(5):518-526, 10.1067/mem.2001.115881.
16. Moher, D, Liberati, A, Tetzlaff, J, Altman, DG, Grp, P. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement (Reprinted from Annals of Internal Medicine). Phys Ther 2009;89(9):873-880, 10.1371/journal.pmed.1000097.
17. Hammick, M, Dornan, T, Steinert, Y. BEME Guide: Conducting a best evidence systematic review. Part 1: From idea to data coding. BEME Guide No. 13. Med Teach 2010;32(1):3-15, 10.3109/01421590903414245.
18. Sharma, R, Gordon, M, Dharamsi, S, Gibbs, T. Systematic reviews in medical education: a practical approach: AMEE guide 94. Med Teach 2015;37(2):108-124, 10.3109/0142159X.2014.970996.
19. Bhanji, F, Cheng, A, Frank, JR, Snell, L, Sherbino, J. Education scholarship in emergency medicine part 3: a “how-to” guide. CJEM 2014;16(Suppl 1):S13-s18, doi:10.2310/8000.2014.141456.
20. Reeves, S, Koppel, I, Barr, H, Freeth, D, Hammick, M. Twelve tips for undertaking a systematic review. Med Teach 2002;24(4):358-363, 10.1080/01421590220145707.
21. Conn, VS, Isaramalai, S, Rath, S, Jantarakupt, P, Wadhawan, R, Dash, Y. Beyond MEDLINE for literature searches. J Nurs Scholarsh 2003;35(2):177-182, 10.1111/j.1547-5069.2003.00177.x.
22. Masic, I. How to Search, Write, Prepare and Publish the Scientific Papers in the Biomedical Journals. Acta Inform Medica 2011;19(2):68-79.
23. Moher, D, Liberati, A, Tetzlaff, J, Altman, DG. Academia and Clinic Annals of Internal Medicine Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Annu Intern Med 2009;151(4):264-269, 10.1371/journal.pmed1000097.
24. Zed, PJ, Rowe, BH, Loewen, P, Abu-Laban, RB. Systematic reviews in emergency medicine: Part II. Critical appraisal of review quality, data synthesis and result interpretation. CJEM 2003;5(6):406-411.
25. Reeves, S, Peller, J, Goldman, J, Kitto, S. Ethnography in qualitative educational research: AMEE Guide No. 80. Med Teach 2013;35(8):e1365-e1379, 10.3109/0142159X.2013.804977.
26. Stroup, DF. Meta-analysis of Observational Studies in Epidemiology: A Proposal for Reporting. JAMA 2000;283(15):2008, 10.1001/jama.283.15.2008.
27. Gordon, M, Trevor, Gibbs, Gibbs T., STORIES statement: publication standards for healthcare education evidence synthesis. BMC Med 2014;12(1):143, 10.1186/s12916-014-0143-0.
28. Whiting, PF, Rutjes, AWS, Westwood, ME, et al. QUADAS-2: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies. Ann Intern Med 2011;155(8):529-536, 10.7326/0003-4819-155-8-201110180-00009.
29. Tong, A, Flemming, K, McInnes, E, Oliver, S, Craig, J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol 2012;12(1):181, 10.1186/1471-2288-12-181.
30. Wong, G, Greenhalgh, T, Westhorp, G, Buckingham, J, Pawson, R. RAMESES publication standards: Meta-narrative reviews. J Adv Nurs 2013;69(5):987-1004, 10.1111/jan.12092.
31. Wong, G, Greenhalgh, T, Westhorp, G, Pawson, R. Development of methodological guidance, publication standards and training materials for realist and meta-narrative reviews: the RAMESES (Realist And Meta-narrative Evidence Syntheses – Evolving Standards) project. Heal Serv Deliv Res 2014;2(30):1-252, 10.3310/hsdr02300.
32. Thistlethwaite, J, Hammick, M. The Best Evidence Medical Education (BEME) Collaboration: Into the next decade. Med Teach 2010;32(11):880-882, 10.3109/0142159X.2010.519068.