Hostname: page-component-546b4f848f-hhr79 Total loading time: 0 Render date: 2023-06-01T11:59:15.542Z Has data issue: false Feature Flags: { "useRatesEcommerce": true } hasContentIssue false

Applying a scoping review approach for identifying effective implementation strategies in oral health settings

Published online by Cambridge University Press:  20 September 2021

Erick G. Guerrero*
I-Lead Institute, Research to End Healthcare Disparities Corp, Los Angeles, CA, USA
Charles D. Kaplan
Neurology Department, University of Southern California, Los Angeles, CA, USA
Inga Gruß
Kaiser Permanente Center for Health Research, Portland, OR, USA
Julie Frantsve-Hawley
Julie Frantsve-Hawley Consulting, CareQuest Institute for Oral Health, Boston, MA, USA
Jeffrey L. Fellows
Kaiser Permanente Center for Health Research, Portland, OR, USA
Nadia Yosuf
Kaiser Permanente Center for Health Research, Portland, OR, USA
Deborah E. Polk
University of Pittsburgh School of Dental Medicine, Pittsburgh, PA, USA
Address for correspondence: E. G. Guerrero, PhD, I-Lead Institute, Research to End Healthcare Disparities Corp, 12300 Wilshire Blvd, Suite 210, Los Angeles, CA 90025, USA. Email:
Rights & Permissions[Opens in a new window]


Dental service providers have limited capacity to identify strategies to implement evidence-based practices (EBPs). We developed a rigorous yet parsimonious scoping review approach to identify, select, and rate implementation strategies based on an oral health system context. From 153 strategies identified, we selected the top 11 strategies, which had a moderate level of support of evidence and where managers were the main actors. The main actions were to educate, remind, structure, and influence. Targets included dentists, dental hygienists, and assistants and managers from a large prepaid dental care delivery system. This approach responds to calls for rapid and innovative methods to implement EBPs in oral health.

Brief Report
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (, which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
© The Author(s), 2021. Published by Cambridge University Press on behalf of The Association for Clinical and Translational Science


Dental care organizations, like other healthcare organizations, are expected to deliver effective care based on evidence-based practices (EBPs) [Reference Clark and Ducharme1]. However, there are often many barriers that challenge the clinical implementation of EBPs. Generally, there is a lack of a process to ensure EBPs are effectively implemented [Reference Waltz, Powell, Fernández, Abadie and Damschroder2]. Implementation strategies, which are the specific arrangements, facilitators, or conditions supporting implementation, are essential tools for healthcare providers to uptake EBPs [Reference Powell, McMillen and Proctor3]. Despite a large number of implementation strategies in use, there is little guidance for healthcare systems to select the strategy that best fits their implementation context [Reference Waltz, Powell and Matthieu4]. Offering healthcare systems an easy to follow, systematic approach to identify, select, and test implementation strategies may improve efforts to deliver EBPs [Reference Ross, Stevenson and Dack5].

There are more than 100 implementation strategies that differ in terms of who initiates implementation efforts, what action is considered in the implementation, who is targeted to carry on the implementation effort, and at what level of influence implementation is conceptualized [Reference Powell, McMillen and Proctor3,Reference Powell, Waltz and Chinman6,Reference Powell, Beidas and Lewis7]. Emerging approaches address these issues of selecting and tailoring implementation strategies [Reference Powell, Waltz and Chinman6]. Concept mapping, for example, is a mixed methods approach used for selecting implementation strategies that can help identify and prioritize factors that may affect the implementation process [Reference Powell, Beidas and Lewis7]. Another approach is conjoint analysis, which quantitatively identifies implementation strategy profiles to indicate implementer preferences [Reference Powell, Beidas and Lewis7]. A third approach is intervention mapping, which enables implementers to gain a better understanding of the existing evidence supporting different pathways of the implementation of specific EBPs or clinical interventions [Reference Powell, Beidas and Lewis7]. One shortcoming of all these approaches is that they are generally designed, led, and deployed by researchers outside the healthcare delivery system rather than by members who actually work within healthcare systems.

To identify and rate implementation strategies clearly and directly, the process of helping healthcare systems classify strategies should be based on foundational definitions that consider several dimensions, such as the actors, action, and target [Reference Proctor, Powell and McMillen8] and level. Each of these elements is described in Table 1.

Table 1. Description of the model of rating based on actor, action, level, target, and evidence

It is critical for health care providers to identify strategies that have been shown through empirical research to be effective along an established continuum of evidence (ranging from anecdotal or small evidence to causal linkage or big evidence) [Reference Rousseau9]. Simultaneously identifying implementation interventions and their respective levels of evidence narrows down the number of implementation interventions to choose from. It also provides an accepted standard of quality of the clinical EBPs to be implemented, all before an investment has to be made to implement the EBP in the field.

Below, we describe a scoping review methodology informed by the Cochran review approach to help healthcare systems in general and dental health systems to identify and classify various evidence-based implementation strategies for stakeholder vetting. Key to the approach is special attention to differences in actors (managers, clinicians, others) and actions (supervising, coaching, guiding) that currently function in the specific system under study. This focus on differentiating the role of actors and actions allows implementers to identify strategies that are most relevant to both the system and to its varying clinical settings. This pragmatic process of scoping implementation strategies puts primary consideration on the lived experience and context of those (providers) who will be most affected by any policy and practice changes. These contextual factors may include external policies and regulation, internal setting conditions, such as workforce issues, training needs, leadership effectiveness, and other factors that may facilitate the implementation of EBPs [Reference Breimaier, Heckemann, Halfens and Lohrmann10].


Setting and Design

This scoping review is part of the Dissemination and Implementation of Sealant Guidelines in Organizations (DISGO) study funded by the NIH (U01DE027452). The DISGO study is a cluster-randomized, stepped-wedge clinical trial of a deliberative loop (DL) implementation intervention for the purpose of improving the uptake of a evidence-based practice guideline on dental sealants at the Kaiser Permanente Northwest (KPNW) Dental program [Reference Polk, Guerrero and Gruß11]. KPNW is characterized by several key inner and outer contextual factors that we thought would influence the implementation of the intervention. KPNW is a large integrated health system that provides prepaid medical and dental care to enrolled members. The KP Dental plan (KPD) provides comprehensive care to over 270,000 patients at 21 dental offices in Northwest Oregon and Southwest Washington. Permanente Dental Associates (PDA), a large group practice of general dentists and specialists, provides prepaid dental services to KPD enrollees. The regulation and leadership style of KPD were contextual factors that were also considered. Our study aimed to identify strategies that could be introduced in the DL implementation intervention that are responsive to Kaiser’s staff needs and sensitive to the characteristic contextual factors of the KPNW dental care system. The questions addressed by the scoping review include the role of actor, actions, and level of actions to improve the uptake of EBPs in dental clinics. We relied on the Cochran review methodology [Reference Chandler and Hopewell12] and systematic reviews of implementation strategies in oral health [Reference Ástvaldsdóttir, Boström and Davidson13] to inform the core steps of our scoping review.

Identifying Implementation Strategies in Peer-Reviewed Publications

In the first stage, we, (IG, CK, EG) identified the peer-reviewed publications with evidence for specific implementation strategies in healthcare settings. We searched five primary electronic databases that included Google Scholar, PubMed, PsycINFO, Cochrane Library, and EBSCO. Our literature searches included the following keywords: implementation strategies, implementation approaches, and implementation facilitators. The inclusion criteria were informed by guidelines used in systematic reviews in implementation science [Reference Waltz, Powell and Matthieu4,Reference Powell, Waltz and Chinman6] and included implementation strategies, EBPs, healthcare, in English language, years 2000–2019. Reviewers were calibrated to the selected criteria. Two rounds of screening were conducted based on abstracts in the first round and full text in the second round. The identified citations were split across the three reviewers, with random audits to verify fidelity to the criteria.

Systematically Selecting Implementation Strategies Relevant to KPD

At the second stage, we (IG, CK, EG) sorted all implementation strategies into the following categories: dissemination strategies, implementation process strategies, capacity-building strategies, and scale-up strategies using the analytical framework of the role of actor, actions, and level of actions [Reference Proctor, Powell and McMillen8]. While sorting strategies into these categories, we removed duplicate strategies. We also removed strategies that were not relevant to the study focus area (e.g. influence policymaking, encourage patients to demand a treatment) or not feasible in the context of KPD (e.g. due to resource constraints). Strategies were screened via rater’s group discussion. We relied on the expert opinion of IG’s organizational expertise at KPD (e.g., leadership and managerial style, organizational norms, and practices) and the expertise in implementation from EG (outer and inner context factors, particularly regulation and leadership style) and clinical interventions (EBPs acceptability and readiness). In this stage, we were guided in the sorting by our main research question of what strategies for implementing the DD would be most relevant to improving the KPD system’s uptake of the EBP of placing dental sealants?


In this third stage, the three raters scored the relevant implementation strategies based on level of evidence. To do so, we conducted an additional targeted literature search to determine the level of available evidence that supports each identified strategy. We adopted a scoring criterion based on Rosseau’s [Reference Rousseau9] notion of small evidence (grey literature, qualitative) and big evidence (experimental studies leading to metanalysis), whereby 1 represents small evidence and 5 represents big evidence. Raters assigned scores independently and in duplicate, discussed discrepancies in ratings, and reached consensus on final ratings. All strategies rated 2 and lower were removed. The remaining strategies were reviewed by KPD’s quality and operations leadership to assess implementation feasibility, including alignment with organizational vision, timing with competing priorities, and potential resource needs.

Classifying Implementation Strategies by Actor, Action, Level, and Target

The final stage consisted of classifying strategies independently and in duplicate based on Proctor et al., [Reference Proctor, Powell and McMillen8] framework of actor, action, [level], and target as described in the Introduction. The main criteria that guided the classification were clarity in conceptualization and distinctness of the strategy from other strategies. Raters discussed the strategies based on the criteria and reached a consensus on the best practices. The final list of strategies is presented in the following narrative.


The literature search resulted in 3970 articles. After screening, 123 articles met the inclusion criteria and yielded 153 strategies. Many studies reported more than one strategy. We did not report implementation interventions that were not used for uptake purposes. Figure 1 provides a schematic representation of the identification process. After removing duplicate and irrelevant strategies, 42 strategies remained. After scoring the strategies based on level of evidence, 28 with a rating of low evidence (score of 1–2) were removed, leaving 14 implementation strategies with an evidence score of 3–5. Three additional strategies were removed based on their low feasibility to be implemented in dental clinics. Eleven final strategies resulted from this review. These strategies considered the expert opinion of KPD’s Associate Director based upon the criteria for implementation and feasibility context (e.g., hectic, fast pace, and task-driven work setting) to adopt, follow, and adhere to EBPs such as the intervention considered in the parent study, i.e. dental sealant guidelines.

Fig. 1. Process of the scoping articles. KP, Kaiser Permanente.

The 11 strategies (e.g., audit and provide feedback, creating learning environment, assign, and deploy experts) (Table 2) had in common that they all relied on full organization operation, interaction, and technology and human resources. For instance, Audit and provide feedback, as a system strategy has three main actions, referred sometimes in the literature as steps: (1) formulate specific goals and provide agreed-upon feedback; (2) provide protected time for implementation; and (3) evaluate results [Reference Tuti, Nzinga and Njoroge14]. The strategy of creating a learning collaborative includes nine steps: (1) develop a framework for online learning via chatrooms; (2) develop listservs; (3) use text messages; (4) conduct educational meetings; (5) conduct outreach visits; (6) deliver training in innovation; (7) develop and distribute educational material; and (8) activate academic partnerships (i.e., Ebert et al. [Reference Ebert, Amaya-Jackson, Markiewicz, Fairbank, McHugh and Barlow15]). Finally, assign/train and deploy sealant expert in each clinic relies on four steps: (1) identify early adopters identified by colleagues; (2) assign/train NCCL champion for each clinic; (3) provide clinical supervision; and (4) use the train-the-trainer. Our categorization of actors, action, level, and target revealed the following mutually inclusive categories:

  • Actor: six strategies focused on leadership, two on supervisors or managers, and five on staff.

  • Action: four focused on education, three on reminders, five on structure, and four on influence relevant to oral health.

  • Level: seven strategies focused on unit, two in organization, one in management, and one in clinic.

  • Target: five focused on all staff, three in dentist staff, and three in leadership/management.

Table 2. Actor, action, level, target, and evidence for identified implementation strategies

IT: information technology; KPD: Kaiser Permanente Dental; NCCL: non-cavitated carious lesions; TPM, Total Productive Maintenance.


This study presented a description of a pragmatic approach to scoping implementation strategies that are contextually relevant to local dental settings. We relied on a process of identifying, selecting, scoring, and classifying implementation strategies for uptaking EBPs in a real-world health care system. Similar methodological approaches have been developed in other areas of health care [Reference Waltz, Powell and Matthieu4], but our approach considered that health care practitioners have the most knowledge of whether a given implementation strategy would be feasible and acceptable.

Our approach highlights the parsimony and transparency needed in the field to improve the rigor in the clinical decision-making of health care providers at all levels. Consistent with other reviews in dental practice [Reference Polk, Guerrero and Gruß11], selected implementation strategies focused on education, reminders, and multifaceted implementation that would apply to the DD intervention. These 11 strategies will be provided in the next stage to KPD practitioners as options to consider in implementing an EBP (e.g., sealant guideline). This approach can inform research designs that highlight the importance of context to improve the implementation process.

Health care providers can determine which among these 11 strategies to select, test, and implement. These systems generally consider a series of factors before committing to implementing a new practice. A few of these factors include cost of the implementation efforts [Reference Bowser, Henry and McCollister16], likelihood of implementation success, utility of the implemented practice, and coherence and fit of the new practice with the culture and/or climate of the organization [Reference Villarosa, Maneze, Ramjan, Srinivas, Camilleri and George17].

Our results add a level of rigor to clinical decision-making by reducing the otherwise ambiguous and resource-intensive process of identifying and selecting implementation strategies that can help health care systems effectively deliver EBPs. Engaging stakeholders in the initial steps of implementation can directly contribute to validating their views to improve implementation strategies and delivery of new practices in a real-world health care context.


The proposed approach is limited based on its selection criteria, available literature review, and reach. We limited our search to English-language articles. The scoping review focused on health care systems in the USA and was conducted from research team members also involved in the parent study. This could be a source of unconscious selective bias. Our scoping review is limited in its reliance on existing implementation strategies published in the literature. It does not allow for new strategies that may be suited for the context to be developed anew. Another limitation is that there may be other categorization schemes that may be consistent with the context. We also did not directly evaluate the effectiveness of the proposed implementation strategies but instead relied on the level of evidence in the literature as an indicator of effectiveness. Despite these limitations, the approach presented in this study offers practitioners with a rigorous yet parsimonious way to improve decision-making about what strategies to employ in the implementation of EBPs.


Our pragmatic approach to scoping implementation strategies provides managers and implementers in dental health care organizations with a pragmatic methodology to select implementation strategies that are context-dependent and supported by an acceptable degree of evidence. This can limit the degree of arbitrary decision-making in selecting strategies in real-world healthcare settings, a critical concern [Reference Guerrero, Fenwick and Kong18Reference Birken, Clary and Tabriz20]. Our approach can deliver a useful tool for bringing order to the chaos of conflicting implementation strategies and give the health care professionals who will be affected the most by changes in policy a greater say in the decisions that affect the trajectories of their respective dental organizations. Using our methodology can inform policy that empowers dental organizations to conduct their own scoping reviews.


The research described was supported in part by the National Institutes of Dental Craniofacial Research (U01DE027452) and Research to End Healthcare Disparities Corp. Funders had no further role in study design; in the collection, analysis, and interpretation of data; in the writing of the report; or in the decision to submit the paper for publication.


The authors have no conflicts of interest to declare.

Previous presentations

This work was presented at the 13th Annual Conference on Dissemination and Implementation Science in December 2020, where it was rated as the top poster in its division.


Clark, DB, Ducharme, L. Charting a course for implementation research in oral health. JDR Clinical & Translational Research 2016; 1(3): 198200. DOI 10.1177/2380084416662846.10.1177/2380084416662846CrossRefGoogle ScholarPubMed
Waltz, TJ, Powell, BJ, Fernández, ME, Abadie, B, Damschroder, LJ. Choosing implementation strategies to address contextual barriers: diversity in recommendations and future directions. Implementation Science 2019; 14(1): 42. DOI 10.1186/s13012-019-0892-4.10.1186/s13012-019-0892-4CrossRefGoogle ScholarPubMed
Powell, BJ, McMillen, JC, Proctor, EK, et al. A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review 2012; 69(2): 123157. DOI 10.1177/1077558711430690.10.1177/1077558711430690CrossRefGoogle ScholarPubMed
Waltz, TJ, Powell, BJ, Matthieu, MM, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implementation Science 2015; 10(1): 109. DOI 10.1186/s13012-015-0295-0.10.1186/s13012-015-0295-0CrossRefGoogle ScholarPubMed
Ross, J, Stevenson, F, Dack, C, et al. Developing an implementation strategy for a digital health intervention: an example in routine healthcare. BMC Health Services Research 2018; 18(1): 794. DOI 10.1186/s12913-018-3615-7.10.1186/s12913-018-3615-7CrossRefGoogle ScholarPubMed
Powell, BJ, Waltz, TJ, Chinman, MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science 2015; 10(1): 21. DOI 10.1186/s13012-015-0209-1.10.1186/s13012-015-0209-1CrossRefGoogle ScholarPubMed
Powell, BJ, Beidas, RS, Lewis, CC, et al. Methods to improve the selection and tailoring of implementation strategies. The Journal of Behavioral Health Services & Research 2017; 44(2): 177194. DOI 10.1007/s11414-015-9475-6.10.1007/s11414-015-9475-6CrossRefGoogle ScholarPubMed
Proctor, EK, Powell, BJ, McMillen, JC. Implementation strategies: recommendations for specifying and reporting. Implementation Science 2013; 8(1): 139. DOI 10.1186/1748-5908-8-139.10.1186/1748-5908-8-139CrossRefGoogle ScholarPubMed
Rousseau, DM. Is there such a thing as “Evidence-Based Management”? Academy of Management Review 2006; 31(2): 256269. DOI 10.5465/amr.2006.20208679.10.5465/amr.2006.20208679CrossRefGoogle Scholar
Breimaier, HE, Heckemann, B, Halfens, RJG, Lohrmann, C. The Consolidated Framework for Implementation Research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC Nursing 2015; 14(1): 43. DOI 10.1186/s12912-015-0088-4.10.1186/s12912-015-0088-4CrossRefGoogle Scholar
Polk, DE, Guerrero, EG, Gruß, I, et al. Study protocol: a stepped-wedge, cluster-randomized trial of the effectiveness of a deliberative loop in identifying implementation strategies for the adoption of a dental sealant guideline in dental clinics. Implementation Science Communications 2021; 2(1): 96. DOI 10.1186/s43058-021-00199-6.10.1186/s43058-021-00199-6CrossRefGoogle ScholarPubMed
Chandler, J, Hopewell, S. Cochrane methods--twenty years experience in developing systematic review methods. Systematic Reviews 2013; 2(1): 76. DOI 10.1186/2046-4053-2-76.10.1186/2046-4053-2-76CrossRefGoogle Scholar
Ástvaldsdóttir, Á., Boström, A-M, Davidson, T, et al. Oral health and dental care of older persons-A systematic map of systematic reviews. Gerodontology 2018; 35(4): 290304. DOI 10.1111/ger.12368.10.1111/ger.12368CrossRefGoogle ScholarPubMed
Tuti, T, Nzinga, J, Njoroge, M, et al. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implementation Science 2017; 12(1): 61. DOI 10.1186/s13012-017-0590-z.10.1186/s13012-017-0590-zCrossRefGoogle ScholarPubMed
Ebert, L, Amaya-Jackson, L, Markiewicz, J, Fairbank, JA. Development and application of the NCCTS learning collaborative model for the implementation of evidence-based child trauma treatment. In: McHugh, RK, Barlow, DH, eds. Dissemination and Implementation of Evidence-Based Psychological Interventions. New York: Oxford University Press; 2012. 97123.Google Scholar
Bowser, DM, Henry, BF, McCollister, KE. Cost analysis in implementation studies of evidence-based practices for mental health and substance use disorders: a systematic review. Implementation Science 2021; 16(1): 26. DOI 10.1186/s13012-021-01094-3.10.1186/s13012-021-01094-3CrossRefGoogle ScholarPubMed
Villarosa, AR, Maneze, D, Ramjan, LM, Srinivas, R, Camilleri, M, George, A. The effectiveness of guideline implementation strategies in the dental setting: a systematic review. Implementation Science 2019; 14(1): 106. DOI 10.1186/s13012-019-0954-7.10.1186/s13012-019-0954-7CrossRefGoogle ScholarPubMed
Guerrero, EG, Fenwick, K, Kong, Y. Advancing theory development: exploring the leadership-climate relationship as a mechanism of the implementation of cultural competence. Implementation Science 2017; 12(1): 133. DOI 10.1186/s13012-017-0666-9.10.1186/s13012-017-0666-9CrossRefGoogle ScholarPubMed
Mackie, TI, Schaefer, AJ, Hyde, JK, et al. The decision sampling framework: a methodological approach to investigate evidence use in policy and programmatic innovation. Implementation Science 2021; 16(1): 24. DOI 10.1186/s13012-021-01084-5.10.1186/s13012-021-01084-5CrossRefGoogle ScholarPubMed
Birken, S, Clary, A, Tabriz, AA, et al. Middle managers’ role in implementing evidence-based practices in healthcare: a systematic review. Implementation Science 2018; 13(1): 149. DOI 10.1186/s13012-018-0843-5.10.1186/s13012-018-0843-5CrossRefGoogle ScholarPubMed
Meyers, DC, Durlak, JA, Wandersman, A. The quality implementation framework: a synthesis of critical steps in the implementation process. American Journal of Community Psychology 2012; 50(3–4): 462480. DOI 10.1007/s10464-012-9522-x.10.1007/s10464-012-9522-xCrossRefGoogle ScholarPubMed
Wandersman, A, Duffy, J, Flaspohler, P, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. American Journal of Community Psychology 2008; 41(3–4): 171181. DOI 10.1007/s10464-008-9174-z.10.1007/s10464-008-9174-zCrossRefGoogle Scholar
Kanter, JW, Busch, AM, Rusch, LC. Behavioral Activation: Distinctive Features. London: Routledge, 2009.10.4324/9780203876060CrossRefGoogle Scholar
Figure 0

Table 1. Description of the model of rating based on actor, action, level, target, and evidence

Figure 1

Fig. 1. Process of the scoping articles. KP, Kaiser Permanente.

Figure 2

Table 2. Actor, action, level, target, and evidence for identified implementation strategies