Hostname: page-component-cd9895bd7-gvvz8 Total loading time: 0 Render date: 2024-12-12T08:59:39.297Z Has data issue: false hasContentIssue false

Bootstrapping implementation research training: A successful approach for academic health centers

Published online by Cambridge University Press:  26 July 2021

Kathleen R. Stevens*
Affiliation:
Institute for Integration of Medicine & Science, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA School of Nursing, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Elisabeth de la Rosa
Affiliation:
Institute for Integration of Medicine & Science, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Robert L. Ferrer
Affiliation:
Institute for Integration of Medicine & Science, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA School of Public Health, San Antonio Regional Campus, University of Texas Health Science Center Houston, San Antonio, TX, USA
Erin P. Finley
Affiliation:
School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA South Texas Veteran Health Care System, San Antonio, TX, USA VA Greater Los Angeles Healthcare System, Los Angeles, CA, USA
Bertha E. Flores
Affiliation:
School of Nursing, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Dana A. Forgione
Affiliation:
School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA School of Public Health, San Antonio Regional Campus, University of Texas Health Science Center Houston, San Antonio, TX, USA Texas A&M Corpus Christi College of Business, Corpus Christi, TX, USA
Polly Hitchcock Noel
Affiliation:
Institute for Integration of Medicine & Science, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA South Texas Veteran Health Care System, San Antonio, TX, USA
Timothy A. Reistetter
Affiliation:
School of Health Professions, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Melissa Valerio-Shewmaker
Affiliation:
Institute for Integration of Medicine & Science, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA School of Medicine, University of Texas Health Science Center at San Antonio, San Antonio, TX, USA
Kevin C. Wooten
Affiliation:
Office of the President University of Houston Clear Lake, Clear Lake City, TX, USA Institute for Translational Sciences University of Texas Medical Branch, Galveston, TX, USA
*
Address for correspondence: Dr. K. R. Stevens, RN, EdD, ANEF, FAAN, Castella Endowed Distinguished Professor, Telephone: 210.567.3135; School of Nursing and Institute for Integration of Medicine & Science, University of Texas Health Science Center San Antonio MSC 7949, 7703 Floyd Curl Drive, San Antonio, TX78229-3900, USA. Email: stevensk@uthscsa.edu
Rights & Permissions [Opens in a new window]

Abstract

Demand for building competencies in implementation research (IR) outstrips supply of training programs, calling for a paradigm shift. We used a bootstrap approach to leverage external resources and create IR capacity through a novel 2-day training for faculty scientists across the four Texas Clinical & Translational Science Awards (CTSAs). The Workshop combined internal and external expertise, targeted nationally established IR competencies, incorporated new National Institutes of Health/National Cancer Institute OpenAccess online resources, employed well-known adult education principles, and measured impact. CTSA leader buy-in was reflected in financial support. Evaluation showed increased self-reported IR competency; statewide initiatives expanded. The project demonstrated that, even with limited onsite expertise, it was possible to bootstrap resources and build IR capacity de novo in the CTSA community.

Type
Brief Report
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2021. Published by Cambridge University Press on behalf of The Association for Clinical and Translational Science

Rationale for Novel Curricular Approach

Despite increasing emphasis on implementation research (IR), experts and traditional training opportunities are in short supply [Reference Ford, Rabin, Morrato and Glasgow1,2], highlighting the need for innovative training approaches. Bootstrapping refers to a self-starting process to better oneself in the face of limited resources. In a paradigm shift, four (4) Texas CTSAs used this novel approach to create capacity in the face of minimal internal expertise and limited IR resources. We sought to provide and evaluate a high-quality regional training program to increase IR scientific workforce capacity and enhance grant success. This paper discusses how generalizable best education practices and adult education methods in professional training were applied: document the educational need; establish competencies as learning outcomes; provide ongoing educational resources; use active instructional strategies; and determine impact using an evaluation model.

Unmet Need for Educational Gap

Although IR is not a formalized element of the CTSA Program, it is recognized to be essential in the translational science enterprise to move research into practice and enhance population health [Reference Liverman, Schultz, Terry and Leshner3,Reference Leppin, Mahoney and Stevens4] and is included in the National Center for Advancing Translational Sciences translational science spectrum [5]. As a nascent field, the nature of IR itself has been recast to ensure actionable and relevant findings [Reference Meissner, Glasgow and Vinson6]. This resulted in the emergence of new paradigms, theories, designs, and methods. This fact turns even accomplished researchers into IR novices who seek training in these new methods. Nationally, only half of CTSAs surveyed reported IR training efforts, funded resources, consultation, or research projects; barriers noted were lack of expertise, training, and tools/methods [Reference Dolor, Proctor, Stevens, Boone, Meissner and Baldwin7]. At the time of the Workshop, no Texas CTSAs housed IR cores/centers, although pockets of expertise existed.

Target Audience

The Workshop was designed for 30 faculty scientists and clinical partners from Texas CTSA hubs. To assure broad representation, faculty scientists from all four Texas CTSAs were invited. Participants registered in advance and completed pre- and post-Workshop activities. Interest was high and Workshop capacity was expanded. Sixty-three participants represented the four Texas CTSAs, multiple disciplines, institutional and military clinical partners, and research support administrators. Ninety-eight percent of participants held a doctoral degree.

Description of the Educational Method and Curricular Program

Because terminology in IR is not yet distinct, the planning group developed the following amalgamated definitions to promote common understanding for the initiative:

  • Implementation science (IS) is a specialized field that addresses uptake of evidence-based practices into everyday care to improve health and health care. This science is built through IR studies.

  • IR builds IS through the scientific investigation of methods and strategies that promote systematic application of research findings in routine clinical practice.

The Community Engagement core at the Institute for Integration of Medicine & Science at University of Texas (UT) Health San Antonio led the design, resourcing, and conduct of an educationally sound 2-day Workshop with its institutional and statewide CTSA partners. Perspectives from multiple disciplines were included in the planning committee.

Workshop planners combined the newly released National Institutes of Health/National Cancer Institute (NIH/NCI) OpenAccess IR training materials [2] with in-person internal and external IR experts to design interactive interprofessional learning activities for maximum impact. Using the NIH/NCI online program as the foundation ensured quality, currency, and generalizability of the Workshop. Funds were used to waive registration fees, support guest speakers travel and honoraria, print Workshop materials, and provide refreshments and lunch for working sessions.

The program was based on an established national curriculum [Reference Tabak, Padek and Kerner8], national consensus on dissemination and implementation (D&I) research competencies [Reference Padek, Colditz and Dobbins9], and recently published NIH/NCI OpenAccess training materials [2], offering a standardized frame. This approach provided two advantages: generalizability beyond the single training opportunity, and alignment with established IR standards to advance the field. The Workshop targeted basic-to-intermediate level competencies with these goals: 1) build interprofessional scientific capacity among teams of faculty and clinical partners to design and conduct relevant, rigorous, high priority studies; 2) enhance success in competing for NIH, Agency for Healthcare Research and Quality, Department of Defense, and Patient-Centered Outcomes Research Institute IR grants; and 3) position UT System research partners as IR leaders for healthcare improvement.

To engage faculty scientists, we employed dissemination strategies [Reference Glasgow, Vinson, Chambers, Khoury, Kaplan and Hunter10] and marketing approaches. A logo “branded” the Workshop made materials quickly identifiable and created a shared mental model of Workshop purpose, with “adoption” as the core idea. Initial logo designs were vetted by the planning group. The final logo (Fig. 1) contained built-in meaning, depicting “adoption.” [Reference Rogers11]

Fig. 1. Workshop logo.

Measurable learning objectives (Table 1) aligned with the NIH/NCI curriculum and guided selection of content, topics, and speakers (see details at https://iims.uthscsa.edu/sites/iims/files/Comm_Engagement/Workshop%20Report%20FINAL.pdf).

Table 1. Learning objectives guiding the training Workshop curriculum

We applied proven instructional design principles [Reference Moore, Chappell, Sherman and Vinayaga-Pavan12,Reference Hew and Lo13] to achieve the learning objectives. Activities included presentations/Q&A, guided individual and group work, panel discussions, and online resources to accommodate adult learner preferences. Learning strategies included pre-Workshop assignments and in-person interactive group activities. The availability of previously developed online IR training resources enabled participants to prepare for the 2-day experience. Pre-Workshop activities (Table 2) and resources were emailed to participants, and they were encouraged to complete these activities as a condition for registration fee waiver.

Table 2. Pre-class assignments: “5 things to do before you come”

* As of December 2019. See NIH updates June 10, 2021.

In tandem with pre-Workshop assignments, in-person activities were employed. Because team-based learning (TBL) (an active, learner-centered but instructor-led strategy) is shown to enhance learning [Reference Reimschisel, Herring, Huang and Minor15,Reference Hrynchak and Batty16], we scheduled group activities throughout the Workshop. Groups applied a step-by-step research design guide [Reference Hull, Goulding and Khadjesari14] to co-develop study ideas. TBL strategies were used to expand and reinforce learning, promote interprofessional collaboration, and move toward the development of IR project ideas.

Participants received a comprehensive four-color hardcopy workbook which included the syllabus, schedule, speaker biographies, worksheets, IR resources, participant list, and evaluation link. An electronic file (PDF) of the workbook was provided to registrants before the Workshop. Additionally, speakers shared presentation slides, helpful supplemental resources, and readings for their respective session. These were saved in a shared Google Drive folder for availability during and after the Workshop.

Interspersed summaries and participant-panel discussions stimulated ideas for collaboration, culminating in recommendations for next steps. To conclude the Workshop and focus on long-range capacity building, one representative from each Texas CTSA hub moderated a panel discussion entitled Building Scientific Workforce Capacity and Opportunities for Collaboration. This brainstorming session provided opportunity for participants to generate ideas and recommendations for preferred strategies to continue expansion of IR across the CTSAs.

Methods of Evaluation

To assess near-term impact and long-range sustainment of IR capacity building among Texas CTSAs, a comprehensive evaluation plan was designed. The Kirkpatrick model [Reference Kirkpatrick and Kirkpatrick17], widely used in evaluating training programs, provided structure. Table 3 identifies the four levels associated with Workshop outcomes and types of data gathered.

Table 3. Evaluation plan

Results

Institutional Support

The formal value proposition stimulated leadership buy-in for faculty involvement in this new research focus and garnered sufficient funds to support the costs of the Workshop. The four-hub planning approach raised interest and catalyzed institutional support of faculty-scientist participation and travel from each hub.

Impact of Training

Results from Workshop quantitative and qualitative data were classified into each of the four Kirkpatrick levels and analyzed.

Level 1 Workshop Experience

Twenty-seven of the 63 participants (42%) completed both the pre- and post-Workshop self-assessment, which included evaluating whether the 11 learning objectives were met (Table 1). The highest rated objective was #2 Evolution of IR (96% agreement); the lowest was #11 Prospectus Plan (59% agreement). Event location was rated excellent (63%) or good (33%). Participants reported networking opportunities as excellent (52%), good (33%), and fair (7%). Survey comments indicated that the Workshop experience was viewed very favorably. Comments included, “Wonderful workshop, thank you for inviting exceptional speakers and planning content that could help us immediately!” Also, “This workshop has made me think about how we build teams and projects. In science we are oriented to strive for the PI role but serving as a co-investigator is just as important.”

Level 2 Knowledge Acquisition

In the pre-post self-assessment of IR competencies, participants reported improvement across all domains of IR competencies (IR definitions, background/rationale, theory/approaches, design/analysis, and practice-based considerations). After the training, fewer participants (48%) considered themselves “novice” than prior to the training (80%). Also, the proportion of “Intermediate/advanced” participants increased from 25% to 52%. Familiarity with the IR theories increased from 56% to 73%. Participant response to open-ended questions indicated that the Workshop had a positive impact on knowledge and attitudes. One example was, “This program and workshop have provided me huge takeaways.”

Level 3 Behavior

In the group activities, participants connected with colleagues with similar interests. Groups used the ImpRES Tool [Reference Hull, Goulding and Khadjesari14] to develop ideas for an IR study. A participant stated, “This workshop has made me rethink how to use mixed methods in Implementation Science.” Another stated, “This workshop has taught me that I should also map to a theory and model and consider the inner setting.” A third commented, “Going forward, how can we make people aware of each other to allow for the development of natural collaborations?”

Level 4 Results

Participants indicated interest in ongoing network interaction, resource availability, and capacity building including consultation. One person commented, “Overall, I consider this to be a successful workshop because we have laid groundwork for collaborations that could lead to possible grant opportunities.” Another participant offered, “A network of clinical sites in Texas could serve to build scientific capacity. Since IR requires multiple sites, the suggestion is to develop formal collaboration across the Texas CTSAs.”

Participants endorsed first steps toward establishing the statewide Texas Implementation Science Research Network (TEX-IS). Given that the Texas CTSAs routinely gather quarterly in regional meetings, this venue was suggested for continuing interaction. Unfortunately, meetings were suspended due to the pandemic. However, ongoing communication has been achieved through the monthly TEX-IS Research Network New Notes e-newsletter.

Progress subsequent to the Workshop

Progress 12 months following the Workshop includes:

  • Posting Workshop learning resources on the host institution’s website https://iims.uthscsa.edu/community/activities.html

  • Launching a D&I Consultation Core to support investigators and healthcare professionals in integrating D&I science in their research and practice. (https://reach.uthscsa.edu/services/di/). As a direct result of the show of interest in IR, new Implementation Research Consultation Core was established at one hub.

  • Offering a series of UT Health Houston IR webinars (https://sph.uth.edu/research/centers/chppr/workshops/tiis/conference).

  • Publishing monthly newsletters of emerging trends in IR, including new resources on equity in IS (e.g., 2). To date, the TEX-IS Research Network News Notes has published eight monthly issues.

  • Creating the TEX-IS Research Network Listserv and registration site.

Discussion

Workshop program design and curriculum were key deliverables of this project. A key element for success was using the NIH/NCI curriculum: Planners capitalized on the newly released NIH/NCI OpenAccess training resources, providing opportunity for ongoing and expanded training to additional faculty scientists across the four UT System CTSAs. Similarly, other notable online resources [Reference Ford, Rabin, Morrato and Glasgow1] that align with the national training curriculum can also be leveraged.

While this test case focused on IR, the educational principles we used are applicable to a wide range of capacity building needs in translational science. Developers of such programs can consider interdisciplinary planning, articulated program goals, measurable learning objectives, active instructional strategies, extended resources, ongoing interaction, and formal evaluation approach. The 2-day curriculum and learning activities provided a comprehensive foundation for extending IR training initiatives throughout our region.

Limitations arise from several sources. First, this training program was offered throughout the four CTSAs hubs in Texas, each with geographical differences. CTSA hub similarities likely arose from the overarching UT System (state-wide) context. Also, regional differences may result in different responses from single hub in faculty participants and populations impacted. Future work could include demographic composition of attendees to throw light on equity. Although measures are in place to promote multi-site IR studies, the reality of the intervening pandemic changed the intended dynamics of collaboration. Finally, although the program is educationally sound, it is a single test case of a novel approach to bootstrapping resources to boost IR workforce capacity.

The program incorporated a standardized foundation for IR training to advance our capacity in IR effectively and efficiently. Our model combines internal strengths with external resources (both in-person experts and online learning resources) and provides a comprehensive plan for building and evaluating workforce capacity, beginning with the 2-day Workshop.

The project demonstrated that, even with limited onsite expertise, it was possible to bootstrap resources and build IR capacity de novo in the CTSA community. Our approach could be useful to other institutions to expand IR capabilities in the face of limited local expertise.

Acknowledgements

We wish to thank the University of Texas Health Science Center institutional and departmental sponsors for their support of this project: Office of the Vice President for Research, Institute for Integration of Medicine & Science, School of Nursing, School of Health Professions, Center for Research to Advance Community Health, UT Population Health Sciences, Institute for Health Promotion, Mays Cancer Center – MD Anderson Cancer Center, Joe R. & Teresa Lozano Long School of Medicine, and School of Public Health – San Antonio Regional Campus.

The project described was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through Grant UL1 TR002645. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.

Disclosures

The authors have no conflicts of interest to declare.

References

Ford, BS, Rabin, B, Morrato, EH, Glasgow, RE. Online resources for dissemination and implementation science: meeting demand and lessons learned. Journal of Clinical and Translational Science 2018; 2(5): 259266. doi: 10.1017/cts.2018.337.CrossRefGoogle ScholarPubMed
Training Institute for Dissemination and Implementation Research in Cancer (TIDIRC) OpenAccess. National Institutes for Health, National Cancer Institute, Division of Cancer Control and Population Sciences [Internet]. Updated June 10, 2020 [cited July 5, 2021]. (https://cancercontrol.cancer.gov/is/training-education/TIDIRC-open-access)Google Scholar
Liverman, CT, Schultz, AM, Terry, SF, Leshner, AI, eds. The CTSA Program at NIH: Opportunities for Advancing Clinical and Translational Research. Washington, DC: National Academies Press, 2013. PMID: 24199260.Google Scholar
Leppin, AL, Mahoney, JE, Stevens, KR, et al. Situating dissemination and implementation sciences within and across the translational research spectrum. Journal of Clinical and Translational Science 2020; 4(3): 152158. doi: 10.1017/cts.2019.392.CrossRefGoogle ScholarPubMed
NCATS strategic plan. National Institutes for Health, National Center for Advancing Translational Science [Internet], Fall 2016. Updated February 18, 2020 [cited March 18, 2021]. (https://ncats.nih.gov/strategicplan)Google Scholar
Meissner, HI, Glasgow, RE, Vinson, CA, et al. The US training institute for dissemination and implementation research in health. Implementation Science 2013; 8(12): 19. doi: 10.1186/1748-5908-8-12.CrossRefGoogle Scholar
Dolor, RJ, Proctor, E, Stevens, KR, Boone, LR, Meissner, P, Baldwin, LM. Dissemination and implementation science activities across the Clinical Translational Science Award (CTSA) Consortium: report from a survey of CTSA leaders. Journal of Clinical and Translational Science 2020; 4(3): 188194. doi: 10.1017/cts.2019.422.CrossRefGoogle Scholar
Tabak, RG, Padek, MM, Kerner, JF, et al. Dissemination and implementation science training needs: insights from practitioners and researchers. American Journal of Preventive Medicine 2017; 52(3): s322s329. doi: 10.1016/j.amepre.2016.10.005.CrossRefGoogle ScholarPubMed
Padek, MM, Colditz, G, Dobbins, M, et al. Developing educational competencies for dissemination and implementation research training programs: an exploratory analysis using card sorts. Implementation Science 2015; 10(1): 114. doi: 10.1186/s13012-015-0304-3.CrossRefGoogle ScholarPubMed
Glasgow, RE, Vinson, C, Chambers, D, Khoury, MJ, Kaplan, RM, Hunter, C. National Institutes of Health approaches to dissemination and implementation science: current and future directions. American Journal of Public Health 2012; 102(7): 12741281. doi: 10.2105/AJPH.2012.300755.CrossRefGoogle ScholarPubMed
Rogers, EM. Diffusion of Innovations. 4th ed. New York: Free Press, 2010.Google Scholar
Moore, DE Jr, Chappell, K, Sherman, L, Vinayaga-Pavan, M. A conceptual framework for planning and assessing learning in continuing education activities designed for clinicians in one profession and/or clinical teams. Medical Teacher 2018; 40(9): 904913. doi: 10.1080/0142159X.2018.1483578.CrossRefGoogle ScholarPubMed
Hew, KF, Lo, CK. Flipped classroom improves student learning in health professions education: a meta-analysis. BMC Medical Education 2018; 18(1): 12. doi: 10.1186/s12909-018-1144-z.CrossRefGoogle ScholarPubMed
Hull, L, Goulding, L, Khadjesari, Z, et al. Designing high-quality implementation research: development, application, feasibility and preliminary evaluation of the implementation science research development (ImpRes) tool and guide. Implementation Science 2019; 14(1): 120. doi: 10.1186/s13012-019-0897-z.CrossRefGoogle ScholarPubMed
Reimschisel, T, Herring, AL, Huang, J, Minor, TJ. A systematic review of the published literature on team-based learning in health professions education. Medical Teacher 2017; 39(12): 12271237. doi: 10.1080/0142159X.2017.1340636.CrossRefGoogle ScholarPubMed
Hrynchak, P, Batty, H. The educational theory basis of team-based learning. Medical Teacher 2012; 34(10): 796801. doi: 10.3109/0142159X.2012.687120.CrossRefGoogle ScholarPubMed
Kirkpatrick, D, Kirkpatrick, J. Evaluating Training Programs: The Four Levels. San Francisco, CA: Berrett-Koehler Publishers, 2006.Google Scholar
Figure 0

Fig. 1. Workshop logo.

Figure 1

Table 1. Learning objectives guiding the training Workshop curriculum

Figure 2

Table 2. Pre-class assignments: “5 things to do before you come”

Figure 3

Table 3. Evaluation plan