Rationale for Novel Application of Curricular Approach

Engaging nontraditional partners (i.e., patients, clinicians, policy-makers, organizational leaders, Community Health Workers [CHWs]) throughout the research process can further research priorities, enhance methodology, and accelerate translation to inform policy and practice and reduce disparities [Reference Mugo, Njuguna and Nduati1–Reference Forsythe, Frank, Workman, Hilliard, Harwell and Fayish3]. This essential spoke of the translational research wheel strengthens research activities through diverse stakeholder expertise and lived experience. A key goal of the Community-Engaged Research Core (CeRC) within the Penn State Clinical and Translational Science Institute is to broaden connections within the community and serve as a critical linkage for researchers and stakeholders [4]. Since stakeholders come from various educational and professional backgrounds, equipping them with tools and resources to support their roles as research partners is critical to successful engagement. Innovative educational approaches to support meaningful involvement of stakeholders in Clinical and Translational Science (CTS) allows for bidirectional learning and greater engagement in translational efforts. With the adult learner in mind, trainings and resources can be tailored to meet stakeholders where they are, providing relevancy and immediacy of knowledge in brief, interactive segments [Reference Bryan, Kreuter and Brownson5–Reference Knowles, Holton, Swanson and Robinson7].

Unmet Need or Educational Gap

Growing evidence suggests a positive resultant impact of stakeholder engagement in research [Reference Mugo, Njuguna and Nduati1–Reference Forsythe, Frank, Workman, Hilliard, Harwell and Fayish3,Reference Barger, Sullivan and Bell-Brown8,Reference Concannon, Fuster and Saunders9]; however, there are limited readily available resources on how to effectively educate stakeholders to promote successful engagement in Patient-Centered Outcomes Research (PCOR) [10]. Community Health Workers (CHWs), for example, represent an untapped workforce well situated to bridge the community with various health services. The American Public Health Association defines a CHW as a “frontline public health worker who is a trusted member of, and/or has an unusually close understanding of, the community served” [11]. Community Health Workers have a proven track record of improving health outcomes, particularly for underserved populations, in a variety of community-based interventions [Reference Andrews, Felton, Wewers and Heath12,Reference Viswanathan, Kraschnewski and Nishikawa13]. Because of their intimate connections with the community, CHWs hold the potential to serve as liaisons between investigators and target populations in PCOR. Further, CHWs can engage hard-to-reach individuals within vulnerable communities, understand the context in which health problems exist, and serve as advocates for their communities’ needs in research [Reference Hill, Bone and Butz14]. Unfortunately, there remain several barriers preventing CHWs from engaging in PCOR, including: (1) lack of PCOR training in CHW certification programs, (2) lack of systems to connect CHWs with investigators interested in community-engaged research, and (3) recently, challenges associated with CHWs shifting to virtual work due to the COVID-19 pandemic [Reference Smith and Wennerstrom15].

The second gap is the utilization of stakeholders in effective research dissemination. Stakeholders representing community organizations, practice, or policy are well positioned to provide insight into research activities and can serve as conduits for results dissemination through their broad networks [Reference Poger, Yeh and Bryce16,Reference Poger, Mayer and Duru17]. Additionally, patients and caregivers sit at the core of patient-centered outcomes and can transform research findings into relevant lay public resources, drawing from their intimate understanding of the day-to-day challenges of their condition [Reference Poger, Yeh and Bryce16]. Delivering a robust educational experience for partners new to PCOR can cast a wider dissemination net and facilitate increased uptake of study findings.

Target Audience

Diverse stakeholders from multiple funded projects participated in our stakeholder educational programming to advance both their knowledge of PCOR and research engagement through project-tailored curriculum (Table 1). To illustrate, we outline two separate project examples highlighting our approach to stakeholder education. In the Patient-Centered Outcomes Research Institute (PCORI)-funded research project, Addressing Health Disparities with CHWs in COVID-19 (AHeaD with CHWs in COVID-19), CHWs (and other appropriate job categories, i.e., patient navigator, social worker) representing health systems, social service organizations, academic institutions, and businesses across the state of Pennsylvania were offered an innovative PCOR and remote work training, embedded in a long-standing CHW training program and as continuing education to previously trained CHWs. Both (1) a diverse Stakeholder Advisory Board (SAB) (n = 17) comprised of CHWs, CHW instructors, CHW managers, directors of health systems and organizations that utilize CHWs, and community-engaged researchers; and (2) a CHW Advisory Council (n = 11) comprised of expert CHWs closely connected with communities disproportionately affected by health disparities assisted in recruitment, collaboratively planned CHW trainings, and assisted in identifying a system to connect CHWs with researchers.

Table 1. Stakeholder educational programming characteristics

^a Bryan RL, Kreuter MW, Brownson RC. Integrating adult learning principles into training for public health practice. Health Promot Pract 2009; 10(4): 557–563.

^b Yonas MA, Jaime MC, Barone J, et al. Community partnered research ethics training in practice: a collaborative approach to certification. J Empir Res Hum Res Ethics 2016; 11(2): 97–105.

In the second PCORI-funded project, PaTH to Health: Diabetes, patient partners, and other community stakeholders representing policy, practice, and professional organizations were engaged in educational trainings at key intervals throughout the 5-year project. Project stakeholders (n = 22) were recruited from prior participation on advisory boards, clinician referrals, or were strategically invited to participate, given their shared focus of improving diabetes outcomes.

Description of the Educational Method Utilized or Curricular Program

This unique educational portfolio developed by the core engagement team is classified into two categories: education about community engagement strategies to support engagement efforts; and specific training at key research phases within PCOR (Fig. 1). All educational modules were offered via the Zoom platform, which increased reach and was particularly valuable during the COVID-19 pandemic in overcoming participation barriers. Interactive breakouts and the use of features such as chat, annotation, whiteboard, and relevant vignettes/case studies were utilized to increase participation, bolster discussion, and strengthen knowledge transfer.

Fig. 1. Patient-Centered Outcomes Research (PCOR) educational framework.

Note: PCORI Engagement Rubric. PCORI (Patient-Centered Outcomes Research Institute) website. http://www.pcori.org/sites/default/files/Engagement-Rubric.pdf. Published February 4, 2014

Methods of Evaluation

The AHeaD with CHWs in COVID-19 project team adapted Kirkpatrick’s Four Levels of Training Evaluation (reaction, learning, impact, results) [Reference DeSilets20] to evaluate the CHW virtual community engagement training. Survey evaluations were administered at three time points (pre-training, immediately post-training, and 1 month post-training) to assess learner’s perception of training quality; skills, confidence, and knowledge regarding PCOR processes; and preparation to engage and apply PCOR skills remotely. The 1 month post-training evaluation included a 5-point Likert scale to assess how the training affected participants’ behaviors and long-term outcomes. Pre/post-knowledge of PCOR concepts and research topics offered in the PCOR training were also assessed.

In the PaTH to Health: Diabetes study, stakeholders had the opportunity to participate in yearly quantitative and qualitative evaluations assessing the breadth of engagement activities, including education offerings, from the prior project period. This data helped elucidate the effectiveness of the research team’s involvement of stakeholders and of training and other engagement strategies as enhancements to stakeholder contributions to the project.

All quantitative evaluation data were collected and managed using Research Electronic Data Capture (REDCap) hosted at the Penn State University College of Medicine.

Initial Evidence of Impact on Learning

Roughly half of the training attendees in the AHeaD with CHWs in COVID-19 project completed the pre-training evaluation, though this was not mandatory (n = 51), and 29 participants completed the immediate post-training evaluation. We found evidence of impact on learning through increased community engagement reported by training participants. When comparing pre- and immediate post-training survey results (n = 15) using paired sample t-tests, there were statistically significant increases in reported knowledge of and confidence in virtual community engagement strategies, COVID-19 resources, motivational interviewing methods, and myth-busting techniques (Table 2). Participants who completed the 1 month post-training evaluation (n = 13) reported that they used the motivational interviewing techniques and community engagement strategies in their work (mean 4.54 ± 0.88; mean 4.62 ± 0.51, respectively). They also reported increased confidence in utilizing myth-busting strategies (mean 4.85 ± 0.38), an important skill given the multitude of misinformation creating barriers to health during a pandemic. Further, CHWs improved in their knowledge of PCOR concepts from a low–moderate pre to a moderate–high post-knowledge level.

Table 2. AHeaD with CHWs ^a in COVID-19 pre- and immediate post-change in mean confidence and knowledge level

^a Community Health Workers.

^b P < 0.01.

The PaTH to Health: Diabetes study also found evidence of impact on learning through enhanced engagement in research activities. For example, after partners attended a data extraction training to better understand how EHR data is used for research purposes while maintaining patient confidentiality, partners co-developed an interactive whiteboard video and dissemination brief on this topic. These patient-facing resources were helpful in mitigating the stigma associated with research by describing how health information can be used safely to answer important research questions. Similarly, after attending results dissemination training on the myriad ways to communicate research results to both scientific and lay audiences, patient partners assisted in the development of lay public dissemination briefs of published study manuscripts. Partners were helpful in pulling out key manuscript highlights and conveying these sections in lay-appropriate language, with visuals to display study findings. These one-pagers effectively translate scientific findings to a general audience, underscoring the important role partners can play in disseminating results to the broader community (see Appendix). Additionally, several stakeholders participated in scientific communications, including presenting updates at network meetings and serving as co-authors on study publications.