Skip to main content Accessibility help
×
Home

Information:

  • Access
  • Open access

Actions:

      • Send article to Kindle

        To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

        Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

        Find out more about the Kindle Personal Document Service.

        Identifying strategies to promote team science in dissemination and implementation research
        Available formats
        ×

        Send article to Dropbox

        To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

        Identifying strategies to promote team science in dissemination and implementation research
        Available formats
        ×

        Send article to Google Drive

        To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

        Identifying strategies to promote team science in dissemination and implementation research
        Available formats
        ×
Export citation

Abstract

Introduction:

Scientific endeavors are increasingly carried out by teams of scientists. While there is growing literature on factors associated with effective science teams, little is known about processes that facilitate the success of dissemination and implementation (D&I) teams studying the uptake of healthcare innovations. This study aimed to identify strategies used by D&I scientists to promote team science.

Methods:

Using a nominal group technique, a sample of 27 D&I scholars responded to the question, “What strategies have you or others used to promote team science?” Participants were asked to individually respond and then discuss within a small group to determine the group’s top three strategies. Through a facilitated consensus discussion with the full sample, a rank-ordered list of three strategies was determined.

Results:

A total of 126 individual responses (M = 9; SD = 4.88) were submitted. Through small group discussion, six groups ranked their top three strategies to promote team science. The final ranked list of strategies determined by the full sample included: (1) developing and maintaining clear expectations, (2) promoting and modeling effective communication, and (3) establishing shared goals and a mission of the work to be accomplished.

Conclusions:

Because of its goal of translating knowledge to practice, D&I research necessitates the use of team science. The top strategies are in line with those found to be effective for teams in other fields and hold promise for improving D&I team cohesion and innovation, which may ultimately accelerate the translation of health innovations and the improvement of care quality and outcomes.

Introduction

Across the globe, a focus on teamwork and collaboration in the workplace is on the rise.1, 2 This growing trend is present in a variety of settings, including scientific research contexts.3 Team research efforts, referred to as team science, involve multiple scientists who carry out interdependent tasks to achieve a goal.4 Compared to single investigators, science teams not only produce more academic publications, but also achieve higher impact factors, are cited more frequently, and are more likely to discover breakthroughs.3, 5 While scientific collaborations are often unidisciplinary in nature (i.e., within the same field), scientists are increasingly working in cross-disciplinary teams that bridge two or more fields and/or patients, community stakeholders, and other relevant stakeholders. However, engaging researchers and collaborators in implementation science teams and fostering the development of these teams can be a slow process.4

Cross-disciplinary collaborations have been characterized as multidisciplinary, interdisciplinary, or transdisciplinary.6 In a multidisciplinary team, researchers from different disciplines work within their respective fields and then apply their findings to a common research goal. For example, an epidemiologist might conduct a study while an anthropologist might address the same issues from a qualitative inquiry perspective, with their findings integrated post hoc. Interdisciplinary research involves more integration, such that researchers from separate fields (e.g., anthropology, medicine, social work) interactively work together to address a common problem (e.g., access to high-quality healthcare). Transdisciplinary research, the most collaborative form, involves a synthesis of distinct fields to develop models that transcend a single discipline and result in a hybrid discipline. For example, neuroscience, bioengineering, and human–computer interaction are hybrid disciplines that evolved from transdisciplinary efforts.7 We use the term cross-disciplinary to refer to science teams that collaborate at any of these levels.

Because cross-disciplinary teams may foster greater potential for innovation and creativity,8, 9 they are especially important for tackling complex public health issues.1012 As such, recent decades have seen a substantial increase in the amount of funding provided by both private foundations and federal agencies for team-based research efforts.13 For example, National Institutes of Health (NIH) awards to projects with multiple principal investigators, first introduced in 2006, accounted for nearly 20% of all major grants funded in 2013.13, 14 To further facilitate team science, NIH has established multiple cross-disciplinary programs and research centers.15, 16 Team-based research efforts increasingly involve scientists from different organizations13; multi-university collaborations have progressively become more common, and there have been recent calls to extend the scope of science team membership beyond academia.1719 Because many cross-disciplinary research initiatives ultimately aim to apply research findings to real-world settings to combat widespread public health issues,12, 20, 21 collaborating with public stakeholders in team-based research efforts is a promising strategy to facilitate translation of these initiatives.22, 23

Moving from research to practice to improve health services and population health is an objective of the new science of dissemination and implementation (D&I).24 There is a well-documented lag between the development of scientific evidence (e.g., new medications, psychosocial interventions) and their broad use in practice. This has resulted in a lag of roughly 17 years in the translation pipeline and then only about 14% of original research goes to the benefit of patient care.25 Teams are critical for D&I science, which naturally facilitates engagement between scientists and stakeholders (e.g., researchers within other disciplines, policymakers, community members) to accelerate the translation of new and effective evidence-based practices.26, 27 Cross-disciplinary D&I science teams have the potential to expedite the translation process through having the most relevant and innovative perspectives come together to determine the most efficient and effective ways to move research to practice. While there are challenges associated with co-production in science teams (e.g., disagreements, professional costs),28 research has described the richness and complexity of collaborative processes while also identifying multiple benefits that outweigh the challenges.29 The requisite for teams to produce translatable outcomes, coupled with increased calls for the use of team science in D&I settings,3033 creates a need for evidence-based strategies for facilitating successful D&I collaborations and synergistic transdisciplinary teams.34

Rich findings from research on team functioning suggest strategies for creating effective cross-disciplinary science teams.35 This literature highlights strategies that enable successful teams, such as communication, the development of shared mental models (i.e., knowledge structures), trust, and shared leadership.3640 While team effectiveness findings have typically been applied across settings, contextual factors may limit their generalizability.11 Many findings come from studies of randomly composed teams in laboratory settings, where tasks and performance measures tend to be narrow. As such, literature on successful teams in real-world settings is on the rise, allowing strategies to be tailored to teams based on their setting and goals.35 For example, studies on cross-disciplinary healthcare teams have found that team processes associated with team success include quick learning, intent listening, speaking up, and psychological safety.41, 42

The increasing calls for team science, however, have outpaced research on how to promote effective science teams. There are a number of questions relevant to science teams that have not been addressed, such as the effects of research structures and funding mechanisms on team functioning.13 These questions and others are beginning to be addressed by the science of team science field, which aims to identify and understand strategies that are associated with successful science teams. Science of team science research has begun to uncover team processes that lead to effective team science, such as learning from other members and communicating face-to-face.34 Yet amidst calls for the greater use and study of team science in D&I contexts,3033, 43 no studies have investigated strategies that facilitate success in D&I science teams.44 The aims of D&I science teams, which may include implementing evidence-based practices with fidelity,45 sustaining these practices in community settings,22, 46 or engaging public stakeholders in the research process,47 introduce unique challenges not faced by cross-disciplinary teams that operate solely within academia.18, 44 The objective of this study was to identify and rank strategies to promote team science among scholars engaged in D&I research.

Methods

Procedures

The nominal group technique (NGT) was used in this study. The NGT is a commonly used group consensus method in health services research that involves a facilitated, multistep structured group process to generate and prioritize responses to a stimulus question among a panel who have expertise in a focal area,4851 without relying on an a priori established conceptual model that might inadvertently bias results. This approach was used for pragmatic reasons (e.g., minimal preparation, only one meeting needed) and for fit with the goal of the meeting (i.e., to identify strategies used by D&I researchers to promote team science). In addition, we wanted to capitalize on the unique opportunity to learn from a diverse group of multidisciplinary researchers engaged in D&I research. We applied the recommendations to ensure methodological rigor when using consensus group methods to report the steps and results from the NGT approach used in this study.52

We used a modified NGT process that proceeded in the following stages. First, the facilitator (GAA) presented the nominal question: What strategies have you or others used to promote team science? Participants were first asked to generate ideas and write down their responses to this question individually. They were also asked to rank order their top three ideas. Next, small groups (consisting of 3–6 individuals) were formed and participants were asked to take turns sharing all of their individual responses with the group, to compile strategies within the group, and to rank order the top three strategies agreed upon by their small group (round 1). Following the first two rounds, the facilitator requested that each small group, in turn, report out their ratings and these were written out on a flip chart for all to view and consider (round 2). The facilitator then led all participants in a full group consensus ranking process to determine the top three strategies (round 3). After each round, the facilitator summarized responses and elicited participant confirmation of the summaries. Collective group consensus was defined as agreement indicated by verbal and/or nonverbal approval.

Setting

This study was conducted during the weeklong in-person training component of the National Institute of Mental Health (NIMH)/Veterans Affairs Health System (VA)/National Institute on Drug Abuse (NIDA)-sponsored Implementation Research Institute (IRI) on June 11, 2018 at the Knight Executive Education and Conference Center and was hosted by the George Warren School of Social Work and the Institute for Public Health at Washington University in St. Louis, Missouri. Data were collected as part of an hour-long interactive activity focused on team science and working on cross-disciplinary teams within D&I research. This study was approved by the Institutional Review Board of the University of California, San Diego as exempt research.

Participants

Twenty-one IRI fellows and six core faculty (n = 27) attended the presentation. The IRI is a 2-year interdisciplinary training program that provides in-depth training in D&I science through an annual learning collaborative, individualized mentoring, and a mentoring network of faculty and fellows to advance the research and careers of implementation science investigators. Participants were predominantly female (67%, n = 18). In terms of professional characteristics, the majority of participants had doctoral (i.e., PhD or equivalent) research degrees (82%, n = 22) and the remaining participants had medical degrees (19%; n = 5). There was a range of disciplines represented across the participants: Psychology (37%, n = 10), Social Work (7%, n = 2), Medicine (7%, n = 2), Psychiatry (7%, n = 2), Pediatrics (7%, n = 2), Public Health (7%, n = 2), Engineering (7%, n = 2), Anthropology (4%, n = 1), Sociology (4%, n = 1), Organizational Behavior and Management (4%, n = 1), Education (4%, n = 1), and Health and Public Policy (4%, n = 1). Participants were conducting implementation research in both domestic and international (i.e., low- and middle-income countries) settings and in a variety of healthcare contexts (e.g., primary care, mHealth, public systems, organizational level, schools, community mental health, prisons, Federally Qualified Health Centers). See Table 1 for participant characteristics.

Table 1. Sample characteristics

Note: n = 27

The facilitator (GAA) requested to collect individual and group written responses from those attendees who were willing to share their data. The NGT process typically includes 5–12 members, so our sample size of 27 exceeds this standard.52 Anonymity of participants was maintained by ensuring that no personally identifying information could be linked to a response, and no additional qualitative data were collected.

Data Analysis

Descriptive statistics were used to summarize the individual data and aggregated rankings. Data analysis and reduction was conducted as part of the NGT process so that data aggregation and rankings were complete at the end of the participatory group exercise.

Results

During the NGT exercise, in round 1, individual participants were asked to report, from their own written list, their top three strategies. In round 2, each of six small groups considered all of the individual written responses of all group members and reported out to the full group the top three strategies agreed upon within their small group. In round 3, group consensus was achieved from all small groups. After the third round, 14 of the 27 participants (response rate = 52%) submitted their individual written responses from round 1 to the research team for a total of 126 responses. The average number of individual responses was 9.00 (SD = 4.88; range = 4–18).

The three strategies from each of the six groups identified through the consensus process, along with their definitions, are reported in Table 2. Through full group discussion, the top strategies endorsed by each small group were consolidated into four primary categories. The categories were: (A) a focus on clear expectations, responsibilities, and well-defined roles of team members, (B) promoting effective communication, which includes respectful and assertive communication styles between team members and leadership, (C) fostering a culture of mutual respect and trust, and (D) developing a shared mission and/or set of goals. All six small groups submitted at least one strategy related to Category A, four of the small groups submitted at least one strategy related to Category B, four of the small groups submitted at least one strategy related to Category C, and three of the small groups submitted a strategy related to Category D. As shown in Table 3, the full group’s consensus process (i.e., all participants) identified the top three strategies for promoting team science in implementation and their respective ranks. Through the facilitated group consensus discussion (round 3), the final rank-ordered list of strategies for promoting team science were: (1) developing and maintaining clear expectations, (2) promoting and modeling effective communication, and (3) establishing shared goals and a mission of the work to be accomplished. Individual participant responses are provided in Table 4.

Table 2. Top three strategies identified by small groups and rank ordered within group

Table 3. Top three strategies identified in full group consensus ranking process

Table 4. Strategies identified by individuals

Discussion

We examined factors recommended to facilitate team science among a cohort of D&I researchers, including experienced faculty and early- to mid-career fellows of the NIMH/VA/NIDA-sponsored IRI. The top three strategies for fostering team science in D&I (by rank order) endorsed by the group were (1) setting clear expectations and roles in conducting research, (2) promoting and modeling effective communication, and (3) establishing shared goals and mission. These findings are broadly consistent with promoting team science in other fields,34, 35, 37 and with findings and conceptual models regarding team building more broadly.53, 54 There are several possible reasons that these aspects of team functioning were seen as particularly relevant by our respondents. First, D&I science typically brings together disparate fields such as clinical services, implementation, organization, policy, and economics – in fact, many implementation science questions are impossible to answer without involving these diverse viewpoints. Each of these fields may bring different expectations regarding what research goals are most important and how best to communicate within the team. Thus, there is a need to reformulate research training to foster a more interprofessional perspective in favor of focusing too much on one’s own discipline.55 Such an approach could lead to more professional empathy regarding other perspectives in team science. Second, because D&I science is a relatively new field, there will likely be variability among team members regarding the breadth and depth of familiarity with D&I concepts, frameworks, research designs, methods, and measures. This also suggests that clear communication may be especially important for D&I science teams. While the NGT resulted in the top three recommended strategies, it is important to acknowledge that there were many other recommendations from one or more of the six small groups that should also be considered. For example, relationships, psychological safety, and trust are important in research and also important in collaborative and mentoring relationships. Developing consensus while modeling respect and developing a sense of psychological safety can foster cross-disciplinary team science where there is a need for respect of differing perspectives and values. We also recommend that D&I research aspires to the level of “transdisciplinary” research to support effective collaboration and to promote synergy and innovation.

This recommendation naturally begs the question: What strategies might be used to develop transdisciplinary teams that will be innovative, effective, and productive in the field of D&I science? The answers are beyond the purview of the present study; however, we suggest that effective leadership and development of collaboration and mentoring networks can help to achieve these goals. These were identified in the NGT process and warrant further consideration. For example, the Leadership and Organizational Change for Implementation intervention,56 while designed to create a climate for implementation of evidence-based practices, could be adapted to create a climate for team science in academic settings and in community academic partnerships.57

There are many frameworks and theories that help to organize and describe implementation determinants and processes.58, 59 The Exploration, Preparation, Implementation, Sustainment (EPIS) framework describes outer (e.g., health system) and inner (i.e., organization) factors that might impact implementation as well as the four phases of the implementation process.60, 61 It is useful to consider how multidisciplinary teams of different configurations might be useful at different phases of the implementation process and also for understanding outer and inner context factors. For example, including an organizational psychology perspective can be useful in understanding operations in hospitals, clinics, or teams. Much of this work can also be considered to occur in the Exploration and Preparation phases of the EPIS framework. For example, the EPIS processes of collaboration or co-production as “bridging factors” that span outer and inner contexts can be considered to be active in bringing together diverse stakeholders including researchers and community or health system partners.61 Such partnerships and collaborations can help to make implementation science more immediately relevant both theoretically and practically.

The call for team science is becoming the norm for research funders and academic institutions. For D&I research, team science and the need for collaboration to share and integrate perspectives, ideas, and ideals are essential and may be even more critical than in other fields. This is a function of the broad purview of concepts and approaches and crossing-cutting issues and contexts in D&I. These include different disease and health promotion areas, patient and service populations, service systems and settings that span geographical contexts (e.g., different continents, countries, municipalities), and technology platforms (e.g., eHealth, mHealth) where implementation occurs. Because of the inherently interdisciplinary and collaborative nature of the field of D&I science, it is imperative that we understand how to promote appropriate team science. This includes building teams with needed expertise and skills for professional empathy to both appreciate and apply the perspectives, theories, and methods of others working together in pursuit of solving D&I issues and advancing the field.

Even the process of funding invokes the need for team science. For example, in the USA, there are formal calls for D&I research that involves over 17 of the institutes within the NIH. In the Dissemination and Implementation Research in Health (DIRH) review committee, tasked with the scientific evaluation of D&I research, proposal members may have specific expertise in a given disease or public health category (e.g., cancer, addiction, mental health). However, the common language, concepts, and approaches of proposals being evaluated is that of D&I. The DIRH review panel itself could be considered an interdisciplinary evaluation team. While this is a promising approach to scientific review of these types of applications, there are still different and varied perspectives depending on disease and contextual issues (e.g., implementation in low- and middle-income countries) that require more effective team science in the conduct of science and, as in this committee, the evaluation of the science.

The findings presented in this study raise questions about the ease and difficulties of teams in implementation science. For example, the top-ranked strategy was related to clear expectations, responsibilities, and well-defined roles of team members. Implementing this strategy may present challenges where there are strong opinions or preference for leadership roles versus team member roles. Promoting effective communication can be dependent on the ability of leaders to engender respectful communication among team members, and the development of mutual respect and trust to create a shared mission and/or set of goals. In addition to the current study results, more detail about potential barriers to team science can be garnered from “lessons learned” from a team science project addressing multiple chronic health conditions.62 Reflecting on that process, the authors identified the need for funding for effective infrastructures to foster team science, taking a solutions-focused approach that supports in-person networking, matching collaborators, and having dedicated support for coordination of activities. Our findings and those from other studies are helping to build the knowledge base and promote testable theories for promoting, enhancing, and sustaining team science.

Limitations

It is important to note some limitations. First, the study was conducted with one group of implementation researchers whose work largely focused on behavioral health or substance use issues. However, this concern is mitigated by the fact that the D&I research of the group is fundamentally multidisciplinary in nature and spans diverse issues and settings. For example, issues addressed by faculty and fellows include mental health in emergency departments, behavioral issues related to gastrointestinal disease, testing multilevel implementation strategies in a range of service settings, implementation of evidence-based interventions in low- and middle-income countries, applying system dynamics and engineering approaches to D&I and use of health technologies. Related, while the participants represented a diverse array of disciplines, we acknowledge that not all disciplines (e.g., econometrics) were represented. Second, we did not use ancillary or other methods of data collection such as surveys. Thus, the results may not have identified all possible issues that would address team science in D&I research. However, the methods utilized allowed for participants to raise issues beyond what might have been included in a survey. Finally, we did not capture the degree to which responses were related to participants’ own use of strategies to promote team science versus strategies they had observed others using. However, members of team research initiatives can be and often are involved in creating, employing, and participating in team science promotion strategies, allowing for scoping observations from a wide variety of perspectives.

Conclusions

Team science is being called for and increasingly expected by funders and academic institutions to facilitate collaboration, innovation, and productivity in regard to translating health innovations to real-world settings. There is a strong argument for using team science in D&I research because of its inherently broad and complex nature. Thus, it is important to understand and develop approaches for fostering and improving team science in D&I research. Our study results indicate that key strategies for fostering team science in D&I include (1) setting clear expectations and roles in conducting research, (2) promoting and modeling effective communication, and (3) establishing shared goals and mission. Pursuing these strategies will likely improve team cohesion and innovation, and result in approaches to accelerate the adoption, implementation, and sustainment of health innovations across contexts and with the promise of improving quality of care and outcomes.

Acknowledgments

Dr. Aarons is core faculty and Drs. Stadnick and Miller are fellows with the Implementation Research Institute (IRI), at the George Warren Brown School of Social Work, Washington University in St. Louis. The IRI is supported through an award from the National Institute of Mental Health (R25MH08091607).

This project was supported in part by the US National Institute of Mental Health grants R25MH080916 (E. Proctor), K23MH110602 (Stadnick), R03MH117493 and R01MH072961 (Aarons), and National Institute on Drug Abuse R01DA038466 (Aarons). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Disclosures

The authors declare that they have no competing interests.

Ethics approval and Consent to Participate

Ethics approval was evaluated and determined to be exempt by the UC San Diego Human Research Protections Program.

References

1.Deloitte. Global Human Capital Trends 2016—The New Organization: Different by Design. Deloitte University Press; 2016 (retrieved from https://www2.deloitte.com/content/dam/Deloitte/global/Documents/HumanCapital/gx-dup-global-human-capital-trends-2016.pdf).
2.Salas, E, Stagl, KC, Burke, CS. 25 years of team effectiveness in organizations: research themes and emerging needs. In: Cooper, CL, Robertson, IT, eds. International Review of Industrial and Organizational Psychology. West Sussex, UK: John Wiley & Sons; 2004. 4792.
3.Wuchty, S, Jones, BF, Uzzi, B. The increasing dominance of teams in production of knowledge. Science 2007; 316(5827): 10361039.
4.Stokols, D, et al. The science of team science: overview of the field and introduction to the supplement. American Journal of Preventive Medicine 2008; 35(2): S77S89.
5.Singh, J, Fleming, L. Lone inventors as sources of breakthroughs: Myth or reality? Management Science 2010; 56(1): 4156.
6.Rosenfield, PL. The potential of transdisciplinary research for sustaining and extending linkages between the health and social sciences. Social Science & Medicine 1992; 35(11): 13431357.
7.Muller, MJ. Participatory design: the third space in HCI. Human-Computer Interaction: Development Process 2003; 4235: 165185.
8.Lungeanu, A, Contractor, NS. The effects of diversity and network ties on innovations: the emergence of a new scientific field. American Behavioral Scientist 2015; 59(5): 548564.
9.Kurtzberg, TR. Feeling creative, being creative: an empirical study of diversity and creativity in teams. Creativity Research Journal 2005; 17(1): 5165.
10.Disis, ML, Slattery, JT. The road we must take: multidisciplinary team science. Science Translational Medicine 2010; 2(22): 22cm29.
11.Stokols, D, et al. The ecology of team science: understanding contextual influences on transdisciplinary collaboration. American Journal of Preventive Medicine 2008; 35(2 Suppl): S96S115. doi: 10.1016/j.amepre.2008.05.003.
12.Hall, KL, et al. News from NIH: resources for team-based research to more effectively address complex public health problems. Translational Behavioral Medicine 2012; 2(4): 373375.
13.Cooke, NJ, et al. Enhancing the Effectiveness of Team Science. Washington, DC: National Academies Press, 2015.
14.Establishment of Multiple Principal Investigator Awards for the Support of Team Science Projects. Secondary Establishment of Multiple Principal Investigator Awards for the Support of Team Science Projects 2006. https://grants.nih.gov/grants/guide/notice-files/not-od-07-017.html.
15.Luke, DA, et al. Breaking down silos: mapping growth of cross‐disciplinary collaboration in a translational science initiative. Clinical and Translational Science 2015; 8(2): 143149.
16.Guise, JM, et al. Organizational and training factors that promote team science: a qualitative analysis and application of theory to the National Institutes of Health’s BIRCWH career development program. Journal of Clinical and Translational Science 2017; 1(2): 101107.
17.Jones, BF, Wuchty, S, Uzzi, B. Multi-university research teams: shifting impact, geography, and stratification in science. Science 2008; 322(5905): 12591262.
18.Tebes, JK, Thai, ND. Interdisciplinary team science and the public: steps toward a participatory team science. American Psychologist 2018; 73(4): 549.
19.Selker, HP, Wilkins, CH. From community engagement, to community-engaged research, to broadly engaged team science. Journal of Clinical and Translational Science 2017; 1(1): 56.
20.Warnecke, RB, et al. Approaching health disparities from a population perspective: The National Institutes of Health Centers for Population Health and Health Disparities. American Journal of Public Health 2008; 98(9): 16081615.
21.Croyle, RT. The National Cancer Institute’s transdisciplinary centers initiatives and the need for building a science of team science. American Journal of Preventive Medicine 2008; 35(2): S90S93.
22.Altman, DG. Sustaining interventions in community systems: on the relationship between researchers and communities. Health Psychology 1995; 14(6): 526.
23.Hall, KL, et al. Understanding cross-disciplinary team-based research. In: Frodeman, R, Thompson, JK, Pacheco, RCS, eds. The Oxford Handbook of Interdisciplinarity. Oxford, UK: Oxford University Press; 2017. 338356.
24.Brownson, RC. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford University Press, 2017.
25.Balas, EA, Boren, SA. Managing clinical knowledge for healthcare improvements. In: Bemmel, J, McCray, AT, eds. Yearbook of Medical Informatics 2000: Patient-Centered Systems. Stuttgart, Germany: Schattauer Verlagsgesellschaft; 2000. 6570.
26.Yarborough, M, et al. Relationships hold the key to trustworthy and productive translational science: recommendations for expanding community engagement in biomedical research. Clinical and Translational Science 2013; 6(4): 310.
27.Proctor, EK, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Administration and Policy in Mental Health and Mental Health Services Research 2009; 36(1): 2434.
28.Oliver, K, Kothari, A, Mays, N. The dark side of coproduction: do the costs outweigh the benefits for health research? Health Research Policy and Systems 2019; 17(1): 33.
29.Aarons, GA, et al. Collaboration, negotiation, and coalescence for interagency-collaborative teams to scale-up evidence-based practice. Journal of Clinical Child and Adolescent Psychology 2014; 43(6): 915928.
30.Lewis, CL, et al. Proceedings of the 3rd Biennial Conference of the Society for Implementation Research Collaboration (SIRC) 2015: Advancing efficient methodologies through community partnerships and team science. Implementation Science; 2016; 11: 85, BioMed Central.
31.Puga, F, Stevens, KR, Patel, DI. Adopting best practices from team science in a healthcare improvement research network: the impact on dissemination and implementation. Nursing Research and Practice 2013; 2013.
32.Glasgow, RE, et al. National Institutes of Health approaches to dissemination and implementation science: current and future directions. American Journal of Public Health 2012; 102(7): 12741281. doi: 10.2105/AJPH.2012.300755.
33.Tinkle, M, et al. Dissemination and implementation research funded by the US National Institutes of Health, 2005–2012. Nursing Research and Practice 2013; 2013.
34.Hall, KL, et al. The science of team science: a review of the empirical evidence and research gaps on collaboration in science. American Psychologist 2018; 73(4): 532.
35.Salas, E, Reyes, DL, McDaniel, SH. The science of teamwork: progress, reflections, and the road ahead. American Psychologist 2018; 73(4): 593.
36.Marks, MA, Mathieu, JE, Zaccaro, SJ. A temporally based framework and taxonomy of team processes. Academy of Management Review 2001; 26(3): 356376.
37.Marlow, SL, et al. Does team communication represent a one-size-fits-all approach? A meta-analysis of team communication and performance. Organizational Behavior and Human Decision Processes 2018; 144: 145170.
38.DeChurch, LA, Mesmer-Magnus, JR. Measuring shared team mental models: a meta-analysis. Group Dynamics: Theory, Research, and Practice 2010; 14(1): 1.
39.De Jong, BA, Dirks, KT, Gillespie, N. Trust and team performance: a meta-analysis of main effects, moderators, and covariates. Journal of Applied Psychology 2016; 101(8): 1134.
40.D’Innocenzo, L, Mathieu, JE, Kukenberger, MR. A meta-analysis of different forms of shared leadership–team performance relations. Journal of Management 2016; 42(7): 19641991.
41.Rosen, MA, et al. Teamwork in healthcare: key discoveries enabling safer, high-quality care. American Psychologist 2018; 73(4): 433.
42.Edmondson, AC. Speaking up in the operating room: how team leaders promote learning in interdisciplinary action teams. Journal of Management Studies 2003; 40(6): 14191452.
43.Chambers, DA, et al. Mapping training needs for dissemination and implementation research: lessons from a synthesis of existing D&I research training programs. Translational Behavioral Medicine 2016; 7(3): 593601.
44.Hall, KL, et al. Moving the science of team science forward: collaboration and creativity. American Journal of Preventive Medicine 2008; 35(2): S243S249.
45.Elliott, DS, Mihalic, S. Issues in disseminating and replicating effective prevention programs. Prevention Science 2004; 5(1): 4752.
46.Israel, BA, et al. Review of community-based research: assessing partnership approaches to improve public health. Annual Review of Public Health 1998; 19: 173202.
47.Concannon, TW, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. Journal of General Internal Medicine 2014; 29(12): 16921701.
48.Jones, J, Hunter, D. Consensus methods for medical and health services research. BMJ: British Medical Journal 1995; 311(7001): 376.
49.Delbecq, AL, Van de Ven, AH, Gustafson, DH. Group Techniques for Program Planning: A Guide to Nominal Group and Delphi Processes. Glenview, IL: Scott Foresman Company, 1975.
50.Van de Ven, AH, Delbecq, AL. The nominal group as a research instrument for exploratory health studies. American Journal of Public Health 1972; 62(3): 337342.
51.Fink, A, et al. Consensus methods: characteristics and guidelines for use. American Journal of Public Health 1984; 74(9): 979983.
52.Humphrey-Murto, S, et al. Using consensus group methods such as Delphi and Nominal Group in medical education research. Medical Teacher 2017; 39(1): 1419.
53.Edmondson, AC. Teaming: How Organizations Learn, Innovate, and Compete in the Knowledge Economy. John Wiley & Sons, 2012.
54.Hackman, JR. Why teams don’t work. In: Tindale, RS, et al. eds. Theory and Research on Small Groups. Boston, MA: Springer; 2002. 245267.
55.Little, MM, et al. Team science as interprofessional collaborative research practice: asystematic review of the science of team science literature. Journal of Investigative Medicine 2017; 65(1): 1522.
56.Aarons, GA, et al. Testing the Leadership and Organizational Change for Implementation (LOCI) Intervention in substance abuse treatment: a cluster randomized trial study protocol. Implementation Science 2017; 12(1): 29. doi: 10.1186/s13012-014-0192-y.
57.Drahota, A, et al. Community-academic partnerships: asystematic review of the state of the literature and recommendations for future research. Milbank Quarterly 2016; 94(1): 163214.
58.Tabak, RG, et al. Bridging research and practice: models for dissemination and implementation research. American Journal of Preventive Medicine 2012; 43(3): 337350.
59.Nilsen, P. Making sense of implementation theories, models and frameworks. Implementation Science 2015; 10: 53.
60.Aarons, GA, Hurlburt, M, Horwitz, SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Administration and Policy in Mental Health and Mental Health Services Research 2011; 38(1): 423. doi: 10.1007/s10488-010-0327-7.
61.Moullin, JC, et al. Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework. Implementation Science 2019; 14(1): 1. doi: 10.1186/s13012-018-0842-6.
62.Garg, T, et al. The AGING Initiative experience: a call for sustained support for team science networks. Health Research Policy and Systems 2018; 16(1): 41.