The University of Michigan created the Practice-Oriented Research Training (PORT) program and implemented it between 2008 and 2018. The PORT program provided research training and funding opportunities for allied healthcare professionals. The program consisted of weekly didactics and group discussion related to topics relevant to developing specific research ideas into projects and funding for a mentored research project for those who submitted a competitive grant application. The goal of this evaluation was to assess the long-term impact of the PORT program on the research careers of the participants. Ninety-two participants (74 staff and 18 faculty) participated in both phases of the program. A mixed-methods approach to evaluation was used; 25 participants who received funding for their research completed surveys, and semi-structured interviews were conducted with eight program participants. In addition, data were collected on participants’ publication history. Fifteen out of the 74 staff participants published 31 first-authored papers after participating in PORT. Twelve out of 15 staff participants who published first-authored papers did so for the first time after participating in the PORT program. Results of quantitative and qualitative analyses suggest that the PORT program had positive impacts on both participants and the research community.

Clinical and translational research relies on a well-trained workforce, but mentorship programs designed expressly for this workforce are lacking. This paper presents the development of a mentoring program for research staff and identifies key programmatic outcomes. Research staff participating in this program were matched with a senior mentor. Focus groups were conducted to identify key program outcomes. Surveys were administered throughout the program period to assess participants’ experience, gains in skill, and subsequent careers. Analysis of the resultant qualitative and quantitative data are used to characterize the implementation and impact of the program. A total of 47 mentees and 30 mentors participated in program between 2018 and 2023. A comprehensive logic model of short-, intermediate- and long-term outcomes was developed. Participants reported positive valuations of every programmatic outcome assessed including their program experience, learning and research careers. The pool of available mentors also grew as new mentors were successfully recruited for each cohort. This mentorship program developed and implemented by senior research staff successfully provided junior research staff with professional development support, mentorship, and professional development opportunities. Junior and senior health research staff built mentoring relationships that advanced their clinical and translational research careers.

OBJECTIVES/GOALS: To improve early career faculty members’ NIH grant writing skills, Clinical and Translational Science Awards (CTSA) hubs have developed a variety of workshop-style programs. However, few articles have evaluated the impact of grant writing workshops on NIH grant submission and award rates. METHODS/STUDY POPULATION: The K Writing program was developed by the Michigan Institute for Clinical and Health Research (MICHR) at the University of Michigan. Since 2012, 435 scholars have participated in the program. The MICHR K Writing program is a three-part workshop series that prepares scholars by providing them with guidelines to write all sections of a career development grant application. Each session focuses on different sections of the K award proposal. During the workshop sessions, participants break into small groups and exchange drafts of their proposal sections and receive peer critique and feedback from senior faculty facilitators who have experience with NIH study sections. RESULTS/ANTICIPATED RESULTS: Between 2012-2018, 273 scholars participated and 57% were female. Our two primary outcomes of interest are submission rates and success rates (the number of grants awarded divided by the number of applications). We plan to examine the effects of several characteristics, including number of sessions attended, cohort year, and faculty vs. postdoctoral status. We will also examine whether there were differences in submission and success rates between female and male researchers and between underrepresented minority scholars and those who identified as white or Asian. Lastly, we will report submission and success rates for each grant mechanism and compare them to the national averages. DISCUSSION/SIGNIFICANCE: Obtaining external research funding is an important part of a faculty career, especially at its early stages. This research has important implications for the design of similar programs intended to increase submission and success rates for federal grant applications.

OBJECTIVES/GOALS: There is a critical need to provide quality training for study teams. Materials need to be flexible and available at the learner’s preferred time and format. DIAMOND was developed to provide nimble education offerings that respond to the changing landscape of clinical and translational research. METHODS/STUDY POPULATION: In 2018 four CTSA institutions (the University of Michigan, the Ohio State University, Rochester University, and Tufts University) collaborated to launch the DIAMOND portal. Developed as a CTSA-wide web-based platform, DIAMOND allows members of clinical and translational research teams to widely share and access training and education resources. In 2022 a MICHR-led update of DIAMOND used principles of user centered design to improve the platform. New features include updated search functions to quickly find and sort training materials, tagging training materials to the characteristics of a translational scientist, and development of user controlled customized playlists. RESULTS/ANTICIPATED RESULTS: DIAMOND currently includes 217 training resources developed by 30 CTSA hubs and private industry. The platform has over 600 page views per day from users across the U.S. and internationally. DIAMOND includes an easy-to-use form to upload new materials to the platform. Contributors are asked to include key words and select competency domains and characteristics of a translational scientist that apply to their materials. Other new features include tagging materials to streamline and improve search results, the ability to sort materials by competency domain or characteristics of a translational scientist, and the ability for users to create and share customized, personalized playlists. DISCUSSION/SIGNIFICANCE: DIAMOND is an important tool to support workforce development for study teams. Updates to the flexible digital platform meet the needs and preferences of adult learners and busy health professionals. Lessons learned from the design process and future plans for the platform will be explored.

OBJECTIVES/GOALS: The objective of this evaluation is to show how the STEP.UP program promoted the professional development at Michigan Medicine by providing clinical and translational research staff an experienced research staff mentor in a structured 9-month program. METHODS/STUDY POPULATION: Participant and mentor data was collected from application forms, online surveys, and interviews with both participating mentors and mentees. Validated assessments of mentoring competencies were administered. Participants were tracked over a period of four years with regular reviews of institutional records. Mentor and mentor data was also collected at the point of application each year and the application forms were aligned with NIH definitions for underrepresented populations in science in 2020. As part of a process of continuous programmatic improvement, a STEP.UP Advisory Board consisting of senior research staff and past mentors was involved in the identification, operationalization and evaluation of programmatic outcomes and is involve din the ongoing governance of this mentoring program. RESULTS/ANTICIPATED RESULTS: Four cohorts of mentees and mentors have participated in this program since its inception. Mentees gained the greatest abilities in, Active listening, Establishing a relationship based on trust, Considering how personal and professional differences may impact expectations, and Working effectively with mentors/mentees whose personal background is different. Mentees reported the program contributed to their Career planning, Professional advancement, networking, personal growth, professional networks, and communication skills. Mentors reported learning about new professional techniques and areas of expertise. As of 2021, 75% the first cohort changed their job-classification since participating as did 25% of the second cohort and 100% of mentees have maintained research careers. DISCUSSION/SIGNIFICANCE: The creation of this program in 2019 marked the beginning of a novel professional development opportunity at Michigan Medicine. The evaluation results show how STEP.UP contributes to advancing clinical and translational study teams and how it can inform and the identification of best practices in clinical and translational workforce development.

OBJECTIVES/GOALS: This study examines faculty mentor experiences in a summer research program for students traditionally underrepresented in translational research. The objectives are to understand mentor perspectives of the program and how their views were impacted by the pandemic-related pivot to a remote format. METHODS/STUDY POPULATION: During the summer research program, students work with a faculty mentor on an ongoing research project. Program staff pair students with diverse faculty members who share their research interests. Our program has traditionally been offered as a residential in-person program but in 2020 we moved swiftly to a fully remote learning format. Students continued to work on research projects remotely and interacted virtually with their faculty mentors. For the past five cohort years, we have collected comparable data about faculty perspectives of their program experience, which allows us to evaluate the impact of the remote format on the faculty experience compared to that of the in-person format. RESULTS/ANTICIPATED RESULTS: For this study, we will examine data from five cohort years (2017-2021). A survey questionnaire was administered to mentors each year at the end of the summer research program. Data were collected on faculty satisfaction with the quality and amount of student work, the amount of time students spent on their projects, and how mentors communicated with students. In 2020 and 2021, three questions were added regarding satisfaction with the remote format. Quantitative data collected from both the in-person and remote cohorts will be compared using independent samples t-tests. Select quotes from open-ended qualitative questions will be used to illustrate mentors attitudes toward the program. DISCUSSION/SIGNIFICANCE: This study addresses a gap in the literature about summer research programs, as previous work has primarily focused on trainees. As many training programs continue to remain remote or adopt hybrid models, these results have implications for the design of similar mentored research programs.

Well-designed, accessible short-term research training programs are needed to recruit and retain underrepresented persons into clinical and translational research training programs and diversify the workforce. The Michigan Institute for Clinical and Health Research developed a summer research program, training over 270 students in 15 years. In response to the 2020 COVID-19 pandemic, we pivoted swiftly from an in-person format to a fully remote format. We describe this process, focusing on factors of diversity, equity, and inclusion including enabling student participation in remote research activities. We collected data about students’ learning experiences since the program’s inception; therefore, we could evaluate the impact of remote vs. in-person formats. We examined data from five cohorts: three in-person (2017–2019; n = 57) and two remote (2020–2021; n = 45). While there was some concern about the value of participating in a remote format, overall students in both formats viewed the program favorably, with students in the remote cohorts rating some aspects of the program significantly more favorably. In addition, more students who identified as Black or African American participated in the remote format than in the in-person format. We describe lessons learned from this unprecedented challenge and future program directions.

Although several initiatives have produced core competency domains for training the translational science workforce, training resources to help clinical research professionals advance these skills reside primarily within local departments or institutions. The Development, Implementation, and AssessMent of Novel Training in Domain (DIAMOND) project was designed to make this training more readily and publicly available. DIAMOND includes a digital portal to catalog publicly available educational resources and an ePortfolio to document professional development. DIAMOND is a nationally crowdsourced, federated, online catalog providing a platform for practitioners to find and share training and assessment materials. Contributors can share their own educational materials using a simple intake form that creates an electronic record; the portal enables users to browse or search this catalog of digital records and access the resources. Since September 2018, the portal has been visited more than 5,700 times and received over 280 contributions from professionals. The portal facilitates opportunities to connect and collaborate regarding future applications of these resources. Consequently, growing the collection and increasing numbers of both contributors and users remains a priority. Results from a small subset of users indicated over half accomplished their purpose for visiting the site, while qualitative results showed that users identified several benefits and helpful features of the ePortfolio.

OBJECTIVES/SPECIFIC AIMS: The purpose of this study was to summarize the existing literature on clinical research competencies and determine what competency assessments currently exist. We also wished to assess which competencies should be included in a research competency assessment tool and to evaluate the validity of current competency assessments. We also examined whether these competency assessments can be used for the purposes of formative and summative evaluation. METHODS/STUDY POPULATION: Prior to conducting our search of the literature, we first compiled a list of search terms (e.g., clinical, research, training, competencies) that could be used to locate articles. We then entered these search terms, in various combinations, on several relevant databases. We evaluated abstracts of the articles revealed by this search to determine whether they met three criteria. The first criterion was that the subjects of the article must be clinical investigators or clinical investigators in training. Relevant disciplines included medicine, public health, nursing, pharmacy, dentistry, and other related fields. The second criterion was that articles should focus on research-based (as opposed to clinical) skills. The last criterion was that research-based competencies (or related terms like skills, abilities, mastery, knowledge) must be assessed in some way. If the abstract suggested that the article met all three criteria, the full article was retrieved and analyzed in-depth. To identify articles that eluded literature search, we then examined the reference section of these articles and examined articles that cited these articles. When no additional articles could be located, the search for articles stopped. Once a pool of potentially eligible articles was identified, the articles underwent peer review by several researchers experienced with clinical research and competency-based education and assessment. Articles that were unanimously judged to meet the criteria were included in the systematic review. RESULTS/ANTICIPATED RESULTS: Approximately 75 articles were selected and reviewed for eligibility. After peer review, we found that only a small fraction of these articles met our criteria for inclusion in the systematic literature review. Our preliminary findings suggest that there are few assessments of clinical research competency and that many of these assessments are poorly validated. DISCUSSION/SIGNIFICANCE OF IMPACT: The findings of the present study suggest that the validation methods used thus far are limited and so the validity of many of these assessments is effectively unproven. Future research on assessments of clinical research competency ought to address these limitations by sampling clinical researchers, using more rigorous validation methods, and by confirming hypothesized factor structures in new samples. The use of better-validated instruments may enhance measurement of trainees’ knowledge and skill levels for the purposes of formative and summative assessment.

OBJECTIVES/SPECIFIC AIMS: The DIAMOND project encourages study team workforce development through the creation of a digital learning space that brings together resources from across the CTSA consortium. This allows for widespread access to and dissemination of training and assessment materials. DIAMOND also includes access to an ePortfolio that encourages CRPs to define career goals and document professional skills and training. METHODS/STUDY POPULATION: Four CTSA institutions (the University of Michigan, the Ohio State University, University of Rochester, and Tufts CTSI) collaborated to develop and implement the DIAMOND portal. The platform is structured around eight competency domains, making it easy for users to search for research training and assessment materials. Contributors can upload links to (and meta-data about) training and assessment materials from their institutions, allowing resources to be widely disseminated through the DIAMOND platform. Detailed information about materials included in DIAMOND is collected through an easy to use submission form. DIAMOND also includes an ePortfolio designed for CRPs. This encourages workforce development by providing a tool for self-assessment of clinical research skills, allowing users to showcase evidence of experience, training and education, and fosters professional connections. RESULTS/ANTICIPATED RESULTS: To date, more than 100 items have been posted to DIAMOND from nine contributors. In the first 30 days there were 229 active users with more than 500 page views from across the U.S. as well as China and India. Training materials were viewed most often from four competency domains: 1) Scientific Concepts & Research Design, 2) Clinical Study Operations, 3) Ethical & Participant Safety, and 4) Leadership & Professionalism. Additionally, over 100 CRPs have created a DIAMOND ePortfolio account, using the platform to document skills, connect with each other, and search for internships and job opportunities. DISCUSSION/SIGNIFICANCE OF IMPACT: Lessons learned during development of the DIAMOND digital platform include defining relevant information to collect for the best user experience; selection of a standardized, user-friendly digital platform; and integration of the digital network and ePortfolio. Combined, the DIAMOND portal and ePortfolio provide a professional development platform for clinical research professionals to contribute, access, and benefit from training and assessment opportunities relevant to workforce development and their individual career development needs.

OBJECTIVES/SPECIFIC AIMS: Identify the impact of the provision of clinical and translational research training awards on investigators’ pursuit of clinical and translational research careers. METHODS/STUDY POPULATION: Propensity score matching and qualitative analysis/investigators receiving MICHR’s KL2 research training awards. RESULTS/ANTICIPATED RESULTS: While the evaluations of the impact of this service have shown participants find them to be valuable it is expected that participation in the workshop may be more beneficial to investigators with certain types of prior research experiences and who utilize more CTSA research support. DISCUSSION/SIGNIFICANCE OF IMPACT: Because this evaluation of a research service incorporate data representing investigator’s receipt of different CTSA resources, the findings can be used to inform the ongoing coordination of these services in ways that optimize their impact on the production of clinical and translational research. There is an enduring need for evaluations of CTSA programs to account for investigators’ use of different constellations of research services in order to identify what combinations of services over time are most effective at fostering successful clinical and translational research careers.

OBJECTIVES/SPECIFIC AIMS: To evaluate the NIH-sponsored Best Practices for Social and Behavioral Research e-learning course. METHODS/STUDY POPULATION: Four universities partnered in a pilot study to evaluate this new course. Outcomes from 294 participants completing the course included efficient progress through the training, perceived relevance of the course to current work, level of engagement with the course material, intent to work differently as a result of the course, and downloading digital resources. RESULTS/ANTICIPATED RESULTS: Participants rated the course as relevant and engaging (6.4 and 5.8 on a 7-point Likert scale) and 96% of respondents said they would recommend the course to colleagues. Qualitative analysis of participant testimonials suggested that most respondents had a readiness to change in the way they worked as a result of the course. Overall, results suggest participants completed the course efficiently, perceived outcomes positively and worked differently after the training. DISCUSSION/SIGNIFICANCE OF IMPACT: These results will inform new guidelines for future participants (e.g., average time to complete, expectations for knowledge checks in the training). Future studies should include larger samples and closer coordination and communication between study sites.

OBJECTIVES/SPECIFIC AIMS: The first goal of this project is to test the reliability and validity of an objective structured clinical exam (OSCE) that was designed to assess competency in clinical and translational research. The second goal is to evaluate the impact of MICHR’s Summer Research Program on the participating trainee’s competency development. METHODS/STUDY POPULATION: The methodology used for this study was reviewed and exempted from oversight by the U-M Institutional Review Board (HUM00113293). The participants in the study include 17 pre-doctoral students in health professions programs at U-M who participated MICHR’s Summer Research Program. The Research OSCE was administered using a pretest, post-test design. The pretest was administered once during the 1st week of program in the Summer of 2016 and the post-test during the 10th week of the program. The Research OSCE was proctored and rated by trained staff members. We will assess the reliability of the Research OSCE using Generalizability Theory (Webb et al., 2006). And the construct validity of the Research OSCE will be tested using factor analysis and other statistical analyses. Growth in the competence of the trainees participating in the Summer Research program will be evaluated by testing for significant differences between their pretest and post-test scores. RESULTS/ANTICIPATED RESULTS: We anticipate that this study will show that the Research OSCE is a reliable competency assessment with proven construct validity. We also anticipate that the use of the Research OSCE will show the trainees participating in the Summer Research program experienced a gain in competence during the course of the 10-week program. DISCUSSION/SIGNIFICANCE OF IMPACT: This project uses a common and standardized testing approach. The primary goal of this project is to evaluate the reliability and validity of an OSCE to assess competency in clinical and translational research. It represents a new application for a well-studied testing method used extensively in the health professions to assess the clinical competency of health practitioners. This project will lead to a better understanding of (a) the reliability and validity of the Research OSCE designed to test research competency and (b) the effectiveness of the Summer Research Program curriculum in better preparing participants to conduct clinical and translational research. Showing how a specific competency assessment can be used for this purpose will provide the administrators, evaluators, and other stakeholders of clinical and translational research training programs with information that can be used to design more rigorous and relevant evaluations of their research training programs.

OBJECTIVES/SPECIFIC AIMS: To conduct a preliminary evaluation of the Social and Behavioral Research Best Practices Course. METHODS/STUDY POPULATION: Learners are sampled from 5 institutions: University of Michigan, University of Rochester, University of Florida, Boston University, and University of Buffalo. Learners who take the course and consent to be in the study receive a web link to a survey immediately after course completion and at 2–3 months follow up. In addition to demographic information, learners will report their perceptions of usefulness and relevance of the course to their job, their satisfaction with the course and associated job aids, and at follow-up, if and how the course impacted their work. Additional information will be collected from the learning management systems which host the course at each institution. The data collected will include the number of participants who take the course, the number who complete, how many times the course was attempted, and pass rates. RESULTS/ANTICIPATED RESULTS: We anticipate that several hundred learners will take the course by the end of our project. Of learners who agree to participate in the survey, we anticipate that they will find the course useful and relevant to social and behavioral clinical trials and will be satisfied with the course. Information including suggestions about missing content, items or content that were not extremely clear, or any other comments will be collected to iterate and expand the course. DISCUSSION/SIGNIFICANCE OF IMPACT: This course was developed to fill a gap in training in good clinical practice for social and behavioral research. An evaluation of how the training provided in the course impacts the jobs of learners is needed both to ensure that the most relevant information is included in the course as well as to identify ways that the training may contribute to the quality and safety of social and behavioral clinical trials.