In total, 294 individuals across the partner sites who participated in the Best Practices in SBR Course consented to participate in the pilot evaluation. Differences in course and survey administration among sites are outlined in Table 1. Over 73% (n=217) of participants completed the training course, while over 49% (n=107) took the survey.
Characteristics of Survey Respondents
Over 70% (n=75) of participants who responded to the survey reported that they took the course to fulfill training requirements, although 17% participated in the training voluntarily, without the expectation of fulfilling any training requirements. Many participants were currently conducting or supporting clinical trials research. Of the 105 individuals who indicated their involvement in clinical trials, 15% were not currently involved in any clinical trials research, and almost 17% were involved in more than one type of clinical trial. A majority of these studies (n=112) were social and behavioral trials (72%), although many were drug, device (17%), or biological clinical trials (8%). Just over 12% of participants supported clinical trials through multiple roles. Of the total number of roles in clinical trials they reported holding (n=103), almost 70% were Research Coordinators, Research Assistants or other team members and roughly 30% were Co- or Principal Investigators. Similarly, about 17% of participants reported having multiple occupational roles regardless of their involvement in clinical trials research. The diversity of roles held by participants is shown in Fig. 1.
Fig. 1 Survey respondents by research role (115 roles reported).
The participants who responded to the survey also reported that they had considerable prior research experience. On average, they reported having been engaged in social and behavioral research for 7.1 years (n=78, SD=8.3) and in clinical trials research for 6.5 years (n=84, SD=7.7). Information about participants’ highest postsecondary degrees was also collected from 93 individuals. Roughly a third of respondents had earned their Doctorate (32%, n=30), Masters (33%, n=31), or Bachelors (32%, n=30) degrees. Two individuals (2%) reported their highest credential was an Associate’s degree. Information about other types of professional credentials, such as SOCRA and ACRP certifications was solicited from respondents, but too few participants responded to the questions to enable conclusions to be drawn about participants’ acquisition of other professional credentials.
We hypothesized that participants would rate the course as relevant, engaging, and impactful to their work, and this was largely supported. Of the 3 training outcomes in which 90 participants rated agreement on a scale from 0 to 7 at baseline, they most strongly endorsed the course’s relevance (6.4±1.0), followed by the training being engaging (5.8±1.2), and working differently as a result of the training (4.7±1.6). There were no significant differences in outcomes by research role, experience in social and behavioral trials, or by whether the training was taken as a requirement or not (see Fig. 2). Participants who did not currently work on social and behavioral trials compared with those who did had a trend of reporting that the course was more engaging (p≤0.10). When asked if they would recommend the course to their colleagues during the initial survey, 96% (n=70) agreed. After 2 months having completed the training, 90% (n=48) agreed.
Fig. 2 Perceptions of course from initial survey by participant characteristics.
An examination of the predictors of working differently as a result of the training 2 months after training showed that 32% of the variance was explained by the model (F3,23, p=0.001; adjusted R
2=0.32) that included participant’s initial endorsement of working differently as a result of the training (β=0.50, p=0.005), and years of experience conducting clinical trials research (β=−0.07, p=0.001). The results indicate that the endorsement of working differently at baseline and fewer years of experience are associated with working differently 2 months later.
Qualitative responses were analyzed to understand if and how people worked differently as a result of the training according to TTM stage of change are summarized in Table 3. The responses described a variety of ways in which respondents were thinking about working differently or making actionable changes. There was a trend towards more active change at the 2 month survey. In the initial survey, 44 participants responded to the question in which they were asked to describe how they would work differently. Eight responses were in the Pre-contemplation stage, and 18 in both the Contemplation and Preparation stages. At the 2 month follow-up survey, 30 participants responded to the question about working differently as a result of having received this training. There were 5 responses in the Pre-contemplation stage, 6 in Contemplation, 5 in Preparation, and 14 in Action. At least half of the responses were descriptions of the ways that participants implemented ideas culled from the training experience.
Table 3 Stages of change, definitions, and selected quotations about working differently as a result of the course
An example of Pre-contemplation in the initial survey is:
I liked the use of real life examples of research impropriety and other situations which I can foresee easily occurring. …These modules gave a good foundation for the principles of protection of human subjects and ethical research.
In the comment given above and for other comments in the Pre-contemplation stage, participants described what they liked about the training content. They remarked about the value of the training; however, they offered no indication of how the learning experiences could be applied to their research roles.
At follow-up, the Pre-contemplation stage was exemplified by the following statement:
[I] haven’t worked differently; either not applicable or already knew.
In this example and others in this stage, participants expressed no intention to change behavior in the foreseeable future.
The following response was an example of participants in the Contemplation stage after the initial survey:
The training was a great way to enforce a non-complacent approach to doing research. It will motivate me to think thoroughly in addition to providing materials to look back to for reference/refreshing.
At the follow-up survey, the following statement exemplifies the Contemplation stage:
Being more mindful of consenting process and overall operational procedures; The training provided clear examples of how members of a research team can face special challenges that may not be captured by [a different training program].; I have limited capacity to approve new forms or [standard operating procedures] (SOPs). However, I have more awareness.
This statement and others in this stage described participants becoming more mindful and conscious and suggesting that they had developed a new understanding of their job responsibilities. Although how and what types of changes the participants might make in the future is not articulated, it is important to recognize that in the absence of awareness, change is unlikely. Participants in this stage explained how they are thinking about the connection between training and their roles and indicate a newfound awareness.
The following response exemplified participants in the Preparation stage after the initial survey:
Engage in QA (Question/Answer) activities during the study more frequently. Maintain better record keeping of study related activities.
In the comment above and others indicative of the Preparation stage, participants expressed an intention of how they planned to change their behavior, such as by becoming more mindful or by engaging more frequently in quality assurance activities and maintaining better records.
At follow-up, the Preparation stage was exemplified by the following statement:
I am more knowledgeable about what should/should not be done in clinical practice, so I am able to communicate what I know to others in order to make sure our entire research team is conducting good clinical practice.
The statement above and others at this stage not only signify participant awareness, but they also specify their intentions to make changes.
At 2 months follow-up, almost half of participants who responded (14/30) described a change that was classified in the Action stage:
I have created better protocol documents for future study coordinators.
[I] started writing SOPs for different task areas, have implemented a data check for QA, and constantly discuss the importance of fidelity to our assessment protocol and procedures.
These statements describe the behavioral changes that participants have made during the training. Creating better protocol documents, writing standard operating procedures, and implementing data checks are clearly actionable steps.