Hostname: page-component-5d59c44645-klj7v Total loading time: 0 Render date: 2024-02-29T03:26:30.711Z Has data issue: false hasContentIssue false

Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India

Published online by Cambridge University Press:  29 November 2023

Sonal Mathur
Affiliation:
Sangath, New Delhi, India
Helen A. Weiss
Affiliation:
Medical Research Council International Statistics and Epidemiology Group, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
Melissa Neuman
Affiliation:
Medical Research Council International Statistics and Epidemiology Group, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine, London, UK
Baptiste Leurent
Affiliation:
Department of Statistical Science, University College London, London, UK
Andy P. Field
Affiliation:
School of Psychology, University of Sussex, Brighton, UK
Tejaswi Shetty
Affiliation:
Sangath, New Delhi, India
James E. J.
Affiliation:
Sangath, New Delhi, India
Pooja Nair
Affiliation:
Sangath, New Delhi, India
Rhea Mathews
Affiliation:
Sangath, New Delhi, India
Kanika Malik
Affiliation:
Jindal School of Psychology and Counselling, O.P. Jindal Global University, Sonipat, India
Daniel Michelson
Affiliation:
School of Psychology, University of Sussex, Brighton, UK Department of Child & Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, UK
Vikram Patel*
Affiliation:
Department of Global Health and Social Medicine, Harvard Medical School, Boston, MA, USA Department of Global Health and Population, Harvard T.H. Chan School of Public Health, Boston, MA, USA
*
Corresponding author: Vikram Patel; Email: vikram_patel@hms.harvard.edu
Rights & Permissions [Opens in a new window]

Abstract

We evaluated a digital learning programme for non-specialists to develop knowledge-based competencies in a problem-solving intervention for adolescents to examine the overall impact of training on knowledge-based competencies among learners; and to compare the effects of two training conditions (self-guided digital training with or without coaching) in a nested parallel, two-arm, individually randomised controlled trial. Eligible participants were 18 or older; fluent in Hindi or English; able to access digital training; and had no prior experience of delivering structured psychotherapies. 277 participants were enrolled from 31 March 2022 to 19 June 2022 of which 230 (83%) completed the study. There was a significant increase in competency score from pre-training (Mean = 7.01, SD = 3.29) to post-training (Mean = 8.88, SD = 3.80), 6 weeks after the pre-training assessment. Knowledge competency scores showed larger increase among participants randomised to the coaching arm (AMD = 1.09, 95% CI 0.26–1.92, p = 0.01) with an effect size (d) of 0.33 (95% CI 0.08–0.58). More participants completed training in the coaching arm (n = 96, 69.6%) compared to the self-guided training arm (n = 56, 40.3%). In conclusion, a coach-supported remote digital training intervention is associated with enhanced participation by learners and increased psychotherapeutic knowledge competencies.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Impact statement

This randomised controlled trial investigates knowledge-based learning outcomes among non-specialist providers following digital training on an evidence-based youth mental health intervention (problem-solving therapy). We compared two digital training formats (self-guided digital training vs. digital training with coaching) and found that both formats led to increased knowledge competency scores, with an incremental effect observed in the coaching arm. We also found higher levels of engagement among participants in the coaching arm. The findings suggest that automated pre-recorded training augmented by periodic coaching is a promising approach that could be used at scale to develop the knowledge base of prospective practitioners of psychosocial interventions in task-sharing initiatives.

Introduction

Task-sharing of psychotherapies is an effective strategy for improving access to evidence-based mental health care, particularly in low-resource contexts (Naslund et al., Reference Naslund, Shidhaye and Patel2019b). Scaling this approach requires the expansion of service delivery roles to include a wide range of non-specialist providers such as lay people and community health workers (Hoeft et al., Reference Hoeft, Fortney, Patel and Unützer2018; Raviola et al., Reference Raviola, Naslund, Smith and Patel2019). While digital innovations have been developed and tested with the goal of increasing access to effective task-sharing interventions (Singla et al., Reference Singla, Kohrt, Murray, Anand, Chorpita and Patel2017; Michelson et al., Reference Michelson, Malik, Parikh, Weiss, Doyle, Bhat, Sahu, Chilhate, Mathur, Krishna, Sharma, Sudhir, King, Cuijpers, Chorpita, Fairburn and Patel2020), these innovations have typically addressed the mode and setting of intervention delivery (e.g., using internet-enabled devices as a vehicle for delivering brief psychotherapies outside of conventional clinic settings). Much less research has been done to evaluate the use of digital technologies for building workforce capacity (Naslund et al., Reference Naslund, Gonsalves, Gruebner, Pendse, Smith, Sharma and Raviola2019a). This evidence gap is a major barrier to scaling up task-sharing of psychotherapies, given that traditional models of in-person, expert-led training are time- and labour-intensive (van Ginneken et al., Reference van Ginneken, Chin, Lim, Ussif, Singh, Shahmalak, Purgato, Rojas-García, Uphoff, McMullen, Foss, Thapa Pachya, Rashidian, Borghesani, Henschke, Chong and Lewin2021; Philippe et al., Reference Philippe, Sikder, Jackson, Koblanski, Liow, Pilarinos and Vasarhelyi2022).

The Premium for Adolescents (PRIDE) programme is a recent exemplar of task-sharing in the field of adolescent mental health. PRIDE was implemented in India from 2016 to 2022 and aimed to address the scarcity of evidence-based interventions for common adolescent mental health problems in the global health context. The goal was to develop and evaluate a suite of scalable, transdiagnostic psychological interventions that could be delivered by non-specialist providers for a variety of mental health presentations in school settings. The programme was intended to generate policy-relevant knowledge in response to India’s national initiative for adolescent health, Rashtriya Kishor Swasthya Karyakram. This national policy programme emphasised mental health as a public health priority and schools as an important platform for youth-focused psychosocial interventions (Roy et al., Reference Roy, Shinde, Sarkar, Malik, Parikh and Patel2019).

PRIDE sought to overcome the resource limitations of expert-led, in-person training by developing a digital learning platform to train non-specialist providers in an evidence-based problem-solving intervention. This learning platform was originally created by Sangath to train non-specialist providers in a brief psychotherapy for adults with depression (Khan et al., Reference Khan, Shrivastava, Tugnawat, Singh, Dimidjian, Patel, Bhan and Naslund2020) It is designed to host modules comprising video lectures with accompanying role-play demonstrations, narrated teaching slides, self-assessment quizzes, and assigned readings. The Sangath learning platform has recently completed evaluation in a 3-arm randomised control trial (Muke et al., Reference Muke, Tugnawat, Joshi, Anand, Khan, Shrivastava, Singh, Restivo, Bhan, Patel and Naslund2020) which compared self-guided digital training and digital training augmented by coaching with the gold standard of in-person, expert-led training.

Building on this body of research, we aimed to evaluate participant engagement and learning outcomes for a modular, digital training course built around a brief transdiagnostic problem-solving intervention for common adolescent mental health problems (i.e., anxiety, depression and conduct difficulties). Our group has previously demonstrated the short- and medium-term, effectiveness of this problem-solving intervention when delivered by lay counsellors in schools serving low-income communities in New Delhi (Michelson et al., Reference Michelson, Malik, Parikh, Weiss, Doyle, Bhat, Sahu, Chilhate, Mathur, Krishna, Sharma, Sudhir, King, Cuijpers, Chorpita, Fairburn and Patel2020; Malik et al., Reference Malik, Michelson, Doyle, Weiss, Greco, Sahu, James, Mathur, Sudhir, King, Cuijpers, Chorpita, Fairburn and Patel2021). The goals of the current study were to:

  1. (1) evaluate the effects of digital training on knowledge-based competencies in relation to problem-solving therapy for common adolescent mental health problems;

  2. (2) evaluate the incremental effect of digital training with coaching (DT-C) in comparison with self-guided digital training (DT) on competencies; and

  3. (3) assess participant engagement in, and satisfaction with, the two training conditions.

Our hypotheses were:

  1. (1) participation in either digital training format will lead to increased knowledge-based competency scores among non-specialists; and

  2. (2) DT-C will be more effective than DT at increasing knowledge-based competency scores.

Methods

Design and setting

The study was a parallel, two-arm, individually randomised controlled trial design (comparing DT and DT-C) nested within a pre-post intervention study (comparing pre- and post-training learning outcomes for participants across both training conditions).

Participants

To increase the generalisability of findings, the study sample was drawn from varied backgrounds in India. Participants comprised two groups: (i) university students currently enrolled in a bachelor’s-level degree programme in psychology, education or allied fields; and (ii) non-governmental organisation (NGO) staff working as teachers, social workers or mental health advocates. Group (i) was recruited from two co-educational private (one being charity-aided) colleges in Delhi-NCR region; one co-educational private college in Bangalore, Karnataka region; and one girls-only government-aided private college in Mumbai, Maharashtra region. Group (ii) was recruited from four NGOs based in Delhi and one NGO based in Mumbai. Eligible participants in both groups were aged 18 years or older; fluent in written and spoken Hindi or English; and able to access an internet-enabled device as needed to engage in the training. We excluded individuals with prior training in/experience of delivering structured psychotherapies for young people or any other population.

Sample size calculation

We aimed to recruit 262 participants in the study, with the expectation that 210 participants would complete a follow-up assessment (i.e., allowing for 20% drop-out). For the first hypothesis, this sample size provides 80% power to detect an effect size of 0.19 (i.e., a standardised mean difference (SMD) of post- vs. pre-training scores for 210 participants) at the 2-sided 5% type-I error rate. This indicative effect size (SMD = 0.19) was informed by a systematic review and meta-analysis of online learning evaluations which compared analogous learning conditions (Means et al., Reference Means, Toyama, Murphy, Bakia and Jones2009). For the second hypothesis, a sample size of N = 210 (105 per arm) provides 80% power to detect an effect size (SMD) of 0.39 between the DT and DT-C arms. Due to the enrolment of participants in weekly batches, the final recruited sample size (N = 277) slightly exceeded the original target. Data collection was completed in August 2022.

Participant enrolment

We held online webinars with the collaborating institutions to raise awareness about the study and associated digital training. Webinars were publicised using existing email lists and WhatsApp groups maintained by the various universities and NGOs. The webinars were hosted on Zoom and facilitated by a member of the research team using a slide show with video demonstration of the digital training course followed by a question-and-answer session with the attendees. Webinars were conducted at regular intervals from March to June 2022 to maintain a rolling flow of referrals. Following the webinar, interested participants were provided with a weblink to the study website (hosted on the REDCap platform) where they were prompted through a series of eligibility questions about age, occupation, device access, language proficiency, and prior training/other experience in psychotherapies. They were subsequently provided with further written information about the study and invited to give consent by means of a digital signature on the study website. Upon completion of the digital training course, all participants received a training completion certificate. Additionally, on completion on the digital training course and the post-training outcome measures, all participants received a gift card worth 500 Indian Rupees (approx. US$6) to offset the cost of data incurred in completing the study.

Randomisation and blinding

Immediately after completing the baseline questionnaire, participants were randomly allocated to one of the two trial arms. Randomisation was based on a computer-generated list of block sizes 4 and 6 stratified by organisation (NGO or university). This list was programmed into REDCap for automated randomisation. Participants were informed of their allocation by email, which also included a link to access the training programme and login details. Only the data manager (JEJ) had access to the randomisation list and all other study members were blinded to the allocation until final analysis. The participants and coaches were not blinded to allocation status.

Interventions

Self-guided DT arm

The digital training programme contained 16 modules, organised sequentially in two sections: non-specific counselling skills and skills that are specific to problem-solving therapy. The course content was adapted from an existing intervention manual (PRIDE, Sangath, & New Delhi, 2022), which was previously tested in an RCT (N = 250) that compared counsellor-led problem-solving (supported by problem-solving booklets) with problem-solving booklets alone in a target population of school-going adolescents with elevated mental health presentations (Michelson et al., Reference Michelson, Malik, Parikh, Weiss, Doyle, Bhat, Sahu, Chilhate, Mathur, Krishna, Sharma, Sudhir, King, Cuijpers, Chorpita, Fairburn and Patel2020; Malik et al., Reference Malik, Michelson, Doyle, Weiss, Greco, Sahu, James, Mathur, Sudhir, King, Cuijpers, Chorpita, Fairburn and Patel2021). The intervention had sustained effects on global psychopathology (d = 0.21), internalising problems (d = 0.22) and idiographic psychosocial problems (d = 0.34) over 12 months. These durable effects were obtained despite a rapid delivery schedule comprising only 4–5 face-to-face sessions (lasting 20–30 min each) over 3 weeks.

The steps taken to translate the manual into a digital curriculum are described in the published study protocol paper (Mathur et al., Reference Mathur, Weiss, Neuman, Field, Leurent, Shetty, J, Nair, Mathews, Malik, Michelson and Patel2023b). The course was available in two languages, English and Hindi, either of which could be selected by the participants. Participants were expected to progress through the material within 6 weeks of enrolment. The material was accessible in a predetermined sequence, with four modules unlocked each week over four successive weeks. Participants could only progress through the modules in a specified order and had to complete the preceding material before the next set of four modules became accessible. Weekly emails and notifications on the digital platform served as reminders and motivators for course completion. Apart from addressing technical queries (e.g., related to accessing and navigating the digital platform) through a dedicated WhatsApp number, participants had no other contact with the study team for the duration of the training programme.

DT-C arm

In addition to the digital training programme, participants in the DT-C arm received up to four personalised coaching sessions, delivered remotely via voice calls at weekly intervals during the course (average duration of coaching calls = 25 min). In line with the wider pedagogical literature (Irby, Reference Irby2018), coaching focused on assistive tasks to support individual performance rather than tasks aimed at specific improvements in learning goals. The latter would be more consistent with the related concept of tutoring. Though coaching was primarily delivered through means of phone calls, participants also had the option to ask queries via SMS text messages to their coach in between scheduled coaching sessions. Coaching sessions involved reviewing course concepts, clarifying content-related queries from participants, assisting with time management, troubleshooting other challenges to course completion, and positively reinforcing progress.

There were four coaches (three females; one male), each of whom was a lay counsellor who had previously completed training in the problem-solving intervention; two of these coaches had additionally gained experience of applying the problem-solving intervention in practice. Two of the coaches had previously obtained bachelor’s degrees and two had master’s degrees. The coaches were also provided with a 2-week training, which consisted of didactic lectures, reading materials, role-play demonstrations, and mock coaching session. Coaches participated in weekly group supervision led by a masters-level Psychologists. Supervision entailed listening to audio-recordings of coaching sessions, which were rated for quality by the coach responsible, their peers and their supervisor (quality rating tool available on request). Ratings covered several aspects of coaching structure and coaching skills, with each item rated from 1–4 (higher scores indicating higher quality). Peers and supervisors also provided formative feedback on recordings and offered suggestions for future coaching sessions, as required. The development and content of the coaching protocol have been described in greater detail in the published study protocol (Mathur et al., Reference Mathur, Weiss, Neuman, Field, Leurent, Shetty, J, Nair, Mathews, Malik, Michelson and Patel2023b).

Measures

Primary outcome

The primary outcome was the change in scores on a knowledge-based competency measure, the Knowledge Of Problem Solving (KOPS) scale (see Supplementary Material). Taking a broad definition of “competency” as “the extent to which a therapist has the knowledge and skill required to deliver a treatment to the standard needed for it to achieve its expected effects” (Fairburn and Cooper, Reference Fairburn and Cooper2011), the KOPS scale focuses on the former knowledge-based domain. As such, the assessed items correspond to “knowing” and “knowing how” rather than “showing how” or “doing” in the nomenclature of Miller’s (Reference Miller1990) hierarchy of clinical competency. Development and validation of the measure have been described elsewhere (Mathur et al., Reference Mathur, Michelson, Shetty, Patel and Field2023a). The measure comprised five session vignettes for a hypothetical case, with each vignette followed by 3–4 multiple-choice questions that asked about the most appropriate response to a practice-based scenario. Two 17-item parallel forms of the KOPS were administered at baseline and endline assessments, with the sequencing of the two forms determined at random. Thus, the participants who received version A of the form at baseline received version B of the form at endline and vice versa. A total KOPS score was assigned by summing correct scores (1 point for each correct answer) for 16 items, with one item discarded due to poor psychometric performance. We also conducted a sensitivity analysis using the full 17-item scale.

Secondary outcomes

Participants’ satisfaction with training: We used a 26-item version of the eMpowerment, Usefulness, Success, Interest, Caring (MUSIC) questionnaire (Jones and Skaggs, Reference Jones and Skaggs2016). MUSIC is a measure of satisfaction with educational programmes that have been used in previous digital training trials in India to compare training experiences between groups (Muke et al., Reference Muke, Tugnawat, Joshi, Anand, Khan, Shrivastava, Singh, Restivo, Bhan, Patel and Naslund2020; Naslund et al., Reference Naslund, Tugnawat, Anand, Cooper, Dimidjian, Fairburn, Hollon, Joshi, Khan, Lu, Mitchell, Muke, Nadkarni, Ramaswamy, Restivo, Shrivastava, Singh, Singla, Spiegelman, Bhan and Patel2021). Items on the questionnaire were rated on a 6-point scale, covering respective subscale domains of feasibility, acceptability, adoption, and appropriateness. These subscales were scored and analysed separately and not as a total, consistent with prior use. The scores for each subscale ranged from 1 to 6 where higher scores indicate greater levels of satisfaction. Two supplementary free-text items were also used to obtain written qualitative feedback from participants about what they enjoyed the most in the course, as well as suggestions for improvement. These qualitative data have not been reported in the current article.

Training completion: This was scored positive for those participants who completed all 16 modules of the digital training.

Process indicators

Fidelity of coaching sessions was measured in two ways: first, through the number of completed coaching sessions; and second, through the assessed quality of coaching using a new scale developed for the study. Only the ratings provided by supervisors on the quality rating scale (CQRS) described above were considered for quality assessment of the sessions with scores ranging from 1 to 4 (higher scores indicating higher quality).

Statistical analysis

A statistical analysis plan was finalised before unblinding. Analyses were conducted on an intention to treat principle. Descriptive statistics were used to describe baseline characteristics of participants and variables related to engagement in study procedures (see CONSORT flow diagram, Figure 1). Missing outcomes were imputed using multiple imputations by chained equations under a missing at random assumption. The imputation model was stratified by arm and included the variables in the analysis and those associated with missingness (see Appendix A1). Fifty imputations were performed.

Figure 1. Trial flow chart.

The first hypothesis (analysis of pre vs. post-training competency score) was analysed by fitting a linear regression of the change in competency score between baseline and 6 weeks. The second hypothesis (comparison of DT vs. DT-C) was analysed by fitting a linear regression of the change in competency score between baseline and 6 weeks, testing for a difference between the two arms, adjusted for baseline competency score and strata (NGO vs. university). A similar linear or logistic model was used to compare secondary outcomes (MUSIC subscales and course completion) between DT and DT-C arms, adjusted for strata. We conducted sensitivity analyses for the primary outcome using the 17-item version of the questionnaire, and without imputation (complete-case analysis).

We assessed the heterogeneity of training and coaching effects between pre-specified subgroups (age, gender, language, and organisation). For the training effect, we tested for a difference in competency score change between subgroups. For the coaching effect, we tested for an interaction term between trials arm and subgroups. Dose–response effect was investigated descriptively by considering the mean change in competency score by number of DT modules and coaching calls completed.

All analyses were conducted in Stata version 17, and statistical significance was considered at the two-sided 5% alpha level.

Results

Participants enrolment and study flow

The collaborating organisations referred 491 individuals, from which 421 (85.8%) were assessed for eligibility (Figure 1). Out of the assessed individuals, 277 (65.8%) enrolled in the study (Figure 1). The mean age of enrolled participants was 26.2 years (SD = 6.8; range: 25.3–27, 95% CI). Most participants were female (229, 82.7%) and NGO members (155, 56.0%) (see Table 1). Participants were randomised to either digital training alone (DT, n = 139) or digital training with additional coaching (DT-C, n = 138). There was a good balance between the two arms on all baseline characteristics (Table 1). Follow-up at 6 weeks was completed by 230 participants, the rest could not be contacted for follow-up (83.0%). Those lost-to-follow-up tended to be younger, were more likely to be in the DT arm, and were less likely to have completed the digital training (Appendix Table A1). Details of data completion and outcomes before imputation have been reported in Appendix Table A2.

Table 1. Baseline characteristics of study participants by arm and combined

Note: No missing data.

DT, digital training; DT-C, digital training with coaching; NGO, non-governmental organisation; SD, standard deviation.

Primary outcome

Change in knowledge-based competency score after digital training

At baseline the overall mean competency score was 7.01 (range: 0–15, 95% CI 6.62–7.40, n = 277; Table 1). At follow-up, the mean score was 8.88 (range: 0–15, 95% CI 8.39–9.38, n = 230). Based on imputed data, the mean change in competency score between baseline and follow-up was 1.72 (95% CI 1.33–2.12, p < 0.001), corresponding to an effect size (standardised mean difference) of 0.52 (95% CI 0.40 to 0.64).

Effect of coaching on knowledge-based competency

Participants randomised to the DT-C arm had greater improvement in competency score compared to those in the DT arm (adjusted mean difference (AMD) adjusted for baseline competency score and stratum = 1.09, 95% CI 0.26–1.92, p = 0.01; Table 3), corresponding to an effect size (d) of 0.33 (95% CI 0.08–0.58). Results of the sensitivity analyses were similar (Appendix Table A3).

Intervention completion

Overall, 152 (54.9%) participants completed all 16 modules of the digital training (Table 2), while 43 (15.5%) did not log in even once. Average time to complete the digital training course was 25.7 days (range: 24.0–27.3, 95% CI). Among the 138 participants in the DT-C arm, 69 (50.0%) completed all four coaching sessions, and 23 (16.7%) did not attend any coaching sessions (Table 2).

Table 2. Change in competency score for participants by number of completed modules and coaching sessions provided

Note: Post-hoc p-values for chi-squared test for trend are p < 0.001 for DT modules completed, and p = 0.02 for coaching sessions.

DT, digital training; DT-C, digital training with coaching; SE, standard error.

a Mean change in competency score between baseline and follow-up, based on imputed data.

b For DT-C arm only.

There was strong evidence that participants in the DT-C arm were more likely to complete the entire digital training (69.6% vs. 40.3%, adjusted odds ratio (OR) = 3.40, 95% CI 2.07–5.60, p < 0.001; Table 3). There was some evidence that MUSIC subscale scores were higher in the DT-C versus DT participants (AMD range: 0.11 (Success) to 0.27 (Interest), however the p-values ranged from 0.02 (Interest) to 0.25 (Success) with interest being the only significant one; Table 3).

Table 3. Comparison of outcomes at follow-up between DT and DT-C arm (n = 277, imputed data)a

CI, confidence interval; DT, digital training; DT-C, digital training with coaching; OR, odds ratio; SE, standard error.

a Based on imputed data, except for completed all modules (no missing data). Participants with follow-up competency scores numbered 230 (109 and 121 in DT and DT-C arms, respectively).

b All analyses have been adjusted for organisation strata (NGO or university). Change in competency score has been adjusted for baseline competency score.

There were a total of four coaches, most of them female (3, 75%). Their mean age was 29.8 (SD = 8.6) years, two had completed education till bachelors and two had completed masters.

Process indicators

There was good quality of the coaching session as evaluated by supervisors (CQRS range 3.59 to 3.96, Appendix Table A4).

Overall, participants in the DT-C arm attended 2.8 coaching sessions (SD = 1.54). Those participants who met the course completion criteria attended 3.5 coaching sessions (SD = 0.92), compared with 1.02 coaching sessions (SD = 1.28) for non-completers.

A total of 114 (82.6%) participants in the DT-C arm raised at least one query regarding course content, navigation or technical aspects of which 91 (79.8%) were during the coaching session and the remaining via WhatsApp messages.

Subgroup analysis

There was some evidence that the increase in competency score after digital training was greater for university students (0.88, p = 0.02), participants fluent in English language (0.92, p = 0.05), and younger participants whereby older participants showed significantly lower change (−0.72, p = 0.07), but not for gender (−0.03, p = 0.96) (Table 4). There was no evidence of heterogeneity of the coaching effect by these subgroups (p-values for interaction from 0.16 to 0.67; Table 4).

Table 4. Effect-modification of change in competency score and in coaching effectiveness by pre-specified sub-groups

Note: Based on imputed data (except baseline competency score, no missing). Participants with follow-up competency scores numbered 230 (109 and 121 in DT and DT-C arms, respectively).

CI, confidence interval; DT, digital training; DT-C, digital training with coaching; NGO, non-governmental organisation.

a Adjusted for strata and baseline competency score.

Dose–response analysis

Table 2 shows the mean competency score according to the number of DT and coaching sessions completed. Participants who did not complete any module showed a reduction in competency score from pre- to post-training (−0.19, SE = 0.65). Those who completed up to half of the modules showed a slight positive increase in competency score (0.78, SE = 0.43). Participants who completed 9–15 modules showed a more substantial increase in competency (1.38, SE = 0.80), with an even larger positive change observed among participants who completed all 16 modules (2.71, SE = 0.22).

Similarly, participants in the DT-C arm who did not attend any coaching sessions showed a small positive change in competency 0.52 (0.89), whereas those who attended 1–3 coaching sessions showed a relatively larger increase (2.30, SE = 0.52). The largest positive change was seen among those participants who attended all 4 coaching sessions (2.77, SE = 0.34).

Discussion

This study aimed to evaluate the effects of two digital formats for training non-specialist providers in an evidence-based psychotherapy for common adolescent mental health problems in India. We found that the month-long digital training programme significantly increased knowledge-based competency scores, with the greatest change scores identified in the group who were randomised to receive weekly coaching sessions. Those who received coaching were three times more likely to complete the full training programme compared to participants in the self-directed learning condition. We infer that digital training is a feasible and effective strategy for building the knowledge base of non-specialists involved in initiatives to scale up the task-sharing of psychotherapies and that remotely delivered coaching can optimise learning outcomes further.

To our knowledge, this is the first RCT to investigate learning outcomes among non-specialist providers following digital training in an evidence-based youth mental health intervention in India or any other low- or middle-income country. The findings are consistent with other research on scalable models of educational delivery, which has shown that digital learning platforms can reach high numbers at relatively low cost but may struggle with engagement when used without systems for interpersonal facilitation (Dimeff et al., Reference Dimeff, Koerner, Woodcock, Beadnell, Brown, Skutch, Paves, Bazinet and Harned2009; Ehrenreich-May et al., Reference Ehrenreich-May, Dimeff, Woodcock, Queen, Kelly, Contreras, Rash, Kelley-Brimer, Hauschildt, Danner, Kennedy and Barlow2016; Rakovshik et al., Reference Rakovshik, McManus, Vazquez-Montes, Muse and Ougrin2016). A related strand of pedagogical research has shown that human interaction can significantly increase engagement with digital educational materials and give rise to better learning outcomes (Reavley et al., Reference Reavley, Morgan, Fischer, Kitchener, Bovopoulos and Jorm2018).

The overall completion rate of 54.9% should be considered in the context of a voluntary training programme where there were no incentives for participation other than a certificate of completion. Such conditions are well known to be associated with high levels of attrition in “Massive Open Online Courses” (MOOCs) and other open-access courses, where completion rates typically cluster around 5–10% (Allione and Stein, Reference Allione and Stein2016; Badali et al., Reference Badali, Hatami, Farrokhnia and Noroozi2022). Against this low benchmark, the observed completion rate appears to be relatively encouraging. A higher completion rate is conceivable under alternative conditions where motivation could be enhanced through formalised academic credit or a clear-cut trajectory from training to practical implementation/qualified practitioner status.

It is likely – and consistent with the dose–response analysis – that the greater knowledge demonstrated by participants in the coaching condition was related to more extensive engagement with the programmed content. The deployment of coaches potentially limits the scalability of digitally delivered training, not least as most existing models for coaching have utilised experts (Frank et al., Reference Frank, Becker-Haimes and Kendall2020). In contrast, the coaches in our study were non-specialist providers themselves who did not have professional qualifications or substantive training, further adding to scalability of the digital platform.

Although the pre-post changes were significant overall and the effect sizes moderate to large, in absolute terms participants were able to answer just 1 or 2 additional questions correctly after the training. Even in the coaching condition, the post-training mean score of 9.30 corresponded to approximately 7 incorrect answers out of 16 questions (43.8%). Hence, there is clearly a need for further learning support, such as supervised practice. The relatively small change in competency scores may also reflect motivational issues in the sample, given that none of our study participants were enrolled in practice-based courses or employed in practice roles that would necessarily facilitate real-world applications. Different results may have been obtained for a more selected service-oriented sample who were expecting to apply the training directly into practice.

Another limitation of our study concerns the use of a knowledge-based competency measure, rather than a measure of demonstrated skills. That said, our competency measure was validated in the study context and consisted of counselling vignettes that approximated real-life situations. This emphasis on applied knowledge (“knowing how”) rather than purely theoretical understanding strengthens ecological validity, though we accept that it cannot substitute entirely for a gold-standard observational assessment of clinical skills. For example, the observer-rated ENhancing Assessment of Common Therapeutic factors (ENACT) scale was designed for training and supervision of non-specialist providers of psychological interventions in culturally diverse and resource-constrained settings (Kohrt et al., Reference Kohrt, Jordans, Rai, Shrestha, Luitel, Ramaiya, Singla and Patel2015). However, we note that ENACT functions as a measure of common factors in psychotherapies (i.e., competencies that are implicated in the effective delivery of any psychotherapy) and does not assess competencies that are unique to problem-solving or other discrete practice elements. ENACT’s broad-based assessment of therapeutic skills, supplemented with observer-rated items covering more specific therapeutic skills, would ideally be deployed after a period of case-based practice, rather than following a didactic training of the type used in the current study. A further limitation is that we did not assess the prospective impact of training on clinical outcomes. However, other research has shown that higher post-training knowledge is associated with better mental health outcomes for treated cases (Rakovshik et al., Reference Rakovshik, McManus, Vazquez-Montes, Muse and Ougrin2016; Milligan-Saville et al., Reference Milligan-Saville, Tan, Gayed, Barnes, Madan, Dobson, Bryant, Christensen, Mykletun and Harvey2017) and knowledge could be considered as a pre-requisite for effective transfer to practice.

In conclusion, digital training is a promising strategy, especially when supplemented by remote coaching, for growing the workforce needed to deliver evidence-based psychotherapies at scale. Importantly, such trainings involve a one-time investment of expert resources in designing the curriculum after which there is a comparably much smaller cost for implementation. Thus, the shift towards automated, pre-recorded training offers a substantial scalability advantage over conventional expert-led workshops, which must be repeated in real-time to successive cohorts. Large-scale digital programmes with relatively low running costs could be used to select promising candidates for more resource-intensive further training and supervised practice.

Future research should examine how these knowledge-based competencies can be translated into actual therapy skills, for example through supervised case-based practice, and directly address questions about how to sustain training benefits over time. Research is also needed to establish the generalisability of digital training formats for other psychosocial interventions and in diverse contexts, ultimately serving to scale up task-sharing initiatives aimed at reducing the mental health care gap globally.

Open peer review

To view the open peer review materials for this article, please visit http://doi.org/10.1017/gmh.2023.81.

Supplementary material

The supplementary material for this article can be found at https://doi.org/10.1017/gmh.2023.81.

Data availability statement

Anonymised participant data, data dictionary and case report forms will be made available on datacompass.lshtm.ac.uk by 12 months after trial completion. Data will be shared after approval by the corresponding author, following a reasonable submitted request. The study protocol and statistical analysis plan are publicly available on clinicaltrials.gov.

Acknowledgements

We are grateful to members of the PRIDE Scientific Advisory Group, chaired by Prof. Chris Fairburn, and colleagues who contributed to intervention and research activities across the various phases of the study, including Shreyas Kamat, Abhija Teli, Keshav Shakya, Niket Agarwal, Rukhsar Akram, Sai Priya Kumar, Aarohi Sharma, Manogya Sahay, Meenakshi Pandey, Sarita Rao, Shouaib Ahmed and Himanshu Gupta. We also gratefully acknowledge the contributions made by the Population Foundation of India in designing the digital training course.

Author contribution

S.M., D.M. and V.P. led on the development of the digital training platform and coaching protocol, with contributions from T.S. and K.M. S.M. led on drafting of the manuscript, with critical inputs from D.M. and V.P. H.A.W., M.N., B.L. and A.P.F. drafted the statistical analysis plan. S.M., T.S., J.E.J., P.N. and R.M. contributed to the coordination and implementation of the trial. All authors read and approved the final manuscript. D.M. and V.P. contributed equally as senior authors.

Financial support

This study was supported by a Principal Research Fellowship awarded to V.P. by the Wellcome Trust (grant number 106919/A/15/Z). The funders had no role in the design of the study, data collection or the writing of the manuscript. For the purpose of open access, the author has applied a CC BY public copyright licence to any Author Accepted Manuscript version arising from this submission.

Competing interest

The authors declare no conflict of interest.

Ethics approval

Institutional Review Board approvals were obtained from Sangath (the implementing organisation in India); Harvard Medical School, USA (the sponsor) and the London School of Hygiene and Tropical Medicine, UK (a collaborating institute).

Appendix – Additional Results

In addition to the primary and secondary outcomes to be imputed (change in competency score and MUSIC subscales), the imputation model included the following complete variables: age, gender, language, education, current occupation, organization, and number of DT module completed. Imputation was conducted stratified by arm.

Table A1. Characteristics of participants who did and did not complete follow-up

Table A2. Observed competency and MUSIC outcomes at follow-up, by arm and overall

DT = Digital Training; DT-C = Digital Training and Coaching; SD = Standard Deviation

Table A3. Sensitivity analyses

DT = Digital Training; DT-C = Digital Training and Coaching

Analysis models:

  1. 1. Primary analysis: based on 16-item competency score and multiply imputed data, see main paper for full details.

  2. 2. 17-items competency score: Using the competency score including all 17 items, before exclusion of 1 item with low discrimination. Based on multiply imputed data, using same imputation model than primary analysis.

  3. 3. Complete cases: Based on 230 participants who completed the competency score at follow-up.

Table A4. Coach characteristics and Quality rating of coaching sessions

Footnotes

* DT-arm only. P-values calculated with Pearson’s chi-squared tests (arm, gender, education, language, type of organization) or Fisher’s exact tests (occupation, experience, organization, DT modules completed, coaching sessions completed) for categorical variables and t-tests for continuous variables (age, baseline competency score).

** Smallest site with 2 participants regrouped with a larger similar site.

* Mean difference between DT and DT-C arm, adjusted for organization strata and competency score at baseline.

* CQRS = Coaching Quality Rating Scale (developed for the current study). 25 assessment points, with overall score ranging from 1 (being limited) to 4 (advanced) fidelity score.

References

Allione, G and Stein, RM (2016) Mass attrition: An analysis of drop out from principles of microeconomics MOOC. Journal of Economic Education 47(2), 174186. https://doi.org/10.1080/00220485.2016.1146096.Google Scholar
Badali, M, Hatami, J, Farrokhnia, M and Noroozi, O (2022) The effects of using Merrill’s first principles of instruction on learning and satisfaction in MOOC. Innovations in Education and Teaching International 59(2), 216225. https://doi.org/10.1080/14703297.2020.1813187.Google Scholar
Dimeff, LA, Koerner, K, Woodcock, EA, Beadnell, B, Brown, MZ, Skutch, JM, Paves, AP, Bazinet, A and Harned, MS (2009) Which training method works best? A randomized controlled trial comparing three methods of training clinicians in dialectical behavior therapy skills. Behaviour Research and Therapy 47(11), 921930. https://doi.org/10.1016/j.brat.2009.07.011.Google Scholar
Ehrenreich-May, J, Dimeff, LA, Woodcock, EA, Queen, AH, Kelly, T, Contreras, IS, Rash, B, Kelley-Brimer, A, Hauschildt, J, Danner, SM, Kennedy, SM and Barlow, DH (2016) Enhancing online training in an evidence-based treatment for adolescent panic disorder: A randomized controlled trial. Evidence-Based Practice in Child and Adolescent Mental Health 1(4), 241258. https://doi.org/10.1080/23794925.2016.1245116.CrossRefGoogle Scholar
Fairburn, CG and Cooper, Z (2011) Therapist competence, therapy quality, and therapist training. Behaviour Research and Therapy 49(6–7), 373378. https://doi.org/10.1016/j.brat.2011.03.005.Google Scholar
Frank, HE, Becker-Haimes, EM and Kendall, PC (2020) Therapist training in evidence-based interventions for mental health: A systematic review of training approaches and outcomes. Clinical Psychology: Science and Practice 27(3), e12330. https://doi.org/10.1111/cpsp.12330.Google Scholar
Hoeft, TJ, Fortney, JC, Patel, V and Unützer, J (2018) Task-sharing approaches to improve mental health care in rural and other low-resource settings: A systematic review. Journal of Rural Health 34(1), 4862. https://doi.org/10.1111/jrh.12229.Google Scholar
Irby, BJ (2018) Editor’s overview: Differences and similarities with mentoring, tutoring, and coaching. Mentoring & Tutoring: Partnership in Learning 26(2), 115121.Google Scholar
Jones, BD and Skaggs, G (2016) Measuring students’ motivation: Validity evidence for the MUSIC model of academic motivation inventory. International Journal for the Scholarship of Teaching and Learning 10(1), 7. https://doi.org/10.20429/ijsotl.2016.100107.Google Scholar
Khan, A, Shrivastava, R, Tugnawat, D, Singh, A, Dimidjian, S, Patel, V, Bhan, A and Naslund, JA (2020) Design and development of a digital program for training non-specialist health workers to deliver an evidence-based psychological treatment for depression in primary Care in India. Journal of Technology in Behavioral Science 5(4), 402415. https://doi.org/10.1007/s41347-020-00154-7.Google Scholar
Kohrt, BA, Jordans, MJD, Rai, S, Shrestha, P, Luitel, NP, Ramaiya, MK, Singla, DR and Patel, V (2015) Therapist competence in global mental health: Development of the ENhancing assessment of common therapeutic factors (ENACT) rating scale. Behaviour Research and Therapy 69, 1121. https://doi.org/10.1016/j.brat.2015.03.009.Google Scholar
Malik, K, Michelson, D, Doyle, AM, Weiss, HA, Greco, G, Sahu, R, James, EJ, Mathur, S, Sudhir, P, King, M, Cuijpers, P, Chorpita, B, Fairburn, CG and Patel, V (2021) Effectiveness and costs associated with a lay counselor–delivered, brief problem-solving mental health intervention for adolescents in urban, low-income schools in India: 12-month outcomes of a randomized controlled trial. PLoS Medicine 18(9), e1003778. https://doi.org/10.1371/journal.pmed.1003778.Google Scholar
Mathur, S, Michelson, D, Shetty, T, Patel, V and Field, AP (2023a) Knowledge of problem solving (KOPS) scale: Design and evaluation of a digitally administered competence measure for a common practice element in task-shared youth mental health interventions. Journal of Technology in Behavioral Science. https://doi.org/10.1007/s41347-023-00356-9.Google Scholar
Mathur, S, Weiss, H, Neuman, M, Field, A, Leurent, B, Shetty, T, J, J, Nair, P, Mathews, R, Malik, K, Michelson, D and Patel, V (2023b) Patel VCoach-Supported Versus Self-guided Digital Training Course for a Problem-solving Psychological Intervention for Nonspecialists: Protocol for a Pre-Post Nested Randomized Controlled Trial. JMIR Research Protocols 12, e41981. https://www.researchprotocols.org/2023/1/e41981.Google Scholar
Means, B, Toyama, Y, Murphy, R, Bakia, M and Jones, K (2009) Evaluation of evidence-based practices in online learning: A meta-analysis and review of online learning studies. Available at www.ed.gov/about/offices/list/opepd/ppss/reports.html (accessed 1 November 2021).Google Scholar
Michelson, D, Malik, K, Parikh, R, Weiss, HA, Doyle, AM, Bhat, B, Sahu, R, Chilhate, B, Mathur, S, Krishna, M, Sharma, R, Sudhir, P, King, M, Cuijpers, P, Chorpita, B, Fairburn, CG and Patel, V (2020) Effectiveness of a brief lay counsellor-delivered, problem-solving intervention for adolescent mental health problems in urban, low-income schools in India: A randomised controlled trial. The Lancet Child and Adolescent Health 4(8), 571582. https://doi.org/10.1016/S2352-4642(20)30173-5.Google Scholar
Miller, GE (1990) The assessment of clinical skills/competence/performance. Academic Medicine 65(9), S63S67.Google Scholar
Milligan-Saville, JS, Tan, L, Gayed, A, Barnes, C, Madan, I, Dobson, M, Bryant, RA, Christensen, H, Mykletun, A and Harvey, SB (2017) Workplace mental health training for managers and its effect on sick leave in employees: A cluster randomised controlled trial. The Lancet Psychiatry 4(11), 850858. https://doi.org/10.1016/S2215-0366(17)30372-3.Google Scholar
Muke, SS, Tugnawat, D, Joshi, U, Anand, A, Khan, A, Shrivastava, R, Singh, A, Restivo, JL, Bhan, A, Patel, V and Naslund, JA (2020) Digital training for non-specialist health workers to deliver a brief psychological treatment for depression in primary care in India: Findings from a randomized pilot study. International Journal of Environmental Research and Public Health 17(17), 122. https://doi.org/10.3390/ijerph17176368.Google Scholar
Naslund, JA, Gonsalves, PP, Gruebner, O, Pendse, SR, Smith, SL, Sharma, A and Raviola, G (2019a) Digital innovations for global mental health: Opportunities for data science, task sharing, and early intervention. Current Treatment Options in Psychiatry 6(4), 337351. https://doi.org/10.1007/s40501-019-00186-8.Google Scholar
Naslund, JA, Shidhaye, R and Patel, V (2019b) Digital technology for building capacity of nonspecialist health workers for task sharing and scaling up mental health care globally. Harvard Review of Psychiatry 27(3), 181192. https://doi.org/10.1097/HRP.0000000000000217.CrossRefGoogle ScholarPubMed
Naslund, JA, Tugnawat, D, Anand, A, Cooper, Z, Dimidjian, S, Fairburn, CG, Hollon, SD, Joshi, U, Khan, A, Lu, C, Mitchell, LM, Muke, S, Nadkarni, A, Ramaswamy, R, Restivo, JL, Shrivastava, R, Singh, A, Singla, DR, Spiegelman, D, Bhan, A and Patel, V (2021) Digital training for non-specialist health workers to deliver a brief psychological treatment for depression in India: Protocol for a three-arm randomized controlled trial. Contemporary Clinical Trials 102, 106267. https://doi.org/10.1016/j.cct.2021.106267.Google Scholar
Philippe, TJ, Sikder, N, Jackson, A, Koblanski, ME, Liow, E, Pilarinos, A and Vasarhelyi, K (2022) Digital health interventions for delivery of mental health care: Systematic and comprehensive meta-review. JMIR Mental Health 9(5), e35159. https://doi.org/10.2196/35159.Google Scholar
PRIDE, Sangath, & New Delhi (2022) PRIDE problem-solving counselling manual. Available at https://sangath.in/pride-problem-solving-counselling-manual/ (accessed 1 October 2022).Google Scholar
Rakovshik, SG, McManus, F, Vazquez-Montes, M, Muse, K and Ougrin, D (2016) Is supervision necessary? Examining the effects of internet-based CBT training with and without supervision. Journal of Consulting and Clinical Psychology 84(3), 191199. https://doi.org/10.1037/ccp0000079.Google Scholar
Raviola, G, Naslund, JA, Smith, SL and Patel, V (2019) Innovative models in mental health delivery systems: Task sharing care with non-specialist providers to close the mental health treatment gap. Current Psychiatry Reports 21(6), 44. https://doi.org/10.1007/s11920-019-1028-x.Google Scholar
Reavley, NJ, Morgan, AJ, Fischer, JA, Kitchener, B, Bovopoulos, N and Jorm, AF (2018) Effectiveness of eLearning and blended modes of delivery of mental health first aid training in the workplace: Randomised controlled trial. BMC Psychiatry 18(1), 312. https://doi.org/10.1186/s12888-018-1888-3.Google Scholar
Roy, K, Shinde, S, Sarkar, BK, Malik, K, Parikh, R and Patel, V (2019) India’s response to adolescent mental health: A policy review and stakeholder analysis. Social Psychiatry and Psychiatric Epidemiology 54(4), 405414. https://doi.org/10.1007/s00127-018-1647-2.Google Scholar
Singla, DR, Kohrt, BA, Murray, LK, Anand, A, Chorpita, BF and Patel, V (2017) Psychological treatments for the world: Lessons from low- and middle-income countries. Annual Review of Clinical Psychology 13, 149181. https://doi.org/10.1146/annurev-clinpsy-032816-045217.Google Scholar
van Ginneken, N, Chin, WY, Lim, YC, Ussif, A, Singh, R, Shahmalak, U, Purgato, M, Rojas-García, A, Uphoff, E, McMullen, S, Foss, HS, Thapa Pachya, A, Rashidian, L, Borghesani, A, Henschke, N, Chong, LY and Lewin, S (2021) Primary-level worker interventions for the care of people living with mental disorders and distress in low- and middle-income countries. Cochrane Database of Systematic Reviews 2021(8), CD009149. https://doi.org/10.1002/14651858.CD009149.pub3.Google Scholar
Figure 0

Figure 1. Trial flow chart.

Figure 1

Table 1. Baseline characteristics of study participants by arm and combined

Figure 2

Table 2. Change in competency score for participants by number of completed modules and coaching sessions provided

Figure 3

Table 3. Comparison of outcomes at follow-up between DT and DT-C arm (n = 277, imputed data)a

Figure 4

Table 4. Effect-modification of change in competency score and in coaching effectiveness by pre-specified sub-groups

Figure 5

Table A1. Characteristics of participants who did and did not complete follow-up

Figure 6

Table A2. Observed competency and MUSIC outcomes at follow-up, by arm and overall

Figure 7

Table A3. Sensitivity analyses

Figure 8

Table A4. Coach characteristics and Quality rating of coaching sessions

Supplementary material: File

Mathur et al. supplementary material
Download undefined(File)
File 20 KB

Author comment: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R0/PR1

Comments

Prof Judy Bass and Dr Dixon Chibanda

Editor-in-Chief, Global Mental Health

7 June 2023

Dear Prof Bass and Dr Chibanda,

RE: Manuscript submission - “Developing psychotherapeutic competencies in non-specialist providers: a pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India”

I am pleased to submit the above-named manuscript for consideration as an original research paper in Global Mental Health. I can confirm the manuscript is not under review with any other journal and we have no conflicts of interest. The trial has been pre-registered at clinicaltrials.gov (NCT05290142).

The submitted paper originates from a larger six-year programme of research called PRIDE (PRemIum for aDolEscents), which aims to develop and scale up a suite of school-based, transdiagnostic psychosocial interventions for adolescents with common mental health problems in India. Previous PRIDE studies, including a mixed-methods evaluation study published in Global Mental Health (link here), have produced a raft of evidence on the performance of linked intervention components organised around a stepped care architecture.

The submitted research paper reports the findings of a complementary evaluation of an innovative digital training course, which is intended to up-skill prospective providers for PRIDE’s first-line problem-solving intervention. The study incorporated a pre-and-post study design with a nested RCT, evaluating digital training delivered in a self-guided format or with coaching. This is the first randomised controlled trial to investigate learning outcomes among non-specialist providers following digital training in an evidence-based youth mental health intervention. We found that digital training significantly increased non-specialists’ competency to deliver the problem-solving intervention, especially when aided by remote coaching which also led to higher engagement in the training. This shift towards automated, pre-recorded training offers a substantial scalability advantage over conventional expert-led workshops is a promising and relatively low running cost alternative.

These findings will be used to support wider efforts to scale up evidence-based mental health interventions for young people. We expect that the paper will be of significant interest to researchers and practitioners in the field of community (youth) mental health, and particularly to those working in low-resource settings.

Thank you in advance for your consideration.

Yours sincerely,

Vikram Patel

The Pershing Square Professor of Global Health and Wellcome Trust Principal Research Fellow, Harvard Medical School

Professor, Harvard T.H. Chan School of Public Health

E: vikram:patel@hms.harvard.edu

T: +91-9822132038

Review: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R0/PR2

Conflict of interest statement

Reviewer declares none.

Comments

Thank you for the invitation to review this interesting manuscript describing an innovative trial aimed at evaluating and comparing a coach-supported and self-guided digital training course for a problem-solving psychological intervention in India. The study methods appear rigorous, and the findings are novel, demonstrating the potential effectiveness of digital training methods in increasing participant knowledge, and the added impact of the coaching on participant engagement, training completion and outcomes. See few suggestions for consideration:

First, additional details are needed about the KOPS primary outcome measure of competency, including how this measure captures clinical competencies or skills, and how it was validated. Given that this is a written self-administered test, it seems like it would be best described as a knowledge assessment. The limitations with the measure are acknowledged in the discussion, though the discussion should be expanded to weigh the pros/cons of such a measure with existing gold-standard approaches for competency assessment (i.e., such as the ENACT scale and ratings).

Second, additional details are needed about the nature of the problem-solving therapy covered in the 2 training programs, and evidence of its effectiveness. The program manual is cited, but clinical impact of this therapy should be summarized in the text. The study also mentions that participants could be Hindi or English speaking. Clarification is needed about the language of the digital training, assuming it was available in both languages.

Third, it is not clear how to interpret the differences in scores in the MUSIC model questionnaire between groups. It does not appear that this questionnaire has been previously used for comparing groups in digital training studies. Some additional details and description of the questionnaire and how to interpret the scores would be helpful.

Fourth, the low engagement among participants across both conditions requires greater attention and reflection in the discussion. While it is impressive that the addition of the coaching substantially improved engagement and outcomes, there are still a large number of participants who did not complete the training, or did not start the training at all. It would be helpful to consider how this represents a challenge for informing the scale up of the training and task-sharing efforts more broadly, and also consider what strategies could be employed to potentially overcome low engagement among target learner groups.

Lastly, the findings are currently overstated, because it is not possible to conclude that digital training was ‘effective’ for growing the workforce or scaling up the task-sharing or psychotherapies, unless there is data on whether any of the participants trained in this study went on to deliver care? Instead, recommend rewording throughout that this is a ‘potentially effective’ strategy, or promising strategy, and that the effectiveness is demonstrated in increasing knowledge (rather than scaling up psychotherapies).

Review: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R0/PR3

Conflict of interest statement

Reviewer declares none.

Comments

The study design is excellent; however the conclusions drawn are not supported by the study. As noted in the limitations, this relates to knowledge and knowledge acquisition pre- and post- course, which is not the same as the transfer of learning to practice. Thus, the conclusions linking this educational design to enhancing access to psychotherapy and task-sharing are not supported by the study.

Early in the introduction, there is reference to knowledge-based competency, and the improvement of competency scores. Whilst there are many, many definitions of competency around, these commonly relate to practice and performance, and the integration of knowledge alongside skills, attitudes and behaviours, to practice - rather than the component knowledge, skills, attitudes or behaviours. Here are a couple of key papers that you may wish to refer to: https://human-resources-health.biomedcentral.com/articles/10.1186/s12960-019-0443-8 and https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4054715/. A useful and commonly cited reference when categorizing learning outcomes is Miller’s pyramid. My interpretation of this study is that it targeted knows and knows how, only’; and thus does not address competency development (shows how and does levels). Unless the study is extended to assess competencies and transfer to practice, my first recommendation is that this paper refers only to the increase in knowledge and knowledge retention as measured pre-test and post-test, not competency. At this stage you may wish to cite literature that links knowledge and knows-how with transfer to practice – but competence (performance) was not targeted during the learning programme or the coaching.

The approach to coaching also seemed to be outside of my interpretation of coaching, which would relate more to practice experience and transfer from learning to practice; instead it resembled tutoring. Thus the comparison is between self-paced study, and self-paced study with access to a course tutor. I also query the quality of training and support provided to the ‘coaches’ – the manuscript refers to the quality of coaching being scored, but there is no reference to providing feedback or support to the coaches. If this was part of the design, this would be important to mention. My second recommendation thus relates to the language used to describe the role of the ‘coach’.

My third reflection may not be relevant if the first recommendation is taken into account, and relates to the discussion around task-sharing. The introduction describes task-sharing in a way that may be misinterpreted to mean that tasks and healthcare services that are traditionally part of regulated scopes of practice may now be provided by lay people after completing a knowledge-based short course. I’m sure that is not the intention of the authors, and the misperception that this is aiming to ‘dumb down’ service provision could be prevented if for example, example tasks are provided, or the discussion relates to role optimization, and the steps taken to ensure that the quality of services received is not affected by the distribution of tasks amongst the team. For example, one argument may be that through enhancing the training in psychotherapies of lay people and community health workers, a first level access to psychotherapy and initial support can be provided which increases access to trained health workers, who can then refer for more specialized care where needed.

There is merit in revising the manuscript – it is an interesting finding that self-paced learning can be enhanced with the support of a tutor. However, the conclusions should reflect what is being assessed and the increase in knowledge, with perhaps recommendations for supported/coached transfer to practice. In relation to the task-sharing discussion, it may be that the conclusions find that certain tasks can be learned through self-paced learning, rather than the blanket recommendation that self-paced supports task-sharing, but instead to reflect on self-paced plus a tutor as a more effective learning intervention than self-paced learning alone.

Recommendation: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R0/PR4

Comments

No accompanying comment.

Decision: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R0/PR5

Comments

No accompanying comment.

Author comment: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R1/PR6

Comments

No accompanying comment.

Review: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R1/PR7

Conflict of interest statement

Reviewer declares none.

Comments

Thank you for resubmitting this revised manuscript. The authors have thoughtfully responded to my comments, and the revised manuscript is significantly improved. This is an interesting study and important contribution to the global mental health literature on efforts to train the non-specialist workforce. I have no further comments at this time.

Recommendation: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R1/PR8

Comments

Thank you for submitting the revisions, they have clarified the reviewers concerns. To finalize the manuscript, we request a few additional minor revisions:

1. Update the title to match better reflect the knowledge competences focus of the intervention (rather than a digital training course for a problem solving intervention).

2. Update the abstract, including the initial sentence, to similarly reflect that the focus was not on training of the actual problem solving intervention as well as edits that reflect the other adjustments made to reviewer comments (line 12 - specifying knowledge competencies).

3. Update the impact statement as the digital training was not on the evidence-based youth mental health intervention (problem-solving therapy) but rather than knowledge content.

4. Update the keywords based on changes to the manuscript.

Decision: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R1/PR9

Comments

No accompanying comment.

Author comment: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R2/PR10

Comments

14th November 2023

Dear Prof, Bass,

Re: Revised manuscript, “Developing knowledge-based psychotherapeutic competencies in non-specialist providers: a pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India”

We are pleased by the positive response to our first round of revisions. We have made further minor changes as requested.

In so doing, we have responded to each of your comments in turn.

We trust that these revisions prove satisfactory and look forward to your response in due course.

Yours sincerely,

Prof. Vikram Patel

1.Update the title to match better reflect the knowledge competences focus of the intervention (rather than a digital training course for a problem solving intervention).

Title has been updated to the following,

“Developing knowledge-based psychotherapeutic competencies in non-specialist providers: a pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India”

2. Update the abstract, including the initial sentence, to similarly reflect that the focus was not on training of the actual problem solving intervention as well as edits that reflect the other adjustments made to reviewer comments (line 12 - specifying knowledge competencies).

The first line in the abstract has been modified and reproduced below for ease of reference,

“We evaluated a digital learning programme for non-specialists, with the goal to develop knowledge-based competencies related to a problem-solving for adolescents.”

3. Update the impact statement as the digital training was not on the evidence-based youth mental health intervention (problem-solving therapy) but rather than knowledge content.

The first line in the impact statement has been modified and reproduced below for ease of reference,

“This randomised controlled trial investigates knowledge-based learning outcomes among non-specialist providers following digital training on an evidence-based youth mental health intervention (problem-solving therapy).”

4. Update the keywords based on changes to the manuscript.

Keywords have been updated to the following,

“Randomised controlled trial; knowledge-based competency; digital training; capacity building; problem-solving intervention; adolescent mental health; India.”

Recommendation: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R2/PR11

Comments

No accompanying comment.

Decision: Developing knowledge-based psychotherapeutic competencies in non-specialist providers: A pre-post study with a nested randomised controlled trial of a coach-supported versus self-guided digital training course for a problem-solving psychological intervention in India — R2/PR12

Comments

No accompanying comment.