Hostname: page-component-586b7cd67f-dsjbd Total loading time: 0 Render date: 2024-12-06T04:50:59.375Z Has data issue: false hasContentIssue false

Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya

Published online by Cambridge University Press:  23 May 2023

Noah S. Triplett
Affiliation:
Department of Psychology, University of Washington, Seattle, WA, USA
Anne Mbwayo
Affiliation:
Department of Psychiatry, University of Nairobi, Nairobi, Kenya
Sharon Kiche
Affiliation:
Department of Psychology, University of Washington, Seattle, WA, USA
Lucy Liu
Affiliation:
Department of Psychology, University of Washington, Seattle, WA, USA
Jacinto Silva
Affiliation:
Department of Psychology, University of Washington, Seattle, WA, USA
Rashed AlRasheed
Affiliation:
Department of Psychology, University of Washington, Seattle, WA, USA
Clara Johnson
Affiliation:
Department of Psychology, University of Washington, Seattle, WA, USA
Cyrilla Amanya
Affiliation:
Research Department, Ace Africa Kenya, Bungoma, Kenya
Sean Munson
Affiliation:
Department of Human Centered Design & Engineering, University of Washington, Seattle, WA, USA
Bryan J. Weiner
Affiliation:
Department of Global Health, University of Washington, Seattle, WA, USA Department of Health Services, School of Public Health, University of Washington, Seattle, WA, USA
Pamela Y. Collins
Affiliation:
Department of Health Services, School of Public Health, University of Washington, Seattle, WA, USA Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, USA
Shannon Dorsey
Affiliation:
Department of Psychology, University of Washington, Seattle, WA, USA
Rights & Permissions [Opens in a new window]

Abstract

Opportunities exist to leverage mobile phones to replace or supplement in-person supervision of lay counselors. However, contextual variables, such as network connectivity and provider preferences, must be considered. Using an iterative and mixed methods approach, we co-developed implementation guidelines to support the implementation of mobile phone supervision with lay counselors and supervisors delivering a culturally adapted trauma-focused cognitive behavioral therapy in Western Kenya. Guidelines were shared and discussed with lay counselors in educational outreach visits led by supervisors. We evaluated the impact of guidelines and outreach on the acceptability, feasibility, and usability of mobile phone supervision. Guidelines were associated with significant improvements in acceptability and usability of mobile phone supervision. There was no evidence of a significant difference in feasibility. Qualitative interviews with lay counselors and supervisors contextualized how guidelines impacted acceptability and feasibility – by setting expectations for mobile phone supervision, emphasizing importance, increasing comfort, and sharing strategies to improve mobile phone supervision. Introducing and discussing co-developed implementation guidelines significantly improved the acceptability and usability of mobile phone supervision. This approach may provide a flexible and scalable model to address challenges with implementing evidence-based practices and implementation strategies in lower-resourced areas.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press

Impact statement

Task-shifting, in which lay counselors without formal mental health training are trained and supported to deliver mental health interventions, is an acceptable and effective way of delivering mental health treatment. However, there are challenges with ensuring the feasibility and sustainability of supervision in task-shifting, particularly in areas with fewer numbers of trained supervisors. Mobile phones may present an opportunity to support lay counselors from afar, but special attention must be paid to ensure that lay counselors are being appropriately supported via mobile phones. Additionally, given the vast differences between mobile phone access, cellular network connection, and other contextual variables in many low-resource areas, approaches to support mobile phone supervision must be flexible and encourage tailoring to specific contexts. The current research describes the co-development of a flexible and pragmatic approach to support mobile phone supervision with lay counselors in Western Kenya. Lay counselors and supervisors delivering a culturally adapted trauma-focused cognitive behavioral therapy participated in a co-design process to design an approach to support mobile phone supervision. The co-design process resulted in the development of implementation guidelines, which were to be shared and discussed in a brief educational outreach visit between supervisors and lay counselors. We compared the effects of the guidelines and visit on the acceptability, feasibility, and usability of mobile phone supervision between lay counselors who did and did not receive guidelines. The introduction and discussion of guidelines was associated with improvements in the acceptability and usability of mobile phone supervision. In comparing our approach and benefits to other implementation strategies, we highlight the importance of scalability, flexibility, and simplicity in our approach. We also note the importance of valuing the expertise of lay counselors and supervisors to select and tailor implementation strategies.

Background

Though there are mental health treatment gaps in nearly every country, there are larger treatment gaps in low-to-middle-income countries (LMICs), compared to high income countries, due to fewer trained mental health care providers (Demyttenaere et al., Reference Demyttenaere, Bruffaerts, Posada-Villa, Gasquet, Kovess, Lepine, Angermeyer, Bernert, de Girolamo, Morosini, Polidori, Kikkawa, Kawakami, Ono, Takeshima, Uda, Karam, Fayyad, Karam, Mneimneh, Medina-Mora, Borges, Lara, de Graaf, Ormel, Gureje, Shen, Huang, Zhang, Alonso, Haro, Vilagut, Bromet, Gluzman, Webb, Kessler, Merikangas, Anthony, von Korff, Wang, Brugha, Aguilar-Gaxiola, Lee, Heeringa, Pennell, Zaslavsky, Ustun and Chatterji2004; Kohn et al., Reference Kohn, Saxena, Levav and Saraceno2004; Moitra et al., Reference Moitra, Santomauro, Collins, Vos, Whiteford, Saxena and Ferrari2022). Task-shifting has emerged as a potential strategy to address the human resource shortages that, in part, contribute to the mental health treatment gap (Murray et al., Reference Murray, Dorsey, Haroz, Lee, Alsiary, Haydary, Weiss and Bolton2014; Weiss et al., Reference Weiss, Murray, Zangana, Mahmooth, Kaysen, Dorsey, Lindgren, Gross, Murray, Bass and Bolton2015; Chibanda et al., Reference Chibanda, Weiss, Verhey, Simms, Munjoma, Rusakaniko, Chingono, Munetsi, Bere, Manda, Abas and Araya2016; Hoeft et al., Reference Hoeft, Fortney, Patel and Unützer2018). As part of task-shifting models, lay counselors (e.g., teachers and health workers) without formal mental health training are trained and supported to deliver interventions (van Ginneken et al., Reference van Ginneken, Tharyan, Lewin, Rao, Meera, Pian, Chandrashekar and Patel2013). Evidence supports the effectiveness of task-shifting to deliver evidence-based practices (EBPs) for mental, neurological, and substance use disorders in LMIC (van Ginneken et al., Reference van Ginneken, Tharyan, Lewin, Rao, Meera, Pian, Chandrashekar and Patel2013; Murray et al., Reference Murray, Dorsey, Haroz, Lee, Alsiary, Haydary, Weiss and Bolton2014; Weiss et al., Reference Weiss, Murray, Zangana, Mahmooth, Kaysen, Dorsey, Lindgren, Gross, Murray, Bass and Bolton2015; Chibanda et al., Reference Chibanda, Weiss, Verhey, Simms, Munjoma, Rusakaniko, Chingono, Munetsi, Bere, Manda, Abas and Araya2016; Hoeft et al., Reference Hoeft, Fortney, Patel and Unützer2018; Dorsey et al., Reference Dorsey, Lucid, Martin, King, O’Donnell, Murray, Wasonga, Itemba, Cohen, Manongi and Whetten2020b). As task-shifting continues to expand, more research is needed to understand how to fully embed and sustain task-shifting in LMIC, including how to sustainably supervise and support lay counselors (Padmanathan and De Silva, Reference Padmanathan and De Silva2013).

A growing body of literature has examined how technology can be used to build capacity and scale-up task-shifting globally (Naslund et al., Reference Naslund, Shidhaye and Patel2019), including as a tool to support supervision during in-person meetings (Rahman et al., Reference Rahman, Akhtar, Hamdani, Atif, Nazir, Uddin, Nisar, Huma, Maselko, Sikander and Zafar2019). In particular, mobile phones may offer a cost-effective and accessible means of replacing or supplementing in-person supervision during task-shifting; however, it is important to consider and address the various challenges and nuances that may accompany mobile phone supervision in LMIC (Triplett et al., Reference Triplett, Johnson, Kiche, Dastrup, Nguyen, Daniels, Mbwayo, Amanya, Munson, Collins, Weiner and Dorsey2023). Though less research has examined mobile phone and digital technology use in task-shifting, a large body of research has examined the challenges and opportunities of mobile phone use in LMIC (Kusimba et al., Reference Kusimba, Yang and Chawla2015; Murphy and Priebe, Reference Murphy and Priebe2017; Wyche and Olson, Reference Wyche and Olson2018). Research has noted the importance of considering the limitations of mobile phones when designing and implementing digital technologies in LMIC (Wyche and Murphy, Reference Wyche and Murphy2012), including digital supports to healthcare in LMICs (Henry et al., Reference Henry, Winters, Lakati, Oliver, Geniets, Mbae and Wanjiru2016; Feroz et al., Reference Feroz, Jabeen and Saleem2020).

Given the challenges with implementing digital technologies in LMIC, flexible and adaptable implementation strategies are needed. Guidelines that contain recommendations to guide mobile phone supervision may be one pragmatic strategy; however, specific attention must be paid to ensure guidelines are flexible, account for contextual differences, and are appropriately introduced to clinicians. Guidelines are frequently used to support implementation of healthcare innovations, including within LMIC (Francke et al., Reference Francke, Smit, De Veer and Mistiaen2008; Nabyonga Orem et al., Reference Nabyonga Orem, Bataringaya Wavamunno, Bakeera and Criel2012). Despite this, there are noted challenges with dissemination of guidelines, such as guidelines being too difficult to understand, challenges implementing without support from peers and managers, and insufficient staff and time to implement guidelines (Francke et al., Reference Francke, Smit, De Veer and Mistiaen2008). These challenges may be intensified in lower-resource settings, such as LMIC (Nabyonga Orem et al., Reference Nabyonga Orem, Bataringaya Wavamunno, Bakeera and Criel2012). There is evidence to suggest that, with proper planning, stakeholder engagement and dissemination, guideline implementation can result in desired impacts on clinician behavior (Peters et al., Reference Peters, Sukumar, Blanchard, Ramasamy, Malinowski, Ginex, Senerth, Corremans, Munn, Kredo, Remon, Ngeh, Kalman, Alhabib, Amer and Gagliardi2022). Thus, if guidelines are designed to allow for flexibility and tailoring and are combined with other minimally intensive implementation strategies, such as educational outreach visits to discuss and plan for their implementation, they may be an appropriate and resource-efficient means of supporting mobile phone supervision.

The present article outlines the results of an iterative, mixed methods study that evaluated the effects of sharing co-developed implementation guidelines in a brief educational outreach visit on the acceptability, feasibility, and usability of mobile phone supervision for lay counselors in Kenya. Guided by human-centered design (HCD), we engaged supervisors and lay counselors to co-develop the implementation guidelines and determine the most feasible way of discussing and disseminating the guidelines – educational outreach visits. Leveraging an ongoing stepped wedge trial evaluating the implementation of trauma-focused cognitive behavioral therapy (TF-CBT; see Dorsey et al., Reference Dorsey, Gray, Wasonga, Amanya, Weiner, Belden, Martin, Meza, Weinhold, Soi, Murray, Lucid, Turner, Mildon and Whetten2020a), we compared effects of guidelines and outreach across quantitative measures of acceptability, feasibility, and usability. Results were contextualized with semi-structured interviews to explain the benefits of guidelines and outreach as well as how they might have impacted outcomes.

Methods

Study design

This study used an iterative and mixed methods approach, with qualitative interviews and a HCD workshop informing the development of the implementation guidelines and outreach strategy, which was subsequently tested in a randomized pilot trial. Full trial procedures are described elsewhere (Triplett et al., Reference Triplett, Munson, Mbwayo, Mutavi, Weiner, Collins, Amanya and Dorsey2021). The trial was situated within a larger trial that examined the effectiveness and implementation of a locally adapted version of TF-CBT (Cohen et al., Reference Cohen, Mannarino and Deblinger2006) in Bungoma, Kenya (Building and Sustaining Interventions for Children [BASIC]; see Dorsey et al., Reference Dorsey, Gray, Wasonga, Amanya, Weiner, Belden, Martin, Meza, Weinhold, Soi, Murray, Lucid, Turner, Mildon and Whetten2020a for parent trial protocol). BASIC utilizes an eight-session version of TF-CBT (“Pamoja Tunaweza”), which was adapted by longstanding Kenyan partners at Ace Africa (i.e., supervisors and counselors) for cultural relevance and acceptability. Lay counselors work together in groups of three to provide the treatment in a group-based format and are trained and supervised by three Kenya-based supervisors. Lay counselors work within two governmental sectors in Kenya, identified as potentially viable systems for scale-up – Education (via teacher delivery) and Health (via community health volunteer [CHV] delivery). Supervisors are Ace Africa employees who were previously trained and subsequently delivered the treatment in a randomized controlled trial (Dorsey et al., Reference Dorsey, Lucid, Martin, King, O’Donnell, Murray, Wasonga, Itemba, Cohen, Manongi and Whetten2020b) that preceded the current trial.

To develop and evaluate the guidelines and outreach, we made use of the parent study’s stepped-wedge cluster-randomized design. BASIC was comprised of seven sequences, in which a total of 40 schools and 40 communities surrounding the schools were randomized to begin TF-CBT implementation at seven different time points throughout the study. Counselors from sequences one through five of the BASIC trial were randomly selected to participate in qualitative interviews regarding their experiences using mobile phones for clinical supervision, which included strategies to improve use (Triplett et al., Reference Triplett, Johnson, Kiche, Dastrup, Nguyen, Daniels, Mbwayo, Amanya, Munson, Collins, Weiner and Dorsey2023). All BASIC supervisors also participated in interviews regarding their experiences using mobile phones to provide supervision.

Following interviews and thematic analysis, BASIC trial supervisors presented back results from the semi-structured interviews and led discussion with lay counselors on any additions, edits, or clarifications they felt were needed for the themes. This workshop also included other HCD activities, which aimed to engage participants and determine how interview findings could be translated into actionable solutions for counselors for the implementation guidelines. Following the workshop, the supervisors and principal investigator of the project (NST) reviewed all solutions and compiled them into a one-page document for supervisors and lay counselors to review together (the implementation guidelines). The co-development process occurred while sequence six of the BASIC trial was implementing TF-CBT, and they served as a no-guidelines control group. After the guidelines were developed, they were implemented with all counselors in sequence seven. This process is depicted in Figure 1. The Institutional Review Boards (IRB) at the University of Washington and Kenya Medical Research Institute approved all study procedures.

Figure 1. Trial sequence and activities.

Implementation guidelines

The guidelines are presented in Supplementary Material S1. One example goal and corresponding strategies are also presented in Table 1. As described above, the guidelines included multiple strategies that were clustered under the four goals: A) Explain to counselors that they will use in-person and mobile phone supervision together to ensure counselors are supported through delivery. B) Ensure counselors get all the information and support that they need through mobile phone supervision. C) Plan for any challenges with network connection. D) Decrease distractions and disruptions during mobile phone supervision. All strategies were presented as optional, and counselors were encouraged to select and attempt strategies that they felt best matched their unique school and community contexts. Counselors were not required to implement any solutions, and there was no expectation of continued follow-up from supervisors.

Table 1. Sample implementation guidelines for Goal A

Participants

Participants included lay counselors recently trained in and who subsequently delivered TF-CBT as part of the parent trial (N = 59; 29 teachers; 30 CHVs). All parent trial supervisors (N = 3) also participated in semi-structured interviews. Lay counselors and supervisors were delivering TF-CBT as part of sequences six and seven of the BASIC parent trial. To be eligible to participate in the BASIC trial, lay counselors had to be nominated from leaders at their site (i.e., head teacher and/or deputy teacher, or community health extension worker). Nominations were based on criteria to identify the most appropriate counselors (e.g., desire to be counselors; experience working with children). There were no exclusion criteria. All participants provided informed consent at the time of enrollment.

Procedure

We compared acceptability, feasibility, and usability measures of mobile phone supervision from counselors in sequences six and seven of the parent trial. Lay counselors in sequence six (n = 29) did not receive any additional support or guidelines for mobile phone supervision. Lay counselors in sequence seven (n = 30) received the implementation guidelines document. Supervisors reviewed and discussed the guidelines with sequence seven lay counselors in an educational outreach visit before they began TF-CBT delivery. These visits were integrated into the first in-person supervision meeting for the parent trial, which occurred shortly following training and before lay counselors began TF-CBT groups. The addition of mobile phone supervision outreach added approximately 30 min to the initial in-person supervision meeting. Supervisors traveled to lay counselors’ sites and facilitated separate meetings with each group of lay counselors (i.e., the meetings included the supervisor and the three counselors who co-delivered the treatment in each site). Some phone contact occurred prior to this meeting to plan a meeting time, then phone supervision continued following this meeting.

Per the parent trial protocol, in-person supervision happened at least four times during each TF-CBT group (8 weeks). Through both sequences, supervisors were also available to conduct additional in-person supervision as needed with lay counselors. The manipulation was solely the introduction of guidelines and an educational outreach visit to support successful supervision by mobile phone. Lay counselors in both sequences completed measures of mobile phone supervision acceptability and feasibility, as well as a measure of usability, after delivering two rounds of TF-CBT. All three supervisors and a randomly selected sub-sample of lay counselors from sequence seven (N = 12; six teachers; six CHVs) also participated in qualitative interviews to gather more information on their experience with the guidelines and perspectives on mobile phone supervision. The final interview guide is included in Supplementary Material S2.

Measures

Measures were adapted from existing measures, prioritizing acceptability and feasibility measures already translated and used cross-culturally in the parent trial (i.e., BASIC) and other studies globally. All adaptations to the usability measure were made following established procedures to ensure common understanding of the construct (Dorsey et al., Reference Dorsey, Gray, Wasonga, Amanya, Weiner, Belden, Martin, Meza, Weinhold, Soi, Murray, Lucid, Turner, Mildon and Whetten2020a) and completed in consultation with longstanding Kenyan partners on the trial.

Acceptability

The four-item Acceptability of Intervention measure (Weiner et al., Reference Weiner, Lewis, Stanick, Powell, Dorsey, Clary, Boynton and Halko2017) was adapted and used to assess lay counselor perspectives of mobile phone supervision acceptability (e.g., “I like mobile phone supervision”). Scores range from 1 to 5, with higher scores representing greater acceptability. This brief, pragmatic measure has acceptable internal consistency and test–retest reliability in other samples (Weiner et al., Reference Weiner, Lewis, Stanick, Powell, Dorsey, Clary, Boynton and Halko2017), and had good internal consistency in our sample (α = 0.89).

Feasibility

The four-item Feasibility of Intervention measure (Weiner et al., Reference Weiner, Lewis, Stanick, Powell, Dorsey, Clary, Boynton and Halko2017) was used to assess lay counselor perspectives of mobile phone supervision feasibility (e.g., “mobile phone supervision seems doable in this school/community”). Scores range from 1 to 5, with higher scores representing greater feasibility. This measure has acceptable internal consistency and test–retest reliability in other samples (Weiner et al., Reference Weiner, Lewis, Stanick, Powell, Dorsey, Clary, Boynton and Halko2017), and had excellent internal consistency in our sample (α = 0.93).

Usability

The 10-item Intervention Usability Scale (IUS; Lyon et al., Reference Lyon, Pullmann, Jacobson, Osterhage, Achkar, Renn, Munson and Areán2020) was used to assess lay counselor perspectives of mobile phone supervision usability (e.g., “mobile phone supervision was easy to use”). Scores range from 0 to 100, with higher scores representing greater usability. The IUS has acceptable internal consistency in other samples (Lyon et al., Reference Lyon, Pullmann, Jacobson, Osterhage, Achkar, Renn, Munson and Areán2020), and had good internal consistency in our sample (α = 0.87).

Analysis

We present descriptive statistics (mean, standard deviation, and range) to understand counselor and supervisor ratings of acceptability, feasibility, and usability following mobile phone supervision. We also conducted independent samples t-tests to compare average ratings of acceptability, feasibility, and usability across sequences that did and did not receive the guidelines and educational outreach visit. Following best practices for smaller sample sizes (Weissgerber et al., Reference Weissgerber, Garovic, Savic, Winham and Milic2016), quantitative data are also visualized to illustrate any outliers and differences between sequences. All quantitative analyses were conducted using R (R Core Team, 2022).

Recordings from interviews were transcribed and identifying information was removed. Transcripts were coded in Dedoose (QSR International Pty Ltd., 2018) by researchers in the US and Kenya. Analysis was informed by thematic analysis (Braun and Clarke, Reference Braun and Clarke2006). Kiswahili interviews were translated by native Kiswahili speakers and trained translators. To develop an initial codebook, coders reviewed three transcripts independently, then met to identify potential codes and produce an initial codebook. This codebook was subsequently applied and refined on the remaining interviews. All coding was done independently, and consensus was reached through group dialog (Hill et al., Reference Hill, Thompson and Williams1997). We followed a QUAN → qual mixed methods approach for data explanation, using the embedded qualitative data to elaborate on or contextualize quantitative results (Palinkas et al., Reference Palinkas, Aarons, Horwitz, Chamberlain, Hurlburt and Landsverk2011).

Results

Sample demographics and quantitative results

Our sample included 29 teachers and 30 CHVs who delivered TF-CBT in sequences six and seven of the BASIC trial. Demographics are presented in Table 2. Differences in mobile phone supervision acceptability, feasibility, and usability between sequence six and sequence seven are presented in Table 3 and Figure 2. As shown, there were statistically significant differences in acceptability and usability between counselors who did receive guidelines and those who did not. Given potential conceptual overlap between quantitative measures, we also present Pearson correlation coefficients between outcome measures in Supplementary Material S3.

Table 2. Demographics and baseline characteristics

a n = 23.

b n = 19.

Table 3. t-Test results for acceptability, feasibility, and usability

Figure 2. Differences in outcomes by guidelines condition.

Qualitative results

Lay counselors and supervisors in sequence seven described benefits of the mobile phone supervision guidelines, and specifically the benefits of having a dedicated educational outreach visit to discuss strategies to improve mobile phone supervision prior to its implementation. The qualitative themes on benefits of the guidelines and outreach included: setting expectations for mobile phone supervision; emphasizing the importance of mobile phone supervision; increasing comfort with mobile phone supervision; and, sharing strategies for mobile phone supervision.

Setting expectations for mobile phone supervision

Guided by the guidelines document (see Goal A), supervisors discussed what counselors might expect during mobile phone supervision and worked to collaboratively set expectations for how supervisors and counselors themselves would approach mobile phone supervision, including expectations for carrying phones and ensuring their phones were working. As one counselor described, “this meeting revealed to me… that everyone is required (to have a phone), if you do not have a phone, try your best to have a phone so that any info comes from a supervisor (you won’t miss it)….” Counselors and supervisors also discussed how the educational outreach visit enabled them to set expectations for scheduling and rescheduling mobile phone supervision, which ultimately led to more successful mobile phone supervision meetings: “The other times before we did these meetings of mobile phone supervision, we were just calling (counselors)…. But now due to these meetings, it made communication work better because now we were planning earlier before the calls.”

Emphasizing the importance of mobile phone supervision

As one supervisor described, having educational outreach visits and discussions about mobile phone supervision before beginning treatment delivery “let the counselors know that it’s not all about in-person (supervision). Because initially, (counselors) were thinking that supervision is mainly important if it’s in person, but then they took the phone supervision more seriously.” Counselors also noted how the visit and guidelines influenced their perceptions of the importance of mobile phone supervision, with one counselor explaining that “(they) had never been subjected to telephone supervision (before this program)…. But, this time (their) supervisor was able to teach (them) the importance of having telephone supervision….” Defining the importance of mobile phone supervision was closely related to setting expectations for supervision, as counselors and supervisors both noted that setting expectations for carrying phones and being available for phone supervision meetings reinforced the importance of mobile phone supervision.

Increasing comfort with mobile phone supervision

This was facilitated by a variety of factors, including decreasing counselor anxiety surrounding mobile phone supervision and increasing comfort with specific strategies. Counselors reported feeling some anxiety calling and getting something wrong or inconveniencing their supervisors; however, as one counselor reported, “(the meeting) put me in a good position to talk on the phone, getting used to doing something like a telephone interview without trembling.” This increased comfort was essential for counselors to engage in all the mobile phone supervision strategies, including those that were intended to reduce counselor airtime usage, such as flashing and reverse calling. Flashing required lay counselors to call and quickly hang up such that their supervisor’s airtime was charged as opposed to their own. Reverse calling is a specific way of dialing that charges the recipient’s airtime. As one supervisor noted, “we made them comfortable to flash us initially. Initially, (the counselors) were like, sometimes they didn’t have their time, so I made the counselors comfortable to just flash me.”

Sharing strategies for mobile phone supervision

The experience of receiving mobile phone supervision was new for counselors, who noted the benefits of not only learning to use phones but also strategies to make their use easier: “We were taught on how we were to use the phones. Now these are the strategies on how we were to use (them).” As one supervisor explained, the educational outreach visit was a more comprehensive and efficient way of sharing strategies that might have naturally occurred later: “Without the meeting, we could have done some of the activities, like making reminder phone calls, but it could have not created as much emphasis, and they could have not gotten as much more information as they did during the meetings.”

Goals and strategies

In describing the benefit of sharing strategies for mobile phone supervision, counselors and supervisors referenced strategies that aligned with each guidelines document goal.

Goal A: Explain logistics of mobile phone supervision

Among the most frequently mentioned (relative to the overall goal) were requests for in-person supervision, in which counselors noted that having this option was essential in building their confidence and supporting them in handling clinical challenges: “As I was delivering the lesson, the girls were so emotional…. It was even difficult for me to continue with the lesson. So, I called my supervisor….” This flexibility and additional support were crucial for supporting counselors as they assumed new counseling roles and the associated emotional difficulties.

Goal B: Ensure counselors get all needed information and support

Counselors frequently mentioned the importance of sharing contact information at the educational outreach visit, both with supervisors and co-counselors. They also discussed how they would vary their communications between phone calls, SMS, and WhatsApp messages. One strategy that was stressed to ensure all counselors were receiving information was to communicate messages to the entire group, either via similar yet separate messages, singular messages to the entire group, or even group calls (i.e., conference calls or calling one counselor and having them place their phone on speaker). Counselors discussed how this helped them to not miss important information from their supervisors, “We could not worry because at least the three of us, there is no way we could (all miss messages).” Another counselor noted that group communications also had the benefit of facilitating learning between counselors: “I learn more because in the, that is WhatsApp, we are in a group, then we share. And as you are sharing that with our supervisor, you get more information from other co-teachers.”

A crucial piece of ensuring counselors received all needed information and support was ensuring that their phones were supplied with airtime, charged, and properly working. To address airtime shortages, counselors discussed various strategies to communicate when their airtime was low, such as “reverse calling” (i.e., calling and charging the recipient’s airtime), “flashing” (i.e., calling and hanging up quickly such that the other person returns the call and charges their own airtime), and sending a free “please call me” text message to supervisors. As one counselor described, “So, we were told in the meeting that in case we don’t have airtime, you are just supposed to flash or even use WhatsApp. We have please call, send a please call me. And our supervisor will call us back.” Counselors also referenced strategies to ensure their phones remained charged and minimize battery use.

Goal C: Plan for challenges with network connection

Counselors mentioned the importance of updating their phones and SIM cards to improve their network connection. Counselors also indicated that the educational outreach visit was helpful in facilitating their identifying locations with strong network connection where they could take mobile phone supervision calls and receive messages. One counselor noted, “I also learned that I have to find a good place where the network is stable to talk to the supervisor for things to be better….” For some counselors, having scheduled mobile phone supervision meetings (another strategy) was also important to ensure they could be in a location with network connection: “there’s just a particular point where we have the network…. So, (the guidelines) forced us to make sure that at a certain time, you need to move somewhere at a certain point so that you get some communication.”

Goal D: Decrease distractions and disruptions

Given that counselors frequently completed supervision meetings at the schools in which they delivered treatment, they noted the importance of identifying quiet and secure places to have supervision phone calls: “during the meeting (the supervisor) also told us to make sure that we are confidential… to make sure that when we decide that we have to make a call, we were to look for a quiet place.” Advance notice of when supervision would occur was crucial for counselors to go to these locations on the school ground, prepare themselves, and notify others that they would be busy with mobile phone supervision. One counselor noted, “I can also tell those who are around, who may make noise, that I have (supervision) now, let us not be noisy. So that when we start, we do not get any disturbance….”

Discussion

Co-developed implementation guidelines and educational outreach visits were associated with improved acceptability and usability of mobile phone supervision. Qualitative interviews with lay counselors and supervisors contextualized how the guidelines and visits impacted acceptability and usability – by setting expectations for mobile phone supervision, emphasizing its importance, increasing comfort, and sharing strategies to improve mobile phone supervision. In discussing how specific strategies were utilized across different contexts, interviews showed how the guidelines were implemented across contexts and highlight the flexibility of our approach. Our results indicate that, with some flexibility and minimal support around their introduction and dissemination, guidelines may be a viable tool to implement and sustain some clinical innovations or implementation strategies across areas with varying levels of resources.

Flexible implementation guidelines

Given the documented challenges with implementing and sustaining interventions and their components (e.g., mobile phone supervision), there have been calls for greater research on how to best tailor implementation support (Baker et al., Reference Baker, Camosso-Stefinovic, Gillies, Shaw, Cheater, Flottorp, Robertson, Wensing, Fiander, Eccles, Godycki-Cwirko, van Lieshout and Jäger2015; Powell et al., Reference Powell, Beidas, Lewis, Aarons, McMillen, Proctor and Mandell2017; Waltz et al., Reference Waltz, Powell, Fernández, Abadie and Damschroder2019). Research has examined implementation support approaches that identify and match implementation strategies to address specific determinants prior to implementation (Baker et al., Reference Baker, Camosso-Stefinovic, Gillies, Shaw, Cheater, Flottorp, Robertson, Wensing, Fiander, Eccles, Godycki-Cwirko, van Lieshout and Jäger2015) as well as more intensive approaches to provide individualized, on-going implementation support, such as implementation facilitation (Smith et al., Reference Smith, Ritchie, Kim, Miller, Chinman, Kelly, Landes and Kirchner2022) and technical assistance (Katz and Wandersman, Reference Katz and Wandersman2016). Though effective, there may be challenges with implementing or scaling complex, multi-faceted implementation strategies in lower-resource settings. We expand the existing research by examining the effect of a pragmatic multi-faceted implementation strategy: co-developed implementation guidelines for mobile phone supervision and brief educational outreach visits. Incorporated into our guidelines were multiple suggested strategies for improving mobile phone supervision, thereby providing guidance to counselors while accounting for needed flexibility and tailoring to different contexts (e.g., multiple options for communicating with different levels of network bandwidth).

In comparing our approach and benefits to other implementation strategies, we highlight the importance of scalability, flexibility, and simplicity in our approach. Noting resource constraints and the need for scalability, we were guided by the concept of “minimal intervention needed to produce change” (Glasgow et al., Reference Glasgow, Fisher, Strycker, Hessler, Toobert, King and Jacobs2014). Though formative work was completed to develop the implementation guidelines (Triplett et al., Reference Triplett, Johnson, Kiche, Dastrup, Nguyen, Daniels, Mbwayo, Amanya, Munson, Collins, Weiner and Dorsey2023), the educational outreach visits were fully integrated into existing research and clinical activities. Lay counselors were supplied with the paper guidelines documents, but they were not required to track strategies or evaluate their own fidelity to the strategies. Increasingly, implementation researchers are trying to understand how little support or intervention can be provided to achieve desired impacts (Glasgow et al., Reference Glasgow, Fisher, Strycker, Hessler, Toobert, King and Jacobs2014; Lyon et al., Reference Lyon, Liu, Connors, King, Coifman, Cook, McRee, Ludwig, Law, Dorsey and McCauley2022). Our approach was designed with this in mind, prioritizing pragmatism and scalability with minimal resources.

Our approach also prioritized flexibility and acknowledged the expertise of lay counselors and supervisors. Children’s mental health implementation research has often only consulted with stakeholders to understand challenges following implementation efforts, which limits the ability of researchers to empower stakeholders, preempt challenges to EBP implementation, and ensure implementation success (Triplett et al., Reference Triplett, Woodard, Johnson, Nguyen, AlRasheed, Song, Stoddard, Mugisha, Sievert and Dorsey2022). Conversely, our approach enabled lay counselors and stakeholders to develop their own guidelines. It also allowed for flexibility and daily tailoring within guidelines by lay counselors and supervisors. As interview quotations illustrate, this approach was highly acceptable as it increased counselor comfort enabled them to seek and receive more support while implementing the intervention. Much of the existing literature on acceptability has focused on the acceptability of interventions to stakeholders (Lewis et al., Reference Lewis, Fischer, Weiner, Stanick, Kim and Martinez2015). However, it is also important to investigate the acceptability of implementation strategies (Proctor et al., Reference Proctor, Powell and McMillen2013), particularly those that may be critical to implementation and eventual sustainment, such as mobile phone supervision. Particularly given the power dynamics inherent in both global mental health and implementation science research, where researchers often put forward EBPs to implement, it is important to assess and potentially improve the acceptability of interventions and implementation strategies.

Impacts on mobile phone supervision

Our approach was associated with greater acceptability and usability of mobile phone supervision. There is no evidence that the introduction of guidelines and the educational outreach visits were associated with significant changes in mobile phone supervision feasibility. There are significant challenges in implementing digital health solutions in LMIC (Kusimba et al., Reference Kusimba, Yang and Chawla2015; Murphy and Priebe, Reference Murphy and Priebe2017; Wyche and Olson, Reference Wyche and Olson2018), including with lay counselors (Triplett et al., Reference Triplett, Johnson, Kiche, Dastrup, Nguyen, Daniels, Mbwayo, Amanya, Munson, Collins, Weiner and Dorsey2023). By focusing specifically on these challenges within our guidelines, we may have even increased the salience of barriers to mobile phone supervision. Interestingly, though average feasibility ratings were high, there was great variability in lay counselor reports of feasibility across both sequences, meaning counselors varied more in their perception of mobile phone supervision as compared to acceptability or usability. This again highlights the importance of flexible approaches that allow tailoring to individual contexts, given that initial perceptions of feasibility may be discrepant across contexts.

The guidelines most often included short-term solutions to challenges that could be implemented with minimal resources. Other potentially viable strategies to improve feasibility may have included the provision of additional resources (e.g., phones or airtime) or formal workload adjustment to allow for lay counselors to receive supervision. There was no intervention on structural challenges (e.g., network coverage), which gave rise to the barriers and could ultimately impact feasibility. Instead, as evidenced in the qualitative findings, the guidelines’ focus on individual actions supported counselors through challenges. Though these types of solutions may be necessary in the present, it should be stressed that equitable and sustainable implementation of mental health interventions, particularly when projects are driven by U.S. investment in lower-resource areas, must also aim to address the structural and systemic factors that have created and maintain inequities (e.g., working to understand and address global power structures that dictate resource allocation).

Limitations

Our implementation guidelines were co-developed alongside lay counselors and supervisors from previous sequences in the trial and were very specific mobile phone supervision in Kenya. While we believe the flexibility and rationale underlying the approach would extrapolate well to other projects, further research is needed to evaluate similar approaches across contexts. Similarly, though the cluster-randomized trail design may have protected against some threats to validity in terms of participant selection, our design cannot account for natural improvements in mobile phone supervision that might have occurred due to the progression of time. Additionally, because of the aims of this trial and desire to reduce reporting burden on participating counselors, we did not track the impact of the facilitation on frequency of mobile phone contacts or the frequency with which specific strategies were used. Further research is needed to investigate the impacts of the program on these as well as other implementation outcomes (i.e., what are the effects of mobile phone supervision on intervention fidelity and clinical outcomes). Finally, our coding team consisted of one native Kiswahili speaker who consulted the Kiswahili audio and answered team questions related to translation and coding for Kiswahili interviews. However, it is possible that some nuance in conversations was lost in the translation to English.

Conclusion

Mobile phones may present an opportunity to increase access to lay counselor supervision; however, implementation of mobile phone supervision must acknowledge and address contextual barriers. Our approach significantly impacted the acceptability and usability of mobile phone supervision. Qualitative interviews with lay counselors and supervisors contextualized how the guidelines and educational outreach visits impacted acceptability and usability – by setting expectations for mobile phone supervision, emphasizing importance, increasing comfort, and sharing strategies to improve mobile phone supervision. Together, results highlight the importance of minimally intensive and flexible implementation supports that can be tailored across contexts and empower stakeholders to select and implement their own solutions. Importantly, we argue this must not come at the cost of addressing structural issues that give rise to downstream barriers. Both elements are crucial to ensuring interventions can be implemented and sustained across areas with varying levels of resources.

Open peer review

To view the open peer review materials for this article, please visit http://doi.org/10.1017/gmh.2023.23.

Supplementary material

The supplementary material for this article can be found at http://doi.org/10.1017/gmh.2023.23.

Data availability statement

The datasets generated and analyzed for this study are available from the corresponding author upon reasonable request.

Acknowledgments

We thank the Ace Africa TF-CBT supervisors, community mobilizer, and interviewers: Elijah Agala, Moses Malaba, Emmanuel Muli, Bernard Nabalia, Micah Nalianya, Michael Nangila, Omariba Nyaboke, Daisy Okoth, Annette Sulungai, Sylvia Wafula, and Nelly Wandera. Their hard work, dedication, and high integrity made this research possible. We are also grateful to the Kenyan Ministry of Health, Ministry of Education, Teachers Service Commission, participating schools, communities, children, and families who are taking part in this trial and the parent trial.

Author contribution

Conceptualization: N.S.T., S.M., B.W., P.C., S.D.; Data curation: N.S.T.; Formal analysis: N.S.T., S.K., L.L., J.S.; Funding acquisition: N.S.T.; Project administration: C.A.; Supervision: N.S.T., A.M., S.M., B.W., P.C., S.D.; Writing – original draft: N.S.T., S.K., L.L., J.S.; Writing – review and editing: N.S.T., A.M., S.K., L.L., J.S., R.A., C.J., C.A., S.M., B.W., P.C., S.D.

Financial support

Funding for this study has been provided by the National Institute of Mental Health (NIMH) through a National Research Service Award (NRSA) Individual Pre-Doctoral Fellowship (NIMH F31 MH124328; N.S.T., PI) and the larger randomized clinical trial that is providing the sample of lay counselors and supervisors (NIMH R01 MH112633; Whetten and S.D., MPI). Funding was also provided by a dissertation grant from the Health Policy Research Scholars Program, a program of the Robert Wood Johnson Foundation. S.M.’s time was supported through the University of Washington ALACRITY Center which is funded through the National Institutes of Health (P50 MH115837). The funders/sponsor have no role in study design or analysis.

Competing interest

N.S.T. has received honoraria for speaking engagements with the Robert Wood Johnson Foundation, a funder of this work. S.D. has received grants and honorariums for providing training and consultation on the treatment model that was adapted and delivered by the lay counselors (trauma-focused cognitive behavioral therapy). The remaining authors declare that the research was conducted in the absence of any significant commercial or financial relationships that could be construed as a potential conflict of interest.

Ethics standard

All authors declare to adhere to the publishing ethics of Global Mental Health.

References

Baker, R, Camosso-Stefinovic, J, Gillies, C, Shaw, EJ, Cheater, F, Flottorp, S, Robertson, N, Wensing, M, Fiander, M, Eccles, MP, Godycki-Cwirko, M, van Lieshout, J and Jäger, C (2015) Tailored interventions to address determinants of practice. Cochrane Database of Systematic Reviews 2015(4), CD005470. https://doi.org/10.1002/14651858.CD005470.PUB3.Google ScholarPubMed
Braun, V and Clarke, V (2006) Using thematic analysis in psychology. Qualitative Research in Psychology 3(2), 77101. https://doi.org/10.1191/1478088706qp063oa.CrossRefGoogle Scholar
Chibanda, D, Weiss, HA, Verhey, R, Simms, V, Munjoma, R, Rusakaniko, S, Chingono, A, Munetsi, E, Bere, T, Manda, E, Abas, M and Araya, R (2016) Effect of a primary care-based psychological intervention on symptoms of common mental disorders in Zimbabwe: A randomized clinical trial. JAMA 316(24), 26182626. https://doi.org/10.1001/jama.2016.19102.CrossRefGoogle ScholarPubMed
Cohen, JA, Mannarino, AP and Deblinger, E (2006) Treating Trauma and Traumatic Grief in Children and Adolescents. New York: Guilford Press.Google Scholar
Demyttenaere, K, Bruffaerts, R, Posada-Villa, J, Gasquet, I, Kovess, V, Lepine, JP, Angermeyer, MC, Bernert, S, de Girolamo, G, Morosini, P, Polidori, G, Kikkawa, T, Kawakami, N, Ono, Y, Takeshima, T, Uda, H, Karam, EG, Fayyad, JA, Karam, AN, Mneimneh, ZN, Medina-Mora, ME, Borges, G, Lara, C, de Graaf, R, Ormel, J, Gureje, O, Shen, Y, Huang, Y, Zhang, M, Alonso, J, Haro, JM, Vilagut, G, Bromet, EJ, Gluzman, S, Webb, C, Kessler, RC, Merikangas, KR, Anthony, JC, von Korff, M, Wang, PS, Brugha, TS, Aguilar-Gaxiola, S, Lee, S, Heeringa, S, Pennell, BE, Zaslavsky, AM, Ustun, TB, Chatterji, S and WHO World Mental Health Survey Consortium (2004) Prevalence, severity, and unmet need for treatment of mental disorders in the World Health Organization world mental health surveys. JAMA 291(21), 25812590. https://doi.org/10.1001/JAMA.291.21.2581.Google ScholarPubMed
Dorsey, S, Gray, CL, Wasonga, AI, Amanya, C, Weiner, BJ, Belden, CM, Martin, P, Meza, RD, Weinhold, AK, Soi, C, Murray, LK, Lucid, L, Turner, EL, Mildon, R and Whetten, K (2020a) Advancing successful implementation of task-shifted mental health care in low-resource settings (BASIC): Protocol for a stepped wedge cluster randomized trial. BMC Psychiatry 20(1), 10. https://doi.org/10.1186/s12888-019-2364-4.CrossRefGoogle ScholarPubMed
Dorsey, S, Lucid, L, Martin, P, King, KM, O’Donnell, K, Murray, LK, Wasonga, AI, Itemba, DK, Cohen, JA, Manongi, R and Whetten, K (2020b) Effectiveness of task-shifted trauma-focused cognitive behavioral therapy for children who experienced parental death and posttraumatic stress in Kenya and Tanzania: A randomized clinical trial. JAMA Psychiatry 77(5), 464473. https://doi.org/10.1001/jamapsychiatry.2019.4475.CrossRefGoogle ScholarPubMed
Feroz, A, Jabeen, R and Saleem, S (2020) Using mobile phones to improve community health workers performance in low-and-middle-income countries. BMC Public Health 20(1), 16. https://doi.org/10.1186/S12889-020-8173-3/PEER-REVIEW.CrossRefGoogle ScholarPubMed
Francke, AL, Smit, MC, De Veer, AJE and Mistiaen, P (2008) Factors influencing the implementation of clinical guidelines for health care professionals: A systematic meta-review. BMC Medical Informatics and Decision Making 8, 38. https://doi.org/10.1186/1472-6947-8-38.CrossRefGoogle ScholarPubMed
Glasgow, RE, Fisher, L, Strycker, LA, Hessler, D, Toobert, DJ, King, DK and Jacobs, T (2014) Minimal intervention needed for change: Definition, use, and value for improving health and health research. Translational Behavioral Medicine 4(1), 26. https://doi.org/10.1007/S13142-013-0232-1.CrossRefGoogle ScholarPubMed
Henry, JV, Winters, N, Lakati, A, Oliver, M, Geniets, A, Mbae, SM and Wanjiru, H (2016) Enhancing the supervision of community health workers with WhatsApp mobile messaging: Qualitative findings from 2 low-resource settings in Kenya. Global Health, Science and Practice 4(2), 311325. https://doi.org/10.9745/GHSP-D-15-00386.CrossRefGoogle ScholarPubMed
Hill, CE, Thompson, BJ and Williams, EN (1997) A guide to conducting consensual qualitative research. Counseling Psychologist 25(4), 517572. https://doi.org/10.1177/0011000097254001.CrossRefGoogle Scholar
Hoeft, TJ, Fortney, JC, Patel, V and Unützer, J (2018) Task-sharing approaches to improve mental health care in rural and other low-resource settings: A systematic review. Journal of Rural Health 34(1), 4862. https://doi.org/10.1111/jrh.12229.CrossRefGoogle ScholarPubMed
Katz, J and Wandersman, A (2016) Technical assistance to enhance prevention capacity: A research synthesis of the evidence base. Prevention Science 17(4), 417428. https://doi.org/10.1007/S11121-016-0636-5.CrossRefGoogle ScholarPubMed
Kohn, R, Saxena, S, Levav, I and Saraceno, B (2004) The treatment gap in mental health care. Bulletin of the World Health Organization 82(11), 858866.Google ScholarPubMed
Kusimba, SB, Yang, Y and Chawla, NV (2015) Family networks of mobile money in Kenya. Information Technologies & International Development 11(3), 121.Google Scholar
Lewis, CC, Fischer, S, Weiner, BJ, Stanick, C, Kim, M and Martinez, RG (2015) Outcomes for implementation science: An enhanced systematic review of instruments using evidence-based rating criteria. Implementation Science 10(1), 117. https://doi.org/10.1186/s13012-015-0342-x.CrossRefGoogle ScholarPubMed
Lyon, AR, Liu, FF, Connors, EH, King, KM, Coifman, JI, Cook, H, McRee, E, Ludwig, K, Law, A, Dorsey, S and McCauley, E (2022) How low can you go? Examining the effects of brief online training and post-training consultation dose on implementation mechanisms and outcomes for measurement-based care. Implementation Science Communications 3(1), 79. https://doi.org/10.1186/S43058-022-00325-Y.CrossRefGoogle Scholar
Lyon, AR, Pullmann, M, Jacobson, J, Osterhage, K, Achkar, MA, Renn, B, Munson, S and Areán, P (2020) Assessing the usability of complex psychosocial interventions: The intervention usability scale. Implementation Research and Practice 2. https://doi.org/10.1177/2633489520987828.Google Scholar
Moitra, M, Santomauro, D, Collins, PY, Vos, T, Whiteford, H, Saxena, S and Ferrari, AJ (2022) The global gap in treatment coverage for major depressive disorder in 84 countries from 2000-2019: A systematic review and Bayesian meta-regression analysis. PLoS Medicine 19(2), e1003901. https://doi.org/10.1371/JOURNAL.PMED.1003901.CrossRefGoogle ScholarPubMed
Murphy, LL and Priebe, AE (2017) My co-wife can borrow my mobile phone! Gender, Technology and Development 15(1), 123. https://doi.org/10.1177/097185241101500101.CrossRefGoogle Scholar
Murray, LK, Dorsey, S, Haroz, E, Lee, C, Alsiary, MM, Haydary, A, Weiss, WM and Bolton, P (2014) A common elements treatment approach for adult mental health problems in low- and middle-income countries. Cognitive and Behavioral Practice 21(2), 111123. https://doi.org/10.1016/j.cbpra.2013.06.005.CrossRefGoogle ScholarPubMed
Nabyonga Orem, J, Bataringaya Wavamunno, J, Bakeera, SK and Criel, B (2012) Do guidelines influence the implementation of health programs? – Uganda’s experience. Implementation Science 7(1), 116. https://doi.org/10.1186/1748-5908-7-98/FIGURES/4.CrossRefGoogle ScholarPubMed
Naslund, JA, Shidhaye, R and Patel, V (2019) Digital technology for building capacity of nonspecialist health workers for task sharing and scaling up mental health care globally. Harvard Review of Psychiatry 27(3), 181192. https://doi.org/10.1097/HRP.0000000000000217.CrossRefGoogle ScholarPubMed
Padmanathan, P and De Silva, MJ (2013) The acceptability and feasibility of task-sharing for mental healthcare in low and middle income countries: A systematic review. Social Science & Medicine 97, 8286. https://doi.org/10.1016/j.socscimed.2013.08.004.CrossRefGoogle ScholarPubMed
Palinkas, LA, Aarons, GA, Horwitz, S, Chamberlain, P, Hurlburt, M and Landsverk, J (2011) Mixed method designs in implementation research. Administration and Policy in Mental Health 38(1), 4453. https://doi.org/10.1007/s10488-010-0314-z.CrossRefGoogle ScholarPubMed
Peters, S, Sukumar, K, Blanchard, S, Ramasamy, A, Malinowski, J, Ginex, P, Senerth, E, Corremans, M, Munn, Z, Kredo, T, Remon, LP, Ngeh, E, Kalman, L, Alhabib, S, Amer, YS and Gagliardi, A (2022) Trends in guideline implementation: An updated scoping review. Implementation Science 17(1), 117. https://doi.org/10.1186/S13012-022-01223-6.CrossRefGoogle ScholarPubMed
Powell, BJ, Beidas, RS, Lewis, CC, Aarons, GA, McMillen, JC, Proctor, EK and Mandell, DS (2017) Methods to improve the selection and tailoring of implementation strategies. Journal of Behavioral Health Services & Research 44(2), 177. https://doi.org/10.1007/S11414-015-9475-6.CrossRefGoogle ScholarPubMed
Proctor, E, Powell, BJ and McMillen, JC (2013) Implementation strategies: Recommendations for specifying and reporting. Implementation Science 8(1), 111. https://doi.org/10.1186/1748-5908-8-139.CrossRefGoogle ScholarPubMed
QSR International Pty Ltd (2018) NVivo qualitative data analysis software (No. 12). https://support.qsrinternational.com/nvivo/s/article/How-do-I-cite-QSR-software-in-my-work.Google Scholar
R Core Team. (2022) R: A Language and Environment for Statistical Computing. (3.3.0). Vienna, Austria: R Foundation for Statistical Computing.Google Scholar
Rahman, A, Akhtar, P, Hamdani, SU, Atif, N, Nazir, H, Uddin, I, Nisar, A, Huma, Z, Maselko, J, Sikander, S and Zafar, S (2019) Using technology to scale-up training and supervision of community health workers in the psychosocial management of perinatal depression: A non-inferiority, randomized controlled trial. Global Mental Health 6, 15. https://doi.org/10.1017/gmh.2019.7.Google ScholarPubMed
Smith, JL, Ritchie, MJ, Kim, B, Miller, CJ, Chinman, M, Kelly, PA, Landes, S and Kirchner, JE (2022) Getting to fidelity: Scoping review and expert panel process to identify core activities of implementation facilitation strategies. SocArXiv. https://doi.org/10.31235/OSF.IO/6XFVJ.CrossRefGoogle Scholar
Triplett, NS, Johnson, C, Kiche, S, Dastrup, K, Nguyen, J, Daniels, A, Mbwayo, A, Amanya, C, Munson, S, Collins, PY, Weiner, BJ and Dorsey, S (2023) Understanding lay counselor perspectives on Mobile phone supervision in Kenya: Qualitative study. JMIR Formative Research 7(1), e38822. https://doi.org/10.2196/38822.CrossRefGoogle ScholarPubMed
Triplett, NS, Munson, S, Mbwayo, A, Mutavi, T, Weiner, BJ, Collins, P, Amanya, C and Dorsey, S (2021) Applying human-centered design to maximize acceptability, feasibility, and usability of mobile technology supervision in Kenya: A mixed methods pilot study protocol. Implementation Science Communications 2(1), 2. https://doi.org/10.1186/S43058-020-00102-9.CrossRefGoogle Scholar
Triplett, NS, Woodard, GS, Johnson, C, Nguyen, JK, AlRasheed, R, Song, F, Stoddard, S, Mugisha, JC, Sievert, K and Dorsey, S (2022) Stakeholder engagement to inform evidence-based treatment implementation for children’s mental health: A scoping review. Implementation Science Communications 3(1), 116. https://doi.org/10.1186/S43058-022-00327-W.CrossRefGoogle ScholarPubMed
van Ginneken, N, Tharyan, P, Lewin, S, Rao, GN, Meera, SM, Pian, J, Chandrashekar, S and Patel, V (2013) Non-specialist health worker interventions for the care of mental, neurological and substance-abuse disorders in low- and middle-income countries. Cochrane Database of Systematic Reviews 11, CD009149. https://doi.org/10.1002/14651858.CD009149.pub2.Google Scholar
Waltz, TJ, Powell, BJ, Fernández, ME, Abadie, B and Damschroder, LJ (2019) Choosing implementation strategies to address contextual barriers: Diversity in recommendations and future directions. Implementation Science 14(1), 42. https://doi.org/10.1186/S13012-019-0892-4.CrossRefGoogle ScholarPubMed
Weiner, BJ, Lewis, CC, Stanick, C, Powell, BJ, Dorsey, CN, Clary, AS, Boynton, MH and Halko, H (2017) Psychometric assessment of three newly developed implementation outcome measures. Implementation Science 12(1), 112. https://doi.org/10.1186/s13012-017-0635-3.CrossRefGoogle ScholarPubMed
Weiss, WM, Murray, LK, Zangana, GAS, Mahmooth, Z, Kaysen, D, Dorsey, S, Lindgren, K, Gross, A, Murray, SMI, Bass, JK and Bolton, P (2015) Community-based mental health treatments for survivors of torture and militant attacks in southern Iraq: A randomized control trial. BMC Psychiatry 15(1), 249. https://doi.org/10.1186/s12888-015-0622-7.CrossRefGoogle ScholarPubMed
Weissgerber, TL, Garovic, VD, Savic, M, Winham, SJ and Milic, NM (2016) From static to interactive: Transforming data visualization to improve transparency. PLoS Biology 14(6), e1002484. https://doi.org/10.1371/JOURNAL.PBIO.1002484.CrossRefGoogle ScholarPubMed
Wyche, SP and Murphy, LL (2012) “Dead China-make” phones off the grid: Investigating and designing for mobile phone use in rural Africa. In Proceedings of the Designing Interactive Systems Conference, DIS ‘12, 186195. https://doi.org/10.1145/2317956.2317985.CrossRefGoogle Scholar
Wyche, SP and Olson, J (2018) Gender, mobile, and mobile internet| Kenyan women’s rural realities, mobile internet access, and “Africa rising”. Information Technologies & International Development 14, 15.Google Scholar
Figure 0

Figure 1. Trial sequence and activities.

Figure 1

Table 1. Sample implementation guidelines for Goal A

Figure 2

Table 2. Demographics and baseline characteristics

Figure 3

Table 3. t-Test results for acceptability, feasibility, and usability

Figure 4

Figure 2. Differences in outcomes by guidelines condition.

Supplementary material: File

Triplett et al. supplementary material

Triplett et al. supplementary material 1

Download Triplett et al. supplementary material(File)
File 22.6 KB
Supplementary material: File

Triplett et al. supplementary material

Triplett et al. supplementary material 2

Download Triplett et al. supplementary material(File)
File 38.7 KB
Supplementary material: File

Triplett et al. supplementary material

Triplett et al. supplementary material 3

Download Triplett et al. supplementary material(File)
File 23.8 KB

Author comment: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR1

Comments

To Drs. Bass and Chibanda,

Editors-in-Chief

Cambridge Prisms: Global Mental Health

My co-authors and I are submitting the attached article titled, Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya, for your consideration for publication in Cambridge Prisms: Global Mental Health. Little scholarship has focused how to scale up and sustain task-shifting of evidence-based practices (EBPs) for mental health disorders in lower-resourced settings, including how to sustainably supervise lay counselors. We describe methods and results from an iterative, mixed methods trial that engaged lay counselors and supervisors providing trauma-focused cognitive behavioral therapy (TF-CBT) in Kenya to redesign supervision to most effectively leverage mobile technology.

We briefly outline the importance of developing scalable and sustainable approaches to supervision—an essential yet resource-intensive implementation strategy for ensuring fidelity to EBPs. The rise in global internet access, coupled with COVID, has highlighted opportunities to leverage technology to increase access to care; however, contextual variables, such as network connectivity and user preferences, must be examined and considered when designing and implementing mobile technology interventions and implementation strategies. We describe our approach, which engaged lay counselors and supervisors to design and test a method of optimizing mobile phone supervision. We honored their contextual expertise and engaged them throughout the research process to identify potential barriers and generate solutions to using mobile technology to provide supervision. Ultimately, we co-developed implementation guidelines that were shared and discussed with newly implementing counselors in a brief outreach visit. Results showed significant improvements in mobile phone supervision acceptability and usability. Qualitative results contextualize how the program was effective. Our findings and approach highlight the importance of co-developing flexible and pragmatic implementation strategies to support task shifting and global mental health approaches.

The manuscript is 4,917 words in length (excluding title pages, abstract, tables, and figures), 3 tables, 3 figures, and 2 supplemental files. This manuscript is not in submission to any other journals. All authors have contributed to and approved this submission. Thank you for your full consideration of this manuscript. I look forward to your review. If you have questions or would like further information, please feel free to contact me at (336) 596-8132 or by email at nst7@uw.edu.

Sincerely,

Noah S. Triplett, MS

PhD Candidate in Clinical Psychology

University of Washington

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR2

Conflict of interest statement

None.

Comments

Comments to Author: Manuscript Review

Title: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya

These authors present findings on the iterative development and implementation of guidelines focused on facilitating mobile supervision among lay providers administering Trauma-Focused Cognitive Behavioral Therapy (TF-CBT) in Kenya. The framework presented is both innovative and pragmatic and has the potential to optimize supervision-focused implementation in other contexts and with various technologies/modalities. Importantly, the guidelines were developed collaboratively, centering on community participatory principles, emphasizing the expertise of local collaborators. Additionally, the manuscript is well-written, and the methodology and design are very clear. In its current state, this manuscript makes a meaningful contribution. There are a few limitations and points of clarification below that if address may further improve the impact of these findings.

Major comments/revisions

Method- Implementation Guidelines

• It may be useful for the reader if the authors could expand a bit more on how the educational outreach sessions were conducted. Were these held in person with all supervisors and sequence seven counselors in attendance? Or were sessions conducted via one-on-one contact with supervisors being assigned to hold sessions with a certain number of counselors? Was there a gap between when these sessions were held and when phone supervision began (if this gap was meaningful, it might be helpful to report this)?

Method- Measures

• It seems plausible that the acceptability, feasibility, and usability constructs may have some conceptual overlap. Would the authors be able to provide correlations between these measures to confirm that these are divergent constructs?

Method- Analysis

• Would the authors consider adding some type of effect size for their independent-samples t-test? From the results/ visualization, it appears that the effect of the guidelines may vary by construct.

• For the qualitative analysis, it would be helpful if the authors included information about the number and qualifications of the coders. Further, it would be informative to know the extent to which the coders were involved in the study and whether/how they were trained on coding procedures.

Results

• The authors present the means/SDs of acceptability, feasibility, and usability across sequences. However, Figure 3 demonstrates these findings across both sequences and sector. This makes it somewhat difficult to visualize the findings collapsed across sectors. Would the authors be able to either collapse across sectors or present an additional figure that highlights the findings as they are presented in Table 3?

Discussion

• A few questions came to mind while reading this manuscript regarding the impact of the guidelines/ outreach sessions: 1) Did these contribute to an increase in number of telephone supervision contacts?; 2) Were there specific strategies that lay counselors/supervisors used more than others?; 3) Did guidelines improve the effectiveness of supervision conducted via phone?; and 4) Did the any of this related to fidelity with implementing the intervention/clinical outcomes. I realize that answering many of these questions is beyond the scope of this manuscript. If possible to address questions 1 and 2, I think those would be interesting to the reader and emphasize the utility of the guidelines/ sessions (above and beyond the utility that is already highlighted!). If you are unable to include these in the manuscript, I may consider a limitation/future direction is that it is unclear how these sessions impact strategies chosen and frequency of these sessions. Questions 3 and 4 may be worth mentioning as future directions if the authors have adequate space. If not, something to ponder on!

Minor comments/revisions

Introduction

• On page 5, the authors briefly mention noted challenges of disseminating guidelines. It may be helpful for the reader to briefly expand on what these challenges are (if the authors have sufficient space).

Method- Participants

• The authors note that there were no exclusion criteria. Was there any inclusion criteria to be a lay counselor for the parent study? If so, would they be able to add that in?

Figures

• Figure 3: The x-axis on these figures is difficult to read. The words of the categories listed on the x-axis are overlapping. It’s possible that this is not an issue for others and may be a result of the way the file downloaded on my computer, but I thought it was worth mentioning if the formatting was somehow not converted correctly during the submission process.

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR3

Conflict of interest statement

None

Comments

Comments to Author: This paper represents an important move forward in providing the use of mobile phone technology supervision of mental health field workers and ultimately better mental health care to those in need. The paper also addresses the need to provide mental health care to formerly underserved and difficult to reach populations in LMIC.

Recommendation: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR4

Comments

Comments to Author: The study explores a very important subject on co-development of implementation guidelines on mobile phone supervision in Kenya. However, the reviewers have made very relevant observations and recommendations that may improve the quality and content of the manuscript. I invite the authors to respond to comments of the reviewer 1 and resubmit the manuscript.

Decision: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R0/PR5

Comments

No accompanying comment.

Author comment: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR6

Comments

April 10, 2023

To Drs. Bass and Chibanda,

Editors-in-Chief

Cambridge Prisms: Global Mental Health

My co-authors and I are re-submitting the attached article titled, Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya, for your consideration for publication in Cambridge Prisms: Global Mental Health. Little scholarship has focused how to scale up and sustain task-shifting of evidence-based practices (EBPs) for mental health disorders in lower-resourced settings, including how to sustainably supervise lay counselors. We describe methods and results from an iterative, mixed methods trial that engaged lay counselors and supervisors providing trauma-focused cognitive behavioral therapy (TF-CBT) in Kenya to redesign supervision to most effectively leverage mobile technology.

We briefly outline the importance of developing scalable and sustainable approaches to supervision—an essential yet resource-intensive implementation strategy for ensuring fidelity to EBPs. The rise in global internet access, coupled with COVID, has highlighted opportunities to leverage technology to increase access to care; however, contextual variables, such as network connectivity and user preferences, must be examined and considered when designing and implementing mobile technology interventions and implementation strategies. We describe our approach, which engaged lay counselors and supervisors to design and test a method of optimizing mobile phone supervision. We honored their contextual expertise and engaged them throughout the research process to identify potential barriers and generate solutions to using mobile technology to provide supervision. Ultimately, we co-developed implementation guidelines that were shared and discussed with newly implementing counselors in a brief outreach visit. Results showed significant improvements in mobile phone supervision acceptability and usability. Qualitative results contextualize how the program was effective. Our findings and approach highlight the importance of co-developing flexible and pragmatic implementation strategies to support task shifting and global mental health approaches.

The manuscript is 4,991 words in length (excluding title pages, abstract, tables, and figures), 3 tables, 2 figures, and 3 supplemental files. This manuscript is not in submission to any other journals. All authors have contributed to and approved this submission. Thank you for your full consideration of this manuscript. I look forward to your review. If you have questions or would like further information, please feel free to contact me at (336) 596-8132 or by email at nst7@uw.edu.

Sincerely,

Noah S. Triplett, MS

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR7

Conflict of interest statement

Nothing to disclose

Comments

Comments to Author: I appreciate the authors' response to my comments. They have addressed all of my concerns, and I’m happy to recommend this article for publication. Thank you for the opportunity to review this exciting and innovative work!

Review: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR8

Conflict of interest statement

This is an excellent paper focused on the urgent need for clinical supervision in low-resource settings. The creative use of mobile phone technology to reach these often overlooked settings is a major movement to level the "playing field" in low-resource settings.

Comments

Comments to Author: This is an excellent paper focused on the urgent need for clinical supervision in low-resource settings. The creative use of mobile phone technology to reach these often overlooked settings is a major movement to level the “playing field” in low-resource settings.

Recommendation: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR9

Comments

Comments to Author: Thank you for taking into account the feedback provided by the reviewers and incorporating the necessary changes in the revised manuscript. We are pleased to inform you that your manuscript has been accepted for publication.

Decision: Co-developed implementation guidelines to maximize acceptability, feasibility, and usability of mobile phone supervision in Kenya — R1/PR10

Comments

No accompanying comment.