Conceptual framework: a holistic understanding of implementation science

This supplement demonstrated the use of a nutrition implementation science framework to facilitate the conceptualisation and understanding of a nutrition intervention that was comprehensively implemented in a real-world setting. The second paper of this supplement proposed a conceptual framework with five components: (1) identifying an ‘effective’ intervention; (2) scaling-up and implementation fidelity; (3) course corrections during implementation; (4) promoting sustainability of interventions and (5) consideration of a comprehensive methodological paradigm^{(Reference Sarma, D’Este and Ahmed1)}. The other four articles (Papers 3–6) in this supplement address the different components of the framework. For example, the third article addresses the implementation fidelity of a home-fortification programme^{(Reference Sarma, Tariqujjaman and Mbuya5)}, and the fourth article describes the use of concurrent evaluation to course correct the implementation plan that resulted in improved implementation fidelity^{(Reference Sarma, Uddin and Islam6)}. The fifth article explains the outcome of course correction in the programme coverage^{(Reference Sarma, Mbuya and Tariqujjaman7)}, and the sixth article describes the cost-effectiveness of the Bangladesh Rural Advancement Committee (BRAC) home-fortification programme^{(Reference Ahmed, Sarma and Hasan8)}. The following sections of this paper clarify each of the components of our implementation science framework, summarise key findings reported in Papers 3–6 of this supplement and critically synthesises these findings with support from the relevant literature.

Identifying an ‘effective’ intervention and BRAC home fortification

The first component of the framework helped to identify an ‘effective’ intervention, whereby our framework proposes four intervention characteristics: intervention sources, evidence strength and quality, relative advantages and adaptability and complexity of intervention^{(Reference Sarma, D’Este and Ahmed1,Reference Damschroder, Aron and Keith9)} . The review of these intervention characteristics allows implementation researchers to assess whether the intervention is ready for scaling-up in larger implementation settings^{(Reference Brouwer, Kroeze and Crutzen10)}. Our conceptual framework proposes four key characteristics of an intervention to make a scaling-up decision: intervention source, evidence strength and quality, relative advantages and adaptability and complexities^{(Reference Sarma, D’Este and Ahmed1)}.

When an intervention is identified through authentic sources and the evaluation demonstrates high-quality evidence, then the implementation stakeholders, including implementers, beneficiary groups and funders, may develop confidence in the intervention. The current study focussed on the micronutrient powder (MNP) supplement that was developed and tested efficacy and effectiveness through several randomised controlled trials and corroboration, and such evidence has been published in peer-reviewed journals^{(Reference Zlotkin, Schauer and Christofides11–Reference Dewey, Yang and Boy13)}. It is generally assumed that the results of a trial published in a peer-reviewed journal constitute authentic evidence, as a peer review process acts to filter out low-quality research, and scholarly research is endorsed by peers who are experts in the same field^{(Reference Kelly, Sadeghieh and Adeli14)}.

The relative advantages of home-fortification intervention have been well established through several effectiveness trials^{(Reference Zlotkin, Schauer and Christofides11,Reference Albelbeisi, Shariff and Mun15–Reference Horton, Shekar and McDonald17)} . Compared to other interventions (e.g. Fe syrups and paediatric drops) for childhood anaemia, the MNP home fortification has some advantages, such as MNP sachets being easy to distribute and store by community-level health workers and being easy to use by child caregivers^{(Reference Zlotkin, Schauer and Christofides11)}. MNP home-fortification with food has fewer side effects compared with other similar interventions, such as supplements in Fe syrups and drops^{(Reference Albelbeisi, Shariff and Mun15)}. The economic evaluation of MNP home fortification demonstrates that it is cost effective in terms of per capita disability-adjusted life years (fifth article of this supplement)^{(Reference Ahmed, Sarma and Hasan8,Reference Boo, Palloni and Urzua16,Reference Horton, Shekar and McDonald17)} .

To reach the targeted children, multiple models, platforms and channels have identified potential for MNP delivery, including the use of community-based distribution channels^{(18,Reference Reerink, Namaste and Poonawala19)} . At the community level, the MNP intervention needs community-based facilities and a dedicated skilled health workforce at the community level, as the intervention needs to be implemented at the household level^{(Reference Reerink, Namaste and Poonawala19–Reference Sarma, Uddin and Harbour21)}. BRAC, a development organisation, based in Bangladesh, aimed to reduce Fe deficiency anaemia (termed as anaemia) by delivering home fortification with MNP among children aged 6–59 months^{(Reference Sarma, Mbuya and Tariqujjaman7,Reference Afsana, Haque and Sobhan20)} . A detailed description of BRAC home fortification is available in Paper 4 of this supplement^{(Reference Sarma, Mbuya and Tariqujjaman7)}. The availability of BRAC community health workers and implementation facilities across sub-districts and districts in Bangladesh are key advantages for BRAC to scale up MNP intervention at the national level^{(Reference Sarma, Uddin and Harbour21,Reference Sarma, Jabeen and Luies22)} .

Implementation fidelity: context and process of BRAC home fortification

The second component of the framework is implementation fidelity. In the scaling-up of an intervention, it is important to understand implementation fidelity and measure it in a timely manner^{(Reference Knapp and Anaya23)} to understand whether there is any gap in the implementation which may need to be addressed. Evaluation of implementation fidelity allows implementers to better understand the intervention outcomes and correlate why a certain outcome is observed^{(Reference Meyers, Durlak and Wandersman24,Reference Blanchard, Livet and Ward25)} . Our framework suggests that evaluators should assess the implementation process and outcomes to understand implementation fidelity and investigate the implementation contexts at the individual, programme and organisation levels^{(Reference Sarma, D’Este and Ahmed1,Reference Sarma, Uddin and Islam6)} . Timely understanding of the implementation process facilitates endorsement by implementers to adjust implementation gaps through timely course correction that eventually ensures high implementation fidelity and expected intervention outcomes^{(Reference Sarma, D’Este and Ahmed1,Reference Sarma, Uddin and Islam6)} .

The framework proposed to assess implementation contexts and processes and timely investigate the bottlenecks in implementation fidelity^{(Reference Sarma, D’Este and Ahmed1,Reference Sarma, Mbuya and Tariqujjaman7)} . We used both qualitative and quantitative analyses to investigate key contextual factors associated with the performance of BRAC community health workers. The performance of BRAC community health workers has been identified as an important issue in the implementation fidelity of BRAC community health workers. Home visits by BRAC community health workers was a key performance indicator for its home-fortification programme^{(Reference Sarma, Uddin and Harbour21)}. The qualitative analysis of BRAC home-fortification programmes published elsewhere^{(Reference Sarma, Jabeen and Luies22)} identified a range of factors at the individual, community, programme and organisational levels, which influenced the performance of BRAC community health workers with respect to implementing the home-fortification programme^{(Reference Sarma, Jabeen and Luies22)}.

Real-time course correction of programme implementation plan

As discussed above, with respect to the key findings related to the implementation process and context of the BRAC home-fortification intervention, it was revealed that implementation fidelity critically suffered from several implementation bottlenecks and required the attention of implementers to address them. The fourth article^{(Reference Sarma, Tariqujjaman and Mbuya5)} of this supplement documented how BRAC considered these evaluation findings to promptly address the implementation bottlenecks in the home-fortification programme while it was still in operation. This article also documented that timely course correction of the programme’s implementation plan improved the programme’s outcomes. Globally, there are extremely limited examples available of a large-scale programme being implemented at the national level and demonstrating improvement over the period through evidence-based programme course correction.

One of the key components of our implementation science framework is the concurrent evaluation and use of real-time evaluation data to address and overcome any obstacles to implementation^{(Reference Knapp and Anaya23)}. The concurrent evaluation data can be used in a way that is consistent with feedback-loop mechanisms to assess a ‘snapshot’ of an individual’s or team’s advancement in institutionalising the intervention in a complex real-world setting^{(Reference Knapp and Anaya23,Reference Meyers, Durlak and Wandersman24)} . These findings also provided additional evidence and highlighted the implementation of the intervention over the period, establishing that the transformation was adopted^{(Reference Knapp and Anaya23,Reference Blanchard, Livet and Ward25)} .

Despite the importance of evaluation in implementation science, there is a recommendation to consider evaluation separate from implementation science^{(Reference Warren, Frongillo and Rawat26)}, which may reduce opportunity for timely assessing the implementation gaps and course correction of these gaps when it is essential. Paper 4 of this supplement described how the BRAC home-fortification programme was difficult to implement at the community level in the initial years due to the low performance of BRAC community health workers. A concurrent evaluation found that several contextual factors associated with the different levels of the BRAC home-fortification programme influenced the programme outcomes. The lessons learned through evaluation supported successful programme implementation and increased the confidence of implementers and investors in the evaluator. This reduced the anxiety usually observed among implementers towards evaluation and positively impacted the relationship between evaluators and implementers^{(Reference Meyers, Durlak and Wandersman24)}. A previous paper suggested a feedback loop mechanism, emphasising ongoing follow-up, gap identification, adoption and extended uptake phases, so that with each cycle the intervention becomes more decisively rooted within a system^{(Reference Knapp and Anaya23)}.

Concurrent evaluation improved programme outcomes

A rigorous evaluation strategy is required from the very beginning of implementation science (to identify an effective intervention)^{(Reference Verhagen, Voogt and Bruinsma27)}. Several implementation science frameworks dealing with nutrition interventions suggest an evaluation strategy to measure the implementation context, process, outcome and impacts^{(Reference Tumilowicz, Ruel and Pelto28–Reference Gillespie, Menon and Kennedy30)}. In implementation science, the evaluation should not be only concerned with measuring the implementation outcome or impact but also with considering how an intervention generates certain outcomes: specifically, how evaluation evidence helps to drive implementation improvements^{(Reference Meyers, Durlak and Wandersman24,Reference Van Achterberg, Schoonhoven and Grol31)} .

The BRAC home-fortification programme benefited from concurrent evaluation, which included several evaluation activities, such as a coverage survey, qualitative investigations, a process evaluation, operations research and an economic evaluation^{(Reference Sarma, Tariqujjaman and Mbuya5)}. The coverage survey measured the programme’s coverage, identifying barriers to coverage and measuring the impact of the programme. The qualitative investigations were supplemented by coverage survey findings and generated in-depth evidence to support the holistic interpretation of the facts. The process evaluation tracked the implementation process of the home-fortification programme, identified challenges in the implementation process and performed an in-depth investigation to understand the root causes of the challenges. The operations research identified additional interventions that helped to improve the performance of BRAC community health workers. The final round of the coverage survey (i.e. three endline surveys in the three-programme platforms) measured the impact of the programme on anaemia reduction.

Sustainability of the BRAC home-fortification programme

The issues of sustainability are becoming increasingly important in implementation science and are of interest among researchers, implementers, donors and stakeholders who wish to improve the sustainability of effective public health programmes^{(Reference Moucheraud, Sarma and Ha32–Reference Walugembe, Sibbald and Le Ber35)}. Previous literature suggests considering sustainability in implementation science frameworks as a theory-driven approach to guide the design, development, implementation, evaluation and sustainability of interventions^{(Reference O’Loughlin, Renaud and Richard36)}. Our conceptual framework for implementation science proposes that a conceptual understanding of sustainability is important and suggests the need to measure the dimensions and determinants of sustainability in a real-world setting. The dimensions related to the sustainability of the BRAC home-fortification programme are dose, reach, fidelity and adaptability, and the determinants are the characteristics of the BRAC home-fortification programme, the organisational factors and the contextual factors^{(Reference Sarma, D’Este and Ahmed1)}. The dose refers to the inputs and continuing investments in the home-fortification interventions, such as continuous training for the BRAC community health workers. The reach refers to the coverage of the home-fortification programme, for example, whether the home-fortification intervention maintains a sustained coverage. The fidelity refers to the quality of the implementation and whether the BRAC home-fortification programme maintains implementation quality after the phase-out of external funding. The final dimension is adaptability, which concerns whether the BRAC home fortification is continuously adapting to the implementation contexts and maintaining a high level of implementation fidelity.

This concurrent evaluation did not assess the dimensions of sustainability of the BRAC home-fortification programme. It was designed to assess the sustainability of an externally funded programme after 2 to 3 years of the phase-out of external funding supports^{(Reference O’Loughlin, Renaud and Richard36–Reference Shelton, Cooper and Stirman38)}. The BRAC home-fortification funding phased out in December 2018. Now BRAC may initiate assessing the sustainability of its home-fortification programme.

There is much positive evidence available regarding the determinants of the sustainability of the BRAC home-fortification programme. The programme used a market-based strategy, which usually ensures better financial sustainability compared with a subsidised programme, where MNP sachets are distributed free of cost to the caregiver. Another important characteristic of the BRAC programme is the use of volunteer community health workers and the incentivisation of them through several methods, such as allowing them to share in the profit margin of BRAC’s product sales, supporting them to maintain a revolving fund and providing them with guidance for income generation. These incentives motivated the BRAC community health workers to better perform in the programme and assumed that they may continue after the phase-out of funding supports for the programme. Previous literature also suggests that both financial and non-financial incentives motivate community health workers and sustain programme implementation with better outcomes^{(Reference Pallas, Minhas and Pérez-Escamilla39–Reference Ge, Fu and Chang42)}.

BRAC has a good organisational set-up across Bangladesh. BRAC programmes cover every district in the country with a strong network from the community to districts, which supports community-level services. At the community level, BRAC has well-equipped field offices that can be used to provide programme-specific training for community health workers. BRAC-paid community health workers (e.g. Shasthya Kormi) are skilled service providers who provide a range of health services to communities. BRAC recently introduced a sustainable community health worker model, wherein the community health workers and their supervisor sell their services to the community and generate revenue for BRAC.

Limitations and strengths

This research project has several limitations and strengths, which have been discussed in the individual article included in this supplement. The conceptual framework was developed based on peer-reviewed articles only and did not include the grey literature, book or book chapters that may have used implementation science frameworks. The review of other sourced frameworks may be worthwhile to identify any new dimensions or components of implementation science that may benefit the comprehensiveness of the conceptual framework. The concurrent evaluation was the main source of data for all the articles included in this supplement, and the evaluation did not consider an appropriate comparison group, which limits our ability to measure counterfactuals of the programme outcomes that we measured. The strength of this research includes the use of multi-methods evaluation data, which allowed us to comprehensively assess programme implementation and outcome. Concurrent evaluation is a flexible strategy; it permits fine-tuning of the evaluation activities during implementation, adjusts with programme needs and eventually improves programme outcomes.