Historical Perspectives: Lessons from the BetterBirth Trial: A Practical Roadmap for Complex Intervention Studies.

Complex interventions—those that contain several interacting components—to improve clinical outcomes and population health are growing because of the increasing recognition that multilevel approaches are needed to solve complex problems in health systems and care delivery. (1) In maternal and neonatal health, complex interventions are needed to improve quality of care. Yet few studies of these complex interventions are powered to detect true differences in mortality because of the numbers needed for a relatively rare outcome, such as maternal mortality. Although perinatal mortality is more common, large sample sizes are still required. Studies are often powered to examine outcomes or process measures proximal to mortality, such as morbidity or complications that could lead to mortality. The BetterBirth Trial was one of the largest cluster randomized controlled trials (RCTs) to target maternal and perinatal mortality as primary outcomes. Given its scale, this trial has led to important lessons learned that could be applied to other large-scale studies with complex interventions in maternal and newborn health.Many epidemiology and biostatistics textbooks have described how to theoretically design RCTs to maximize scientific rigor. (2) However, there are few, if any, resources on how to actually implement complex interventions on a large scale, evaluate their impact, and disseminate the results. (3) Balancing an unbiased and robust study design with the reality of implementing a complex intervention in the real world can pose ongoing challenges for research teams. Some generally accepted best practices to minimize common challenges of trial implementation do exist, such as for institutional review board navigation, site selection, and stakeholder engagement. This perspective reflects on the experience of the BetterBirth Trial and provides insights around best practices of important components of trial design and implementation that may be initially overlooked.The BetterBirth Trial was a matched-pair cluster RCT designed to establish whether the BetterBirth intervention was effective in reducing early (7-day) maternal mortality and morbidity and perinatal mortality in Uttar Pradesh, India. (4) The BetterBirth intervention was developed to improve the quality of intrapartum care through the World Health Organization (WHO) Safe Childbirth Checklist (WHO Checklist) and accompanying implementation strategy focused on facility leadership buy-in, staff engagement, peer-to-peer coaching, and data feedback. The trial included a total of 120 primary-level facilities and enrolled approximately 160,000 woman-newborn pairs. While adherence to essential birth practices increased among birth attendants in intervention facilities, there was no impact on maternal mortality and morbidity or perinatal mortality. (5) Given the scale of this trial, we propose a roadmap of practical lessons learned to mitigate or plan for similar challenges in designing and implementing other studies around large-scale complex interventions: partnerships and collaboration, theory of change (ToC), formative work and pilot testing, intervention implementation strategy, data quality assurance systems, and dissemination strategy.A foundational principle of global health research is partnership with local experts through designing, testing, and communicating results. (6) To implement a trial of this size, scope, and scale, we established a collaboration among research, implementation, and government partners, which was essential to the success of the study. Ariadne Labs, a joint innovation center at Brigham and Women’s Hospital and Harvard T.H. Chan School of Public Health, led the research in collaboration with Community Empowerment Lab and Jawaharlal Nehru Medical College in India. The Governments of India and Uttar Pradesh collaborated on study design and implementation, leading the study’s tripartite agreement for implementation. Key priorities in our collaboration were rigorous methodology, high-quality data collection and interpretation, and dissemination of results locally and globally through context-specific communications. WHO provided a global policy perspective and aided in development and early testing of the WHO Safe Childbirth Checklist, and Population Services International implemented the study in Uttar Pradesh. The trial funder, the Bill and Melinda Gates Foundation, participated in decisions about study location and design, but did not provide input on data collection, management, analysis, or interpretation of the data. Their partnership was important for understanding the funding landscape and learning about other projects in Uttar Pradesh that may have influenced the BetterBirth Trial. In addition, we established a scientific advisory committee of researchers, clinicians, and public health professionals that provided oversight for the study and supported interpretation of the results.A ToC is a “theory of how and why an initiative works, which can be empirically tested by measuring indicators for every expected step on the hypothesized causal pathway to impact.” (7) This is often represented as a visual map of the underlying hypothesis behind the research question for complex interventions and program evaluations. The BetterBirth Trial had a basic ToC: the BetterBirth intervention (WHO Checklist implemented through facility engagement, peer coaching, and data feedback) would lead to increased adherence with the 28 essential birth practices listed in the WHO Checklist, and this behavior change would lead to improved maternal and newborn outcomes (Fig). This ToC, however, was overly simplified and did not account for every expected step between the increased adherence to practices and decreased mortality and morbidity.Nearly all complex intervention studies require a phase of formative work or background research to understand the relevant contextual factors for the study site(s). This phase may occur while developing the grant proposal and may continue through the beginning of the grant period. Subsequent pilot testing may take place depending on the stipulations of the grant. In preparation for the BetterBirth Trial, formative work was done with a diverse group of experts and stakeholders around developing the items on the WHO Checklist. (8)Once the content of the WHO Checklist was established, a pilot pre-post intervention study was conducted in Karnataka, India, in 2010, which demonstrated improvement from an average of 10 of 29 essential birth practices to 25 of 29 after implementation of the WHO Checklist. (9) This pilot study provided promising results that supported the first part of the underlying ToC at a subdistrict-level hospital in Karnataka, India. However, the BetterBirth intervention needed to be adapted for primary-level facilities in Uttar Pradesh. The BetterBirth team revised the intervention through an adaptive design process with implementation, evaluation, and feedback in Uttar Pradesh. (10) An important lesson learned through this process was the importance of peer-to-peer coaching instead of physician-led coaching to support behavior change in birth attendants.Complex interventions developed for large-scale trials or evaluations require clear implementation strategies so that they are implemented in a standardized way. The BetterBirth team developed an implementation guide that outlined the distinct phases of intervention implementation: Engage, Launch, and Support. (11) First, the Engage phase included gaining commitment from facility, district, and state leadership for implementing the BetterBirth intervention. This phase also included adaptation of the WHO Checklist to Uttar Pradesh Ministry of Health guidelines. Second, the Launch phase included a motivational event to introduce the checklist to the birth attendants and assess existing quality gaps at the facility. We used locally produced videos and flipbooks to support the Launch event. (12)(13) Last, the Support strategy included a tapered 8-month peer-to-peer coaching program to encourage adoption of practices in the checklist and resolve barriers, and a data feedback structure through which coaches shared visual observation charts with facility and district staff to foster change. To facilitate sustainability, local facility champions were identified to support continued use of the WHO Checklist beyond the trial period.The BetterBirth Trial achieved high ascertainment of health outcomes through a call center that successfully followed up with the majority of women (89,858/91,770, 97.9%) in addition to field workers who followed up with the remaining 1.6% (n=1,447). Only 0.5% (n=465) of enrolled women were lost to follow-up. (14) The Data Quality Monitoring and Improvement System was a robust, multicomponent system that ensured high-quality data throughout the BetterBirth Trial. The 5 components were: 1) monitoring and evaluation expert team who supported data quality assurance, 2) standard operating procedures for data collection, 3) training for data quality, 4) electronic data collection and reporting system, and 5) data quality assurance protocol, including data collection audits, rapid data feedback, and supportive supervision. (15)Once preliminary data analyses were complete, the BetterBirth team worked through a systematic process to refine analyses and develop key messages from the findings. First, the BetterBirth team presented the results among the internal partners, including the governments of India and Uttar Pradesh, Ariadne Labs, the Scientific Advisory Committee, and the Bill and Melinda Gates Foundation. We also partnered with the communications specialists at Ariadne Labs to develop a dissemination strategy for the results of the trial. Study participants, including facility and district staff and leadership, learned about the results at district-level meetings across Uttar Pradesh. Then, the BetterBirth team disseminated the results globally and targeted diverse audiences, such as academics, program implementers, and clinicians. Some examples include a keynote address and panel discussion at the Royal College of Obstetricians and Gynecologists World Congress; peer-reviewed publications; grand rounds and lectures at universities; podcasts and webinars; media interviews and blogs; and donor reports. In addition, the BetterBirth team worked with WHO to build a checklist collaborative between 2012 and 2015. This collaborative included 34 groups in 29 countries that implemented the WHO Checklist in 234 sites. Some of these groups conducted feasibility and evaluation studies of their adaptation and implementation of the WHO Checklist (16) and trial results were shared with this group through a technical consultation in Sri Lanka. (17)We believe this roadmap of lessons learned in the BetterBirth Trial can aid the design, implementation, and dissemination of future large-scale studies of complex interventions. Recommendations include partnering with multidisciplinary collaborators, developing a theory of change, investing in formative work and pilot testing, building an intervention implementation strategy, creating data quality assurance systems, and partnering with communications specialists in a dissemination strategy. It is important for the research community to consolidate lessons learned from other similar trials and use (or create if needed) the standard frameworks for these large-scale trials with complex interventions so that best practices are identified and implemented. One example of such a framework is the Standards for Reporting Implementation Studies, which consist of a checklist for reporting key elements from the intervention and implementation strategy in implementation studies. (18) With the growth of translational research between clinical effectiveness and implementation in diverse global populations, advancing the field of large-scale studies with complex interventions will require guidance around intervention design and implementation strategy.