mHealth intervention “ImTeCHO” to improve delivery of maternal, neonatal, and child care services—A cluster-randomized trial in tribal areas of Gujarat, India

Summary

Background

The coverage of community-based maternal, neonatal, and child health (MNCH) services remains low, especially in hard-to-reach areas. We evaluated the effectiveness of a mobile-phone–and web-based application, Innovative Mobile-phone Technology for Community Health Operations (ImTeCHO), as a job aid to the government’s Accredited Social Health Activists (ASHAs) and Primary Health Center (PHC) staff to improve coverage of MNCH services in rural tribal communities of Gujarat, India.

Methods and findings

This open cluster-randomized trial was conducted in 22 PHCs in six tribal blocks of Bharuch and Narmada districts in India. The ImTeCHO mobile-phone–and web-based application included various technology-based job aids to facilitate scheduling of home visits, screening for complications, counseling during home visits, and supportive supervision by PHC staff. Primary outcome indicators were a composite index calculated based on coverage of important MNCH services and coverage of at least two home visitations by ASHA within the first week of birth. Primary analysis was intention to treat (ITT). Generalized Estimating Equation (GEE) was used to account for clustering. Eleven PHCs each were randomly allocated to the intervention (280 ASHAs, population: 234,134) and control (281 ASHAs, population: 242,809) arms. The intervention was implemented from February, 2016 to January, 2017. At the end of the implementation, 6,493 mothers were surveyed. Most of the surveyed women were tribal (5,571, 85.8%), and reported having a government-issued certificate for living below poverty line (4,916, 75.7%). The coverage of at least two home visits within first week of birth was 32.4% in the intervention clusters compared to 22.9% in the control clusters (adjusted effect size 10.2 [95% CI: 6.4, 14.0], p < 0.001). Mean number of home visits within first week of birth was 1.11 and 0.80 for intervention and control clusters, respectively (adjusted effect size 0.34 [95% CI: 0.23, 0.45], p < 0.001). The composite coverage index was 43.0% in the intervention clusters compared to 38.5% (adjusted effect size 4.9 [95% CI: 0.2, 9.5], p = 0.03) in the control clusters. There were substantial improvements in coverage home visits by ASHAs during antenatal period (adjusted effect size 15.7 [95% CI: 11.0, 20.4], p < 0.001), postnatal period (adjusted effect size 6.4, [95% CI: 3.2, 9.6], p <0.001), early initiation of breastfeeding (adjusted effect size 7.8 [95% CI: 4.2, 11.4], p < 0.001), and exclusive breastfeeding (adjusted effect size 13.4 [95% CI: 8.9, 17.9], p < 0.001). Number of infant and neonatal deaths was similar in the two arms in the ITT analysis. The limitations of the study include potential risk of inaccuracies in reporting events that occurred during pregnancy by the mothers and the duration of intervention being 12 months, which might be considered short.

Conclusions

In this study, we found that use of ImTeCHO mobile- and web-based application as a job aid by government ASHAs and PHC staff improved coverage and quality of MNCH services in hard-to-reach areas. Supportive supervision, change management, and timely resolution of technology-related issues were critical implementation considerations to ensure adherence to the intervention.

Trial registration

Study was registered at the Clinical Trial Registry of India (www.ctri.nic.in). Trial number: CTRI/2015/06/005847. The trial was registered (prospective) on 3 June, 2015. First enrollment was done on 26 August, 2015.

Keywords:

Neonates – Neonatal care – pregnancy – Surveys – Child health – Health services administration and management – Infants – Web-based applications

Introduction

Every year, there are 303,000 maternal deaths and 5.4 million child deaths worldwide [1,2]. Many of these deaths are preventable by increasing coverage and quality of proven, cost-effective maternal, neonatal, and child health (MNCH) services; quite a few are community-based services delivered by frontline health workers [3–5]. Mobile-phone-technology–based health (mHealth) solutions are promising, innovative strategies with the potential to increase effective coverage of some of the MNCH services through improving performance of frontline health workers in controlled settings on a limited scale [6–8]. However, there is a lack of good-quality evidence about the effectiveness of mHealth solutions implemented within the public health system towards improving the performance of frontline health workers throughout the continuum of care for MNCH, especially among hard-to-reach populations having poor coverage of services using robust implementation research methodologies [9–11].

India has made significant progress through its well-established primary healthcare system to deliver community-based services. A Primary Health Center (PHC) is a second-level health facility providing a range of curative and preventive primary healthcare services, covering populations of 20,000–25,000 in tribal areas, and is at the core of India’s primary healthcare system [12]. India, because of its large population, has a huge burden of malnutrition and maternal and child mortality [13–15]. The government of India introduced a cadre of village-based health workers, Accredited Social Health Activists (ASHAs), in 2005 to improve coverage of preventive MNCH services [16]. Some evidence-based community-based MNCH services delivered or facilitated by ASHAs through home visits and community mobilization activities include promotion to seek Antenatal care (ANC), iron and folic acid supplementation for pregnant women, promotion of facility-based delivery by skilled birth attendants, home visits for postnatal care of mothers and newborns, promotion of exclusive breastfeeding for first six months of life, immunization and complementary feeding, vitamin A supplementation, and identification of common childhood illnesses like fever, diarrhea, and pneumonia and their management [3,4,17].

In hard-to-reach and tribal areas, the performance of ASHAs and coverage of MNCH services have been reported as suboptimal, mainly because of inadequate training and insufficient supportive supervision and motivation [16,18–21]. To address these issues, we developed an mHealth application that consisted of a variety of technology-based job aids for the different levels of health workers involved in the delivery of health services. We called it "Innovative Mobile-phone Technology for Community Health Operations" (ImTeCHO) [22,23].

The objective of our trial was to implement and evaluate the effectiveness of ImTeCHO in tribal and rural areas of Gujarat to increase coverage of community-based MNCH services [23]. We hypothesized that ImTeCHO would lead to improved supervision, support, and motivation of ASHAs and PHC staff, resulting in improved coverage of proven MNCH services, including recommended number of home visits by ASHAs within first week of delivery and a composite outcome consisting of services across the continuum of care.

Methods

The details about development of ImTeCHO and protocol of the study (S1 Text) has been already published earlier [22,23]. We provide some salient features below.

Study design and setting

This open cluster-randomized implementation research trial was conducted within the existing public health system involving 22 PHCs serving a total population of 4,76,943 in predominantly tribal areas in the southern part of Gujarat, India. Eleven PHCs were randomized to the intervention and 11 to the control arm. Each PHC served a population of approximately 22,000. Cluster randomization was chosen because this was a complex intervention to be delivered at the community level. PHCs were selected as a cluster to reduce contamination because participants of the trial included ASHAs and staff of the PHC. Each PHC had a team of two medical doctors/officers, one Auxiliary Nurse Midwife (ANM), and one male multipurpose worker for every 4,000 to 6,000 people; one female and male health supervisor and one ASHA for approximately 1,000 people; and administrative staff. An ASHA is a female village-based frontline health worker who is a native of the village, has at least eight years of formal education, spends 3–5 hours working in the community every day, and is provided incentives based on performance [24]. An ANM is a qualified health provider who provides MNCH services (immunization, ANC, delivery, etc.) at the community and facility level. The medical officer is a medical doctor who leads the PHC team [12].

The study area included six blocks of the Bharuch and Narmada districts identified as high-priority blocks by the state government because of poor health and development indicators compared to other areas and relatively lower literacy of 65% and inhabited by a tribe called Bhil or Vasava [25–28].

Twenty-six PHCs (except two PHCs where ImTeCHO was implemented already as part of an earlier pilot) were eligible to be included in these six blocks, having a 100% rural population and with at least 45% of the population consisting of scheduled tribe caste. Four PHCs that had more than 10% villages without availability of internet through mobile data were excluded, considering the study’s need for at least a few minutes every day for intervention to work.

This study was conducted by SEWA Rural, which is a nongovernmental, voluntary grassroots organization located at the border of the study area in active partnership with the Department of Health and Family Welfare, Government of Gujarat, and an information technology (IT) partner, Argusoft India Ltd. The Department of Health and Family Welfare, Government of Gujarat provided written approval and formal notifications for conducting the study within existing public health system, and one of the senior state-level officers joined the team of investigators. No major changes were made to the design of the study after its commencement.

Interventions

We used Medical Research Council (Swindon, UK) framework and user-centered design to develop and evaluate a complex health intervention in the form of a mobile-phone-technology–based job aid for the ASHAs and PHC staff with an aim to make their work easier and more effective [29]. A formative evaluation, as part of a pilot, was done in 45 villages in 2014 in a location outside the study area [22]. An online video is available that provides a short demonstration of the intervention [30].

Table 1 shows various mHealth strategies included in the application, and S1 Table shows details of service delivery in the intervention and control arms [23]. ASHAs were given Samsung Galaxy Star Pro (GT-S7262) android smart phone (cost US $86) with a postpaid data plan costing US $3 per month per ASHA. ASHAs registered all existing and new pregnant women and children under the age of two years in their villages using the ImTeCHO mobile phone application throughout the study period. Subsequently, the software prepared an entire schedule of home visits as per the national standards and sent reminders to ASHAs. Every day, ASHAs logged in to the application to review their schedule, conducted a home visit based on the task list, and completed and submitted a digital form during the home visits to close out the scheduled task. To emphasize key health and wellness messages, ASHAs showed short video clips on the mobile phone to family members during the home visits. The digital forms had checklists to screen for complications. The application had a decision support system to show probable diagnosis with risk stratification to ASHAs based on the entries made in the digital forms and a customized management plan that included tools to call emergency transport vehicles and suggested home-based remedies along with names and doses of drugs in compliance with national standards for the ASHA program. Additionally, ASHAs recorded details of the services, including immunization, delivered by the ANMs during monthly Village Health and Nutrition Days (VHNDs) in the ImTeCHO application as well. Based on the data entered by an ASHA throughout a month, her performance report was generated and shared with each ASHA on the first day of the subsequent month through the ImTeCHO application, and performance-based incentives were calculated automatically to ensure timely payment. The data transfer happened through the internet; however, data entry was possible even offline. Each medical officer was provided with a Samsung Galaxy Tab3V (cost US $165) tablet to use the ImTeCHO web interface. The medical officers and PHC staff used the web interface to track high-risk cases, view reports, and manage incentives and supplies.

Complete enumeration of study population and the preintervention baseline survey in the study area was conducted from February, 2015 to July, 2015 [23]. SEWA Rural and government staff conducted training using audiovisual training tools and hands-on practice in groups from August, 2015 to January, 2016. The intervention was implemented from February, 2016 to January, 2017. The postintervention endline survey was conducted from February, 2017 to July, 2017, and its methodology was published as part of the protocol [23].

Control arm (comparator)

The government and other providers continued to provide usual health services in the control area. A refresher training (three days) and one-time supply of commodities were provided to ASHAs from the control and intervention area [23].

Participants

All health providers (ASHAs, ANMs, medical officers, and PHC support staff) belonging to PHCs in the intervention and control arms were the participants of the study. All pregnant women, neonates, and infants from study area were the participants. The mHealth facilitator and helpline (description in S1 Table) were only present in the intervention arm. The mHealth facilitators (one per 50 ASHAs) and one counselor at helpline were part of the project team from SEWA Rural [23].

Management

The project team stationed at the headquarters included an implementation coordinator and project associate who supervised the team of mHFs and helpline counselor. It intervened in cases of emergencies, organized a training at the beginning of the trial, paid additional token monetary incentives to intervention ASHAs (ranging from US $4 to $11 monthly per ASHA based on performance), managed SIM cards, and coordinated with partner organizations.

The project team did not conduct community mobilization, regular field-level supervision, or separate meetings with ASHAs other than participating in regular monthly PHC meetings. In addition, they also did not deliver any health services at the community level or provide any other kind of support to ASHAs not listed in S1 Table or ongoing provision of commodities after initial training.

The district- and state-level government officers actively participated in proposal development workshops, provided feedback about the software, participated in regular review meetings, and led implementation jointly with project management team. The IT partner carried out regular software maintenance and ensured timely resolution of any related issues on an ongoing basis.

Outcomes

The primary and secondary outcomes of interest were coverage of various health services to be delivered or facilitated by ASHAs, which were expected to change as a result of ImTeCHO intervention, along with process indicators to measure fidelity to the intervention. The two primary outcomes of interest were (i) the proportion of neonates/mothers visited at home by ASHA at least two times within one week after delivery, and (ii) modified ASHA-centric composite coverage index (MACCI). The details of MACCI have been published elsewhere [23]. See S2 Text for the definition and rationale for the choice of primary outcomes of interests.

The ImTeCHO program data were used to obtain results of the process indicators to assess adherence to the intervention. The definitions of process indicators and secondary outcomes of interest were included in the published protocol [23]. There were no changes in trial outcomes after its commencement.

Recruitment of respondents, and measurement of outcomes

An endline household survey was conducted from February, 2017 to July, 2017 to measure the primary as well as secondary outcomes of interest. The household survey tools used standard methodology for the district-level health and facility survey (DLFHS-4) [31]. There were two kinds of respondents.

• For maternal and newborn health outcomes (respondent type A): All women who were natives of the study villages and were mothers of a one- to four-month–old infant at the time of survey were eligible. Women whose infants died before the survey were excluded.

• For child health outcomes (respondent type B): All women who were natives of study villages and were mothers of a six- to eight-month–old infant at the time of survey were considered eligible. Women whose infants died before the survey were excluded.

All pregnancies, their outcomes, and infant and maternal deaths from the entire study area were counted prospectively from 2016 onwards as part of ongoing pregnancy and mortality surveillance. The information from pregnancy registration and mortality surveillance system was used to identify and enroll type A and type B respondents for the endline survey.

The data were entered into an in-house–developed data entry software, different from the ImTeCHO application, through smart phones that had inbuilt provisions to avoid illogical data entries and missing data. A data administrator checked for any inconsistencies, which were clarified with data collectors and corrected. The endline survey was conducted jointly by an independent evaluation team comprising trained data collectors, who worked separately and were not involved in the implementation activities, and an independent research organization, Population Research Centre (PRC), Vadodara, Gujarat. PRC validated data on a randomly selected 3% of respondents through household survey using a truncated questionnaire.

Sample size

We calculated sample size for both the primary outcomes of interest and used the larger one. Eleven PHCs each in the intervention and control clusters were included to address both questions [23]. Based on the results of an evaluation of an earlier pilot conducted prior to the trial, we assumed that ASHAs would visit 46% of neonates/mothers at least twice at home within first week of delivery and MACCI would be 36% in control area [32]. We assumed there would be 25 ASHAs associated with one PHC. We assumed intraclass correlation (ICC) to be 0.02 in light of no existing information available regarding ICC. We estimated that there might be loss of one cluster per arm and three ASHAs per PHC over the course of the study period. For a power of 80% and 5% two-sided significance level, we estimated that six PHCs/clusters (150 ASHAs) per arm were required to detect 20% absolute improvement in the proportion of neonates/mothers who received at least two postnatal home visits within first week of delivery by an ASHA in the intervention arm compared to the control arm at endline survey. Similarly, we estimated that 11 PHCs/clusters (275 ASHAs) per arm were required for detecting 15% absolute improvement in MACCI in the intervention arm compared to the control arm at endline survey.

Randomization

An independent senior statistician not involved in study implementation carried out randomization and assigned 11 PHCs to the intervention and 11 to the control arm. MACCI was calculated for each PHC after the baseline survey; the mean value of MACCI was calculated, which was then used as a cutoff to allocate each PHC into one of the two strata. The PHCs in each of the two strata were randomly allocated to the intervention and control groups in a 1:1 ratio to ensure that MACCI was similar at the baseline for the intervention and control arms using the software nQuery. Blinding was not possible because this was a community-based intervention. All pregnant women and infants identified through complete enumeration and identification of new pregnant women along with their newborn babies throughout the study period in the study area were eligible to participate.

Ethical review

The Multi-institutional Ethics Committee (Mumbai), Institutional Ethics Committee of SEWA Rural, and Ethics Review Committee of WHO approved the study. All respondents provided written informed consent to trained data collectors prior to the household survey. Every respondent who participated in the endline survey was assigned a unique identification number, and personally identifiable information was removed. Standard security and data encryption practices were used by the ImTeCHO application system for accepting, transmitting, processing, and storing data.

The results were reported in accordance with CONSORT statement for reporting cluster-randomized trials (see S2 Table).

Statistical methods

Primary analysis was intention to treat (ITT) and secondary analysis was per protocol (PP) at the ASHA level. A Generalized Estimating Equation (GEE) was used to account for clustering and to adjust for maternal age, caste, parity, poverty status, and education. For ITT analysis, all enrolled respondents were included in the analysis irrespective of whether they received the treatment/intervention or not. However, there is a common local custom for a large proportion of pregnant women to move out of their own homes and spend the last trimester and first few weeks after delivery at their maternal village, which might be out of the study cluster and might not have been completely exposed to the treatment arm. Therefore, PP analysis was done for indicators related to antenatal and postnatal services (respondent type A).

Two-level random-effects regression models were applied for each of the primary and secondary outcomes. For continuous and categorical outcomes, cluster-level means and proportion were used, respectively. In the case of proportion, the mean of proportion by cluster was calculated and reported. Analysis was done at the ASHA level. The differences with effect size and p-value for each outcome were also reported. The effect size was adjusted for maternal age, caste, parity, below poverty line (BPL) card status, and women’s education level. The denominator for outcomes related to pregnancy and postnatal services included respondents of type A. The denominator for outcomes related to postneonatal services included respondents of type B. STATA 13.0 was used for data analysis.

Trial registration

This trial was registered with the Clinical Trial Registry of India (www.ctri.nic.in, registration number CTRI/2015/06/005847, date of registration 3 June, 2015).

Data safety monitoring board

The data safety and monitoring board, comprising one scientist, one ethicist, one community representative, and one statistician, monitored the trial during annual meetings.

Results

Fig 1 shows trial profile. The 11 randomly allocated intervention clusters from six tribal blocks had a population of 234,124 with 280 ASHAs, and the 11 control clusters had a population of 242,803 with 281 ASHAs at baseline. All 22 clusters were followed until the end of the trial. 9,334 new pregnancies and 8,230 live births were recorded in the study area during the implementation period from February, 2016 to January, 2017.

During the baseline survey, 1,419 respondents were interviewed (Table 2). The sociodemographic profile, coverage of MNCH services, and cluster-level characteristics were largely similar in the intervention and control arms at the time of the baseline survey in 2015 except for caste (79.0% women from scheduled tribe in intervention clusters versus 87.3% in control), place of delivery (73.9% women delivered at a hospital in the intervention clusters versus 85.0% in control), low standard of living index (37.6% in intervention versus 28.6% in control), and at least two home visits within first week (19% in the intervention clusters versus 14% in control). There were more vacancies in ASHA positions at baseline in the control clusters (7.5% vacancies in control clusters versus 5.0% in intervention clusters). All 11 PHCs in the control clusters had a functional delivery facility compared to seven PHCs in the intervention clusters.

Throughout of the study period, uptake and adherence to the ImTeCHO mobile phone application was satisfactory among ASHAs (Table 3). The ASHAs logged into the ImTeCHO mobile phone application on 85% of days and marked 79% of all scheduled home visits as complete using the application. However, the PHC staff and PHC medical officers’ use of ImTeCHO web interface was less than expected (Table 3). Throughout the study period, 17 mobile phones (285 ASHAs) were lost, stolen, or irreparably damaged. 96 software issues were escalated to and addressed by Argusoft India Ltd over 12 months. Three out of 11 PHCs disbursed ASHA incentives as per the ImTeCHO system. The incremental cost per birth was US $54. The results of the cost-effectiveness study, nested within this trial, will be reported in a separate publication.

At the time of the end line survey, 3,023 type A respondents and 3,470 type B respondents were interviewed and analyzed. Most of the respondents were tribal (5,571, 85.8%), and reported having a government-issued certificate for living BPL (4,916, 75.7%). 868 (13.3%) women had not received any formal education. All 22 PHCs (578 ASHAs) were included in the analysis. The proportion of neonates who were visited at least twice during the first week of delivery and MACCI was significantly higher in the intervention arm compared to the control arm in both ITT and PP analysis (Table 4). In ITT analysis, the coverage of at least two home visits within first week of birth was 32.4% in the intervention clusters, compared to 22.9% in the control clusters (adjusted effect size 10.2 [95% CI: 6.4, 14.0], p < 0.001). There were baseline differences in the place of delivery and coverage of at least two home visits within the first week of birth. After adjusting for baseline differences in place of delivery, the coverage of at least two home visits within the first week of birth remained significantly higher in the intervention clusters compared to the control clusters (adjusted effect size 10.2 [95% CI: 6.4, 13.9], p < 0.001). Similarly, the coverage of at least two home visits within the first week of birth remained significantly higher in the intervention clusters compared to the control clusters (adjusted effect size 6.6 [95% CI: 2.5, 11.7], p < 0.001) after adjusting for baseline differences for this primary outcome. The mean number of home visits within the first week of birth was 1.11 and 0.80 for the intervention and control clusters, respectively (adjusted effect size 0.34 [95% CI: 0.23, 0.45], p < 0.001). The mean number of home visits within the first month of birth was 2.70 and 2.00 for the intervention and control clusters, respectively (adjusted effect size 0.75 [95% CI: 0.47, 1.04], p < 0.001). The composite coverage index was 43.0% in the intervention clusters compared to 38.5% (adjusted effect size 4.9 [95% CI: 0.2, 9.5], p = 0.03) in the control clusters. Table 5 reports the results of secondary outcome indicators. Exclusive breastfeeding, home visitations during antenatal period, satisfactory counseling during antenatal period, early initiation of breastfeeding, ASHAs’ visitation at home within 24 hours of delivery (in case of home delivery) or within 24 hours of return to home from hospital in case of hospital delivery, satisfactory counseling during postnatal period, home visitations during postnatal period, newborn examination during home visits, and administration of Oral Rehydration Solution (ORS) in case of diarrhea were significantly higher in the intervention clusters compared to the control clusters. Care-seeking for antenatal and neonatal complications was also significantly higher in the intervention clusters compared to the control clusters. However, there were no differences in the rates of full ANC provision by the ANM, facility delivery, care-seeking from ASHA for pneumonia, and Diphtheria, Pertussis, Tetanus-3 (DPT-3) vaccination coverage. The concordance rate of responses between the data collectors and team of independent quality assessors from PRC was 89% during the endline survey.

There were 4,171 live births in intervention clusters and 4,059 in control clusters during 1 year of the study period. There were 233 infant deaths in intervention clusters and 236 in control clusters. 90 stillbirths occurred in intervention clusters, and 107 occurred in control clusters. The number of neonatal deaths was 104 in intervention clusters and 102 in control clusters.

Discussion

The coverage and quality of many of the MNCH health services improved in the intervention arm compared to the control arm. Additionally, the mHealth strategies provided support to the ASHAs and encouraged them to adhere to protocols. The targeted client communication in the form of short video clips were found to be effective. The strategies involving longitudinal digital tracking and scheduling resulted in an improvement in the coverage of recommended number of home visits by ASHAs during antenatal and postnatal periods. Furthermore, there was an improvement in the quality of home visits with regards to behavior change communication and clinical examination, early initiation of and exclusive breastfeeding, care-seeking from ASHAs for antenatal and neonatal complications, and use of ORS for diarrhea. The uptake of the ImTeCHO intervention was satisfactory among ASHAs as reflected in high login and task completion rate, whereas it was lower than expected among the PHC staff. As expected, ImTeCHO intervention was not found to be effective towards increasing coverage of MNCH health services in which the ASHA was not the primary provider and relied on other cadres or infrastructure of the health system (such as full ANC, vaccination, or institutional delivery).

Contribution of study to existing literature

To our knowledge, this is one of the first randomized controlled trials of its kind that assessed effectiveness of multiple mHealth strategies implemented in a low- and middle-income country (LMIC) through frontline health workers in an existing public health system throughout the continuum of care to improve a wide range of MNCH outcomes among hard-to-reach populations using a robust research methodology. The ImTeCHO intervention targeted the entire scope of potential users of mHealth-based solutions, including health providers, beneficiaries, and health system managers. Published data have shown improvement in outcomes related to only one phase of RMNCH life cycle or use of only one mHealth strategy (most commonly Short Message Service [SMS]-based) or in controlled settings outside a government-supported public health system and targeted either beneficiary or health providers [10,33]. A study in rural Ethiopia, which did not involve random allocation of treatment, concluded that mHealth strategies, in the form of reminders through SMS to frontline health workers, resulted in an increase in the coverage of antenatal visits and facility delivery [34]. A randomized trial in Tanzania, conducted in a controlled setting by a nongovernmental organization, reported that mHealth strategies implemented during the antenatal period resulted in an increase in facility delivery [8]. Another randomized trial conducted in Bihar, India using similar mHealth strategies reported significant improvement in the coverage of a wide range of MNCH services delivered by frontline health workers; however, there were intensive efforts to strengthen the health system in addition to the deployment of the mHealth intervention [35]. Systematic reviews have stated a low quality of evidence and advocated randomized trials using robust research methodologies in uncontrolled settings [10,36–38]. Randomized trials in Ethiopia and Democratic Republic of Congo reported that a safe delivery mobile phone application as a training tool for health workers resulted in significant improvement in health workers’ knowledge and skills about neonatal resuscitation [39, 40].

ASHAs and PHC staff faced some challenges to adopt ImTeCHO on a consistent basis: first, to balance competing priorities in the context of various other programs. The uptake of ImTeCHO web application among PHC staff was less than expected, especially in light of ImTeCHO not being mainstreamed in the public health system in the state at the time of study and other competing priority public health programs; second, to deal with the perception of increase in workload due to introduction of ImTeCHO; third, to overcome technology problems involving internet coverage, hardware issues with mobile phones, and resolving bugs in the software in a timely manner. ASHAs and PHC staff from eight intervention PHCs did not use ImTeCHO’s incentive management system because they were concerned that digitization of incentive calculation might take away the flexibility of a paper-based system and might reduce the amount of incentive they might get. We established systems and processes to overcome these challenges by ensuring health system preparedness, effective change management, training, supportive supervision, motivation, and timely technology support. In addition to the existing components of ImTeCHO, the use of automated voice calls for targeted client communication, verification of data quality, digitization of birth and death registration, and expansion of the user base to involve other cadres of frontline health workers may help us further improve coverage of health services [41].

Study strengths and limitations

Our study had a few strengths. First, we aligned programmatic objectives (i.e., increasing coverage of MNCH services) with the personal priorities of ASHAs (e.g., calculation of performance-based incentives using ImTeCHO for timely disbursement). Second, we took health workers’ legitimate concerns into consideration about the reasons for their suboptimal performance and used technology as a job aid (not job add) to make their tasks more efficient, effective, and easy. These approaches resulted in a higher uptake of technology-based interventions and improved coverage of MNCH services. Finally, we identified critical operational requirements that improved adherence to the intervention; this included supportive supervision, timely resolution of technology problems, and change management, including monetary and/or nonmonetary incentives depending on the context. Such operational requirements could be an integral part for an mHealth program to succeed at scale.

Our study also had some potential limitations. First, the outcomes were reported by the mothers, and there was a risk of recall bias. The mean number of days between date of birth and date of questionnaires completed during the endline survey was 116 days and 112 days for the intervention and control areas, respectively. However, standard data collection methodologies were used, and the methodology was the same across the study arms to negate any problems with recall. There was high concordance (89%) between the data collected by the data collectors and the team of independent quality assessors. Second, there was a higher proportion of women from the scheduled tribe and from the lowest standard of living index in the intervention arm. These differences were adjusted in the final model. Finally, the total duration of the intervention was 12 months, which might be considered short. The choice of duration of the intervention was affected by experiences of the earlier pilot, logistic concerns related to expected events in future such as upcoming state elections making survey activities difficult, and possible reorganization of PHCs resulting in reorganization of clusters. Some of the indicators showed improvement over the course of study period even in the control clusters (at least two home visits by ASHA during first week of delivery improved from 14.0% at baseline to 22.9% at endline). This might indicate overall strengthening of ASHA performance, and additional benefits from the intervention could be viewed in this context.

Conclusions

In conclusion, our data show that mHealth strategies can improve the coverage of proven MNCH services, especially in hard-to-reach populations, if there is adequate supportive supervision, change management, and ongoing technology assistance to ensure satisfactory adherence to the intervention. Encouraged by the results of the study and the demonstrated feasibility in the existing public health system, the Gujarat state government is now leading the scale-up of the modified ImTeCHO intervention in the entire state. The scope of the intervention is being expanded to include other health domains and health cadres to prepare one integrated health IT platform. Many of the activities conducted by the project team are being taken over by another nonprofit organization, GVK EMRI, which is currently providing free emergency ambulance services across the state as part of a public–private partnership, thus improving the chances of a successful scale-up while minimizing the dilution of effectiveness of the intervention [42]. We recommend that it would be very useful to evaluate the scale-up after a longer period of implementation.

Supporting information

S1 Table [docx]
Service delivery by study arm.

S2 Table [docx]
CONSORT 2010 checklist of information to include when reporting a cluster-randomized trial.

S1 Text [doc]
Protocol.

S2 Text [docx]
Definition and rationale for primary outcomes.