Net Benefits: A Multicountry Analysis of Observational Data Examining Associations between Insecticide-Treated Mosquito Nets and Health Outcomes

Summary

Background:
Several sub-Saharan African countries have rapidly scaled up the number of households that own insecticide-treated mosquito nets (ITNs). Although the efficacy of ITNs in trials has been shown, evidence on their impact under routine conditions is limited to a few countries and the extent to which the scale-up of ITNs has improved population health remains uncertain.

Methods and Findings:
We used matched logistic regression to assess the individual-level association between household ITN ownership or use in children under 5 years of age and the prevalence of parasitemia among children using six malaria indicator surveys (MIS) and one demographic and health survey. We used Cox proportional hazards models to assess the relationship between ITN household ownership and child mortality using 29 demographic and health surveys. The pooled relative reduction in parasitemia prevalence from random effects meta-analysis associated with household ownership of at least one ITN was 20% (95% confidence interval [CI] 3%–35%; I² = 73.5%, p<0.01 for I² value). Sleeping under an ITN was associated with a pooled relative reduction in parasitemia prevalence in children of 24% (95% CI 1%–42%; I² = 79.5%, p<0.001 for I² value). Ownership of at least one ITN was associated with a pooled relative reduction in mortality between 1 month and 5 years of age of 23% (95% CI 13–31%; I² = 25.6%, p>0.05 for I² value).

Conclusions:
Our findings across a number of sub-Saharan African countries were highly consistent with results from previous clinical trials. These findings suggest that the recent scale-up in ITN coverage has likely been accompanied by significant reductions in child mortality and that additional health gains could be achieved with further increases in ITN coverage in populations at risk of malaria.

: Please see later in the article for the Editors' Summary

Introduction

Several sub-Saharan African countries, with support from international donors, have rapidly scaled up the fraction of households that own insecticide-treated mosquito nets (ITNs) from essentially zero to above 60% over the last decade [1]. Although there has been variable progress across countries, the push to increase ITN coverage continues with more dramatic improvements seen in the last few years [2].

The large expansion in the distribution of ITNs has been motivated by evidence from cluster-randomized controlled trials (RCTs) that showed pooled relative reductions in child mortality of 18% [3] and parasite prevalence of 13% as a result of net use [4]. There are several reasons why improvements in health outcomes of the same magnitude might not be observed under routine conditions [5]. These include, for example, reduced net integrity and improper use. As a result, efforts should be made to measure not only the coverage of ITNs, but also their impact on health outcomes under real-world settings [6],[7].

Evaluating the impact of malaria control strategies, including the scale-up of ITNs on health outcomes, is difficult. Weak routine health information and vital registration systems mean that it is often not possible to accurately determine malaria-specific mortality and morbidity. Evidence about the impact of ITNs under routine conditions has been limited to selected studies such as those conducted in rural Kenya [8], the Gambia [9], Tanzania [10],[11], and rural Somalia [12]. These studies, however, have used different approaches to assess the relationship between ITNs and health outcomes and represent only some of the countries where ITN coverage has been scaled up.

In this paper, using routinely collected household surveys, we demonstrate an approach to measure in a comparable way the association between use and ownership of ITNs and parasitemia prevalence and child mortality across a large number of countries where ITNs have been distributed. This method quantifies the impact of ITNs, under routine conditions, to allow a better understanding of the effect on child health of the recent ITN scale-up.

Methods

Data

We considered all demographic and health surveys (DHS) and malaria indicator surveys (MIS) from sub-Saharan Africa countries conducted since 2000 for which the unit-record data were available. Prior to 2000, ITN ownership and use in sub-Saharan Africa was universally low [13]. We included only surveys that collected data on the health outcomes of interest (child mortality or parasitemia prevalence) as well as information on ITN ownership and use (including when the ITN was received or purchased, and when it was retreated) and all covariates specified in the analyses. We excluded the Ghana DHS conducted in 2003 as no child deaths were observed in the small number of households that owned ITNs. The results on the association between ITNs and child mortality are based on 29 DHS in 22 sub-Saharan African countries, while the results on the association between ITNs and parasitemia prevalence are based on 6 MIS and one DHS from seven sub-Saharan African countries.

Ownership and Use of ITNs

Mosquito nets were classified as conventional ITNs, which require retreatment at least every year, or long-lasting insecticide-treated mosquito nets (LLINs), which should be replaced after 3 y [14]. While the data collection procedure varied slightly across surveys, in general survey interviewers visually confirmed presence of nets in the household and recorded the following information for each net in a net roster: how long ago it was acquired; brand, specifically, if it is an LLIN; and for conventional ITNs, how long ago the net was last treated. We considered a net to be an ITN if it was an LLIN that was less than or equal to 3 y old or a conventional ITN that was less than or equal to 1 y old or had been retreated in the last year. The net roster was linked to the household roster and this was used to identify which member slept under the net the previous night.

Using this information, we estimated three variables of net ownership and use, two at the household level and one at the child (aged less than 5 y) level: (i) whether or not the household owned an ITN, (ii) how many ITNs each household owned per household member, and (iii) whether the child slept under an ITN the night prior to the survey.

Health Outcomes

Parasitemia in children under the age of 5 y of age was ascertained in surveys using a rapid diagnostic test (RDT) and/or microscopy using thick or thin blood smears. Survey data and documentation did not always indicate whether the positive result was determined from RDT or microscopy.

Survival of children from age 1 mo to 59 mo was determined from complete birth histories of women of reproductive age (15 to 49 y). We examined mortality between age 1 mo and 59 mo as this is the same age period used in RCTs and previous observational studies [4],[8]; malaria deaths in the neonatal period are very rare.

Malaria Transmission Intensity

All analyses controlled for the effect of malaria transmission intensity. To determine malaria transmission intensity, we used global positioning system (GPS) coordinates for each of the primary sampling units (PSUs) in the MIS or DHS and linked this to data on malaria transmission from the Malaria Atlas Project (http://www.map.ox.ac.uk; [15]) using ArcGIS. All households in the PSU were assigned the malaria transmission based on the PSU-level GPS coordinates. We categorized malaria transmission intensity into the following categories: (i) high transmission, defined as PfPR_2–10 or P. falciparum parasite rate (2 to 10 y) between 40%–100%; (ii) medium transmission, defined as PfPR_2–10 between 5%–40%; and (iii) low transmission, defined as PfPR_2–10 between 0%–5% [16]. Seven DHS did not have PSU-level GPS coordinates available (Benin 2006, Congo 2005, Eritrea 2002, Niger 2006, Rwanda 2000, São Tomé & Príncipe 2008, and Zambia 2001–2002). For these seven surveys, households were assigned a malaria transmission category on the basis of the average population-weighted parasite rate in the province where the household was located.

Effect of ITN Ownership and Use in Children under 5 on Parasitemia Prevalence

We examined the effect of ITN ownership and use on parasitemia prevalence using exact matching. The literature on the use of matching for causal inferences is sophisticated and growing, and includes several applications in global health and evaluations of health policies [17]–[21]. Matching provides a way of preprocessing the data so that the treated group is as similar to the control group as possible, thus making the treatment variable (in this case, ITN ownership or ITN use) as independent of the background characteristics as possible. By breaking or reducing the link between the treatment variable and the control variables, matching makes estimates based on subsequent analyses less dependent on model specification.

Within each survey, we exactly matched children who live in a household that owns an ITN or children who slept under an ITN the night prior to the survey to children from households without an ITN on the basis of the following covariates: (i) age of the child (0–1, 2–3, 4+ y); (ii) mother's education (none, any); (ii) urban/rural residence; and (iv) malaria transmission intensity. We implemented the exact matching procedure using the MatchIt software in R [22].

We then used logistic regression on the matched dataset to provide added control of potential confounders using the following covariates: (i) age of the child (0–1, 2–3, 4+ y); (ii) mother's education (none, primary, secondary or more); (iii) urban/rural residence; (iv) household wealth quintile; (v) malaria transmission intensity category; and (vi) wet or dry season at the time of the survey. We estimated household wealth using information on asset ownership [23]–[25].

A separate analysis was conducted for each survey and we determined the odds ratio (OR) associated with ITN ownership or use. We determined a pooled OR across all surveys using DerSimonian-Laird random effects meta-analysis [26].

Effect of ITN Ownership on Child Mortality

We used complete birth history data from DHS to construct a retrospective cohort that traces survival of children from age 1 mo to 59 mo for the 3 y prior to the survey. From the household net roster, using the information on when each net was acquired and/or retreated, we determined household ownership of an ITN for each month during the 3 y prior to the survey. As the surveys only record use of ITNs for children who are alive at the time of the survey, we were not able to study the relationship between ITN use and child mortality.

We analyzed the relationship between household ownership of ITNs and child mortality using Cox proportional hazards models where analysis time was the age of the child in months. We controlled for the following covariates: (i) maternal age (in 5-y age groups); (ii) parity and birth interval (less than 12 mo, 12–23 mo, greater or equal to 24 mo or first born); (iii) sex of the child; (iv) single or multiple birth; (v) maternal education (no education, less than primary, less than secondary, secondary or more); (vi) household wealth quintile; (vii) urban/rural residence; (viii) skilled birth attendance (SBA) coverage at the PSU level; (ix) three-dose diphtheria, pertussis and tetanus (DPT3) immunization coverage at the PSU-level; (x) calendar year; (xi) malaria transmission intensity; and (xii) wet or dry season specific to the month of the observation. Wet and dry seasons were determined from the Mapping Malaria Risk in Africa project (http://www.mara.org.za/).

A separate analysis was conducted for each survey and we determined the relative risk (RR) of child mortality associated with ITN ownership. We determined a pooled RR across all surveys using DerSimonian-Laird random effects meta-analysis [26]. We examined the sensitivity of the results to recall bias by restricting the analysis to observations for just the one year prior to the survey.

Effect of ITNs by Malaria Transmission Intensity, Number of ITNs Owned, and Urban and Rural Residence

Malaria transmission varies considerably within countries and it is likely that the effect of ITNs varies by transmission level. The effect of ITN ownership may also vary according to the number of ITNs owned by the household. Finally, the majority of RCTs and observational studies of ITNs were conducted in rural areas and the effect of ITNs in urban areas is less well characterized.

To test for these effects, we pooled individual observations from all surveys and grouped observations by transmission intensity (high, medium, and low), the number of ITNs owned per household member (0, <0.25 ITNs per household member, ≥0.25 ITNs per household member), and urban or rural residence. We ran separate models for each stratum. For the analysis of child mortality, we included a random effect term across surveys to capture systematic variation in the outcome across surveys. We did not include this term for parasitemia prevalence given the small number of surveys included.

All analyses were conducted in Stata 11 (Stata Corporation) and R 2.9.2 (University of Auckland).

Results

Table 1 describes the characteristics of the 29 surveys included in the analysis of child mortality; Table 2 provides information about the seven surveys included in the analysis of parasitemia prevalence. These surveys cover the majority of malaria-endemic countries from sub-Saharan Africa with varying sized populations at risk of malaria. ITN household ownership coverage at the time of the survey ranged from less than 2% to almost 60% of households.

Figure 1A shows the results of the analysis of the effect of household ownership of at least one ITN on the prevalence of parasitemia. Four countries demonstrated a statistically significant association between ITN ownership and parasitemia prevalence: Zambia with a 45% relative reduction in parasitemia prevalence (95% confidence interval [CI] 22%–61%); Rwanda with a 45% relative reduction (95% CI 7%–67%); Senegal with a 33% relative reduction (95% CI 10%–50%); and Uganda with a 29% relative reduction (95% CI, 13%–41%). Across the seven surveys, there was a significant pooled reduction in parasitemia prevalence of 20% (95% CI 3%–35%) associated with household ownership of an ITN. There was, however, significant heterogeneity in the association between ITN household ownership and parasitemia prevalence (I² = 73.5%, p<0.01). The pooled effect on the prevalence of parasitemia of children sleeping under an ITN the previous night (Figure 1B) was of a similar magnitude (relative reduction of 24%, 95% CI 1%–42%; I² = 79.5%, p<0.001) and not significantly different from the pooled effect on parasitemia of ITN ownership (p>0.05).

Figure 2 shows the results of the analysis of the effect of household ownership of at least one ITN on child mortality. In the individual surveys, there were statistically significant reductions in five surveys: Zambia 2001–2002 with a 69% RR reduction in child mortality (95% CI 24%–87%); Kenya 2008–2009 with a 68% RR reduction (95% CI 29%–86%); Rwanda 2007–2008 with a 55% RR reduction (95% CI 28%–72%); Niger 2006 with a 41% RR reduction (95% CI 14%–59%); and Madagascar 2008 with a 30% RR reduction (95% CI 2%–50%). Across the 29 surveys, there was a statistically significant pooled RR reduction in child mortality of 23% (95% CI 13%–31%) with the effect being consistent across the 29 surveys (I² = 25.6%, p>0.05 for the I² value). Restricting the analysis of ITN ownership on child mortality to observations in the 1 y prior to the survey, and thereby reducing the influence of recall bias, did not markedly change the estimated mean effect of ITN ownership (unpublished data).

Tables 3 and 4 show results of the logistic regression of ITN household ownership and use in children under-five on parasitemia by malaria transmission risk. The effect of ITN household ownership and use in children under-five were statistically the same across the three levels of transmission risk (p>0.05). In general, wet season, increasing child age, lower maternal education, and lower household wealth were significantly associated with higher odds of parasitemia (Tables 3 and 4). Table 5 shows the result of the Cox Proportional Hazards model of ITN household ownership on child mortality by transmission level. There were no statistically significant differences in the effect of ITNs on child mortality by malaria transmission level (p>0.05). In general, wet season, shorter birth intervals, a multiple birth, older maternal age, lower maternal education, lower household wealth, fewer household members, lower coverage of other childhood immunization, and skilled birth attendance were associated with higher probability of child mortality (Table 5). All the relationships observed between child mortality and parasitemia and the covariates controlled for are as expected and support the validity of the analytical approach.

We did not observe statistically significant differences in the effect of the number of ITNs per household member for either parasitemia prevalence or child mortality when stratified by transmission level (Figure 3; p>0.05). We found a statistically significant association between ITNs and child mortality in urban areas with high and medium levels of malaria transmission (Figure 4); however, we did not observe statistically significant differences in the effect of ITNs in rural versus urban areas when stratified by transmission level (Figure 4; p>0.05).

Discussion

Our findings from a large number of countries suggest that the rapid scale-up in ITN coverage observed in several sub-Saharan African countries has likely been accompanied by reductions in child mortality. Our results are also highly consistent with findings from previous RCTs. We found a 23% (95% CI 13%–31%) pooled relative reduction in child mortality across 29 surveys compared to the pooled 18% (95% CI 10%–25%) relative reduction observed in three RCTs [3]. For parasitemia, we found a 20% (3%–35%) reduction across seven surveys, which is not statistically distinguishable from the pooled 13% reduction observed in seven RCTs [4]. The lack of a major difference between the RCTs and our analysis may be partly explained by the intention to treat analysis used in RCTs, although ITN coverage in the RCTs was almost universal. It is also important to note that the RCTs targeted provision of ITNs across all age groups, while the scale-up in most sub-Saharan African countries has initially focused on children and pregnant women.

Our results are also consistent and statistically indistinguishable from previous observational studies of ITNs on child mortality. A cohort study in Kenya found a 44% (4%–67%) relative reduction in mortality among children age 1 mo to 59 mo associated with ITN use [8]. A case-control study in Tanzania found a 27% (95% CI 3%–45%) relative reduction in mortality among children aged 1 mo to 4 y associated with ITN use [11]. Our analysis adds to the existing literature by providing evidence of the effect of ITNs on health outcomes under routine conditions over a much broader range of transmission levels and countries; previous studies were predominantly in high endemicity areas. Overall, this finding suggests that on average at least, ITNs have a similar and sizeable effect on health outcomes under routine use compared to that seen in efficacy trials.

This finding supports the continued scale-up of ITNs in sub-Saharan Africa, such as the more recent efforts in Nigeria and Democratic Republic of Congo that had previously low levels of ITN coverage and large populations at risk of malaria [1]. It also emphasizes the importance of ongoing and future efforts to maintain coverage of ITNs in those countries with successful scale-ups by replacing worn out ITNs. Furthermore, it also suggests that the massive effort to scale up ITN coverage over the past decade has paid off and that it is possible for health systems to increase coverage of interventions and affect health outcomes over a relatively short period of time. Continued coordinated efforts between local and national governments, international organizations, funding agencies, and researchers are needed to ensure that ITNs are reaching all populations at risk of malaria. With the relatively large impact of ITNs on child mortality, our findings also support the continued emphasis on malaria control more generally, including the push towards malaria elimination, as a way of improving child health in endemic countries.

We found no evidence of substantial heterogeneity in the effect of ITNs on child mortality across the countries studied here. On the other hand, we found evidence of heterogeneity in the association between ITNs and parasitemia prevalence across countries. One possible explanation is because parasitemia may persist for some time after initial malaria infection; this heterogeneity may reflect different levels of malaria transmission intensity. That is, in high transmission areas, parasitemia may be so prevalent that it is a poor indicator of the incidence of malaria. This heterogeneity in the effect of ITNs on parasitemia prevalence is an important topic for future investigation. We were also not able to detect a significantly different effect on parasitemia of children sleeping under an ITN compared to just household ownership of an ITN; this may simply reflect limited statistical power to detect a true difference. However, we must examine other possible explanations. One possible explanation is that even though MIS data collection is designed to be in high transmission seasons, some of the data collection does occur in low transmission seasons and as the MIS only record information about sleeping under an ITN for the previous night, use of ITNs by children during the low transmission season may not be indicative of use in high transmission seasons. Mothers responding to a question by interviewers about whether their child slept under an ITN the previous night may also be more likely to respond in the positive because of social pressure.

In our study we were not able to detect significant differences in the effect of ITNs by transmission level, number of ITNs owned per household member, or urban and rural residence. These findings likely reflect inadequate power, as indicated by the width of the confidence intervals, to detect statistically significant differences. A previous meta-analysis of RCTs suggested that the efficacy of ITNs is lower in areas with higher malaria transmission [4], while an observational study from rural Kenya [8] found greater effects in areas of high malaria transmission. In our pooled analysis we found significant effects of ITNs in urban areas, which supports previous studies that have shown significant impacts of ITNs on malaria outcomes in urban areas [27],[28]. We were also able to detect significant impacts of ITNs in only a limited number of individual surveys because of small sample sizes, and in general, we did not have the power to detect significant differences between surveys. On the basis of our analysis we cannot discount the possibility that the effect of ITNs varies by these and other factors, such as the extent of education on the proper use of ITNs that are accompanied with distribution programs. Given the large investments in malaria control over the past 10 y, future research and better ways to monitor how the impact of malaria control interventions might vary across populations are required.

Our study provides a method for understanding the real-world impact of not only ITNs but also other interventions on health outcomes using data that are routinely collected. There are, however, a number of limitations of our analysis. First, several MIS do not specify whether the parasitemia tests were based on microscopy or rapid diagnostic test (RDT), and as a result we were not able to standardize the parasitemia measurements. Second, our analysis was limited to publically available datasets; therefore we were not able to access the full range of MIS that have been conducted, although steps are being taken to make these data more widely accessible (e.g., www.malariasurveys.org). Third, in our analysis of parasitemia, we were limited to a cross-sectional analysis and were therefore not able to determine whether ITN exposure occurred prior to malaria infection. Fourth, we were only able to examine the relationship between ITNs and all-cause mortality as the surveys we used do not include information on cause-specific mortality. Increased use of verbal autopsy may allow for refined assessment of the impact of ITNs on malaria-specific mortality, although concerns have been raised about the predictive power of verbal autopsy for malaria [29]. Sixth, the DHS do not collect information on skilled birth attendance and immunizations for children who have died, so in our analysis we could only control for use of these interventions at the PSU level. Seventh, we were not able to control for the effect of other malaria interventions such as indoor residual spraying or drug treatment. Finally, our analysis, like others based on observational studies, may be prone to residual confounding that has not been controlled for by the methods used.

Monitoring and evaluation of interventions to improve population health must include not only measurement of utilization but also whether the delivery of the intervention at scale results in real-world changes in health outcomes. The latter is critical if we are to understand whether interventions are being delivered and used correctly. We used routinely collected survey data to assess the association between intervention use and health outcomes across a large number of countries. Our results suggest that, on average, the scale-up of ITNs in sub-Saharan Africa has led to significant reductions in child mortality—comparable to those found in previous RCTs. While further work is needed to elucidate possible variations in the effect of ITNs, these findings add to the body of evidence that ITNs are effective under usual program conditions and support the continued efforts to scale-up ITN coverage in sub-Saharan Africa.