The impact of continuous quality improvement on coverage of antenatal HIV care tests in rural South Africa: Results of a stepped-wedge cluster-randomised controlled implementation trial

Summary

Background

Evidence for the effectiveness of continuous quality improvement (CQI) in resource-poor settings is very limited. We aimed to establish the effects of CQI on quality of antenatal HIV care in primary care clinics in rural South Africa.

Methods and findings

We conducted a stepped-wedge cluster-randomised controlled trial (RCT) comparing CQI to usual standard of antenatal care (ANC) in 7 nurse-led, public-sector primary care clinics—combined into 6 clusters—over 8 steps and 19 months. Clusters randomly switched from comparator to intervention on pre-specified dates until all had rolled over to the CQI intervention. Investigators and clusters were blinded to randomisation until 2 weeks prior to each step. The intervention was delivered by trained CQI mentors and included standard CQI tools (process maps, fishbone diagrams, run charts, Plan-Do-Study-Act [PDSA] cycles, and action learning sessions). CQI mentors worked with health workers, including nurses and HIV lay counsellors. The mentors used the standard CQI tools flexibly, tailored to local clinic needs. Health workers were the direct recipients of the intervention, whereas the ultimate beneficiaries were pregnant women attending ANC. Our 2 registered primary endpoints were viral load (VL) monitoring (which is critical for elimination of mother-to-child transmission of HIV [eMTCT] and the health of pregnant women living with HIV) and repeat HIV testing (which is necessary to identify and treat women who seroconvert during pregnancy). All pregnant women who attended their first antenatal visit at one of the 7 study clinics and were ≥18 years old at delivery were eligible for endpoint assessment. We performed intention-to-treat (ITT) analyses using modified Poisson generalised linear mixed effects models. We estimated effect sizes with time-step fixed effects and clinic random effects (Model 1). In separate models, we added a nested random clinic–time step interaction term (Model 2) or individual random effects (Model 3). Between 15 July 2015 and 30 January 2017, 2,160 participants with 13,212 ANC visits (intervention n = 6,877, control n = 6,335) were eligible for ITT analysis. No adverse events were reported. Median age at first booking was 25 years (interquartile range [IQR] 21 to 30), and median parity was 1 (IQR 0 to 2). HIV prevalence was 47% (95% CI 42% to 53%). In Model 1, CQI significantly increased VL monitoring (relative risk [RR] 1.38, 95% CI 1.21 to 1.57, p < 0.001) but did not improve repeat HIV testing (RR 1.00, 95% CI 0.88 to 1.13, p = 0.958). These results remained essentially the same in both Model 2 and Model 3. Limitations of our study include that we did not establish impact beyond the duration of the relatively short study period of 19 months, and that transition steps may have been too short to achieve the full potential impact of the CQI intervention.

Conclusions

We found that CQI can be effective at increasing quality of primary care in rural Africa. Policy makers should consider CQI as a routine intervention to boost quality of primary care in rural African communities. Implementation research should accompany future CQI use to elucidate mechanisms of action and to identify factors supporting long-term success.

Trial registration

This trial is registered at ClinicalTrials.gov under registration number NCT02626351.

Keywords:

pregnancy – South Africa – HIV – HIV diagnosis and management – Medical risk factors – Primary care – Antenatal care – Virus testing

Introduction

Continuous quality improvement (CQI) is an important approach to improving quality of care and adherence to clinical guidelines in the health sector [1,2]. Originally developed to streamline production processes in the consumer industry [3], CQI was adopted in the 1990s by the healthcare sector to improve organisational systems to create better quality of care and health outcomes [4]. CQI focuses on developing healthcare providers’ capacity to improve quality of care. It consists of a set of adaptable but systematic techniques to diagnose quality problems using real-time data [1,2] and to address these problems with existing resources [5]. These properties make CQI an attractive approach to improve primary care in resource-poor countries and communities, such as in sub-Saharan Africa [6]. Indeed, CQI is being increasingly rolled out at national scale in several countries in sub-Saharan Africa, with the aim to improve quality of care in primary care [7–11]. The focus on quality improvement in primary care is important, because a primary care visit is often the first and only contact people have with their local health system when seeking care [12].

To date, there have only been 2 randomised controlled trials (RCTs) that have tested whether CQI improves quality of care in the primary care systems in sub-Saharan Africa, one in Malawi [13] and the other one in Nigeria [14]. Both trials found CQI to be ineffective [13,14]. One possible reason for the lack of effectiveness in these 2 trials is that the investigators evaluated CQI in terms of health or distal healthcare utilisation outcomes—neonatal and perinatal mortality rates in the study in Malawi [13] and 6-month postpartum retention in the study in Nigeria [14]. These endpoints are important, and they are rigorously assessed in the 2 studies. However, these endpoints are quite distal from the CQI activities that took place in the primary care clinics. The variability of these endpoints is therefore likely largely a function of factors outside the sphere of influence of CQI, such as emergency care availability, road infrastructure, and mothers’ educational attainment.

For our RCT of CQI effectiveness, we thus decided to use 2 primary endpoints that are closely and directly linked to CQI activities. Our trial took place in an HIV hyperendemic community in rural South Africa, where antenatal care (ANC) in primary care fulfils 2 important HIV care–related functions: to ensure that all pregnant women living with HIV are (1) identified and (2) successfully treated with antiretroviral therapy (ART). We choose measures that capture the procedural quality of care of these 2 key ANC functions as primary endpoints: viral load (VL) monitoring and repeat HIV testing. Both endpoints are important measures of ANC quality and are necessary conditions for preventing mother-to-child transmission of HIV (MTCT) [15,16]. Ultimately, the purpose of healthcare is to improve health outcomes. Implementation science, however, focuses on the improvement of processes whose intermediate outcomes, such as testing coverage, can be directly influenced by changes to health services and are known to be strong determinants of health outcomes [17].

To capture proximate effects of CQI relevant to successful ART, we chose VL monitoring, because VL is the marker of ART response—and MTCT risk—prescribed in the South African national guidelines for ART and for the elimination of MTCT (eMTCT) [18, 19]. Regular VL monitoring enables timely management of virologic failure to achieve eMTCT, including treatment switches and enhanced antiretroviral prophylaxis for HIV-exposed infants [20–22]. Despite the importance of VL monitoring and clear stipulation in the national guidelines [23], most pregnant women on ART are not regularly monitored for VL [24,25].

To capture proximate effects of CQI relevant to the detection of HIV infection, we chose repeat HIV testing, because incident HIV infection in pregnancy increases risk of MTCT as VL peaks shortly after infection [26]. In South Africa, where antenatal HIV incidence is high [27,28], early diagnosis of incident HIV through repeat testing during pregnancy is a critical step towards eMTCT [29]. Yet national guideline recommendations for retesting pregnant women who tested HIV negative during a prior HIV test in ANC are not well adhered to [26].

Our stepped-wedge cluster RCT aimed to establish the effectiveness of CQI on important proximate indicators of the quality of HIV-related ANC in primary care. We hypothesised that a CQI intervention would be effective at improving (i) VL monitoring in pregnant women living with HIV and (ii) repeat HIV testing in pregnant women not living with HIV.

Methods

Details of the Management and Optimisation of Nutrition, Antenatal, Reproductive, Child health & HIV care (MONARCH) implementation project and study have been previously published [30] and are summarised below.

Study setting

The Africa Health Research Institute (AHRI) at Somkhele (previously known as the Africa Centre for Population Health) is located in a rural community in northern KwaZulu-Natal, South Africa. Our CQI intervention was conducted at 7 nurse-led South African National Department of Health (DoH) primary care clinics: 6 were located within the geographic bounds of the AHRI Population Intervention Platform Surveillance Area (PIPSA) South [31], and 1 clinic was located in the market town of Mtubatuba, which is often used by PIPSA residents (Fig 1). Management of the primary care clinics is overseen by Hlabisa Hospital, the local district hospital. HIV prevalence amongst women of reproductive age in this area is approximately 37% [32]. Additional contextual information is described in the Supporting Information: laboratory results workflow (S1 Text), clinic size (S1 Table), and staffing (including lay counsellors; S2 Table). The intervention was delivered by an external CQI team of mentors (including 2 isiZulu-speaking nurses) from the Centre for Rural Health (CRH) at the University of KwaZulu-Natal, who travelled to the study community (hereafter referred to as the CRH team). The mentors were closely supported by an improvement advisor (consultant obstetrician), a scientific advisor, and a data manager.

Trial design

We conducted a stepped-wedge cluster RCT (www.clinicaltrials.gov; NCT02626351) from 15 July 2015 to 30 January 2017. Each clinic formed a cluster except for the 2 smallest clinics, which were merged into one cluster. After a 2-month baseline data collection period, the first cluster rolled over to the intervention on 29 September 2015. Each subsequent cluster rolled over from control to intervention in random order every 2 months (Fig 2).

Trial registration occurred after the baseline and the first step of this 8-step stepped-wedge RCT, on 10 December 2015. The reason for this timing was that it became clear during the baseline that a rigorous scientific evaluation of the CQI intervention would be feasible and desirable for both government and implementing partners (S1 Text). The stepped-wedge design was selected for both pragmatic and ethical reasons [30]. The description of our results follows the 2018 Consolidated Standards of Reporting Trials (CONSORT) extension for stepped-wedge RCTs [33] (S1 CONSORT Checklist).

Intervention

We use the Template for Intervention Description and Replication (TIDieR) to describe the intervention in detail (Table 1) [34]. Briefly, the intervention focused on developing the capacity of local ANC health workers in study clinics and aimed to improve implementation of the national eMTCT guidelines. The intervention was based on the Institute for Healthcare Improvement (IHI) breakthrough collaborative CQI model [35]. Clinical processes and resources were first ascertained during situational analyses conducted in the 2-week lead-up to intervention rollover. CQI tools provided a structured approach to improving process change with clearly defined goals and activities and were implemented flexibly based on need (Table 1). Patient care pathways in the clinic were documented using process maps [36] to identify areas for improvement (e.g., filing of VL results). Barriers and enablers of target endpoints (e.g., VL monitoring) were identified with fishbone diagrams [37], providing the opportunity for comprehensive, clinic-wide improvement. Improvement activities were reviewed using iterative Plan-Do-Study-Act (PDSA) cycles during which “one learns from taking action” in real time (tests of change), unlike awaiting the results of a formal research study [38]. Run charts—outcome time trends plotted during the course of improvement activities—provided visual feedback on whether any changes were likely due to the intervention [39]. As part of the IHI breakthrough collaborative model, the CRH CQI mentors also conducted action learning sessions to consolidate learning, share experiences between healthcare facilities, and motivate collaboration.

The intervention delivery according to the stepped-wedge study design is described in Table 1. The CRH team delivered CQI intensively to each cluster during the 2-month intervention step and then continued with the less intensive intervention during the maintenance phase (Table 1, Fig 2). They delivered a standard “dose” of approximately 19 visits during the intervention phase (2–3 visits per week) and continued with approximately monthly visits during the maintenance phase (Fig 2) for ongoing support and mentorship. The CRH mentors held action learning sessions at the end of each intervention step. In typical CQI delivery, health workers from all clinics in an intervention community would concurrently engage in the intervention and attend all action learning sessions. In our CQI delivery, health workers participated in the intervention and the action learning sessions only during the phase when their clinic was in the intervention arm of our trial. Table 1 further describes materials; procedures; how and where the intervention was delivered (including duration and timing of CQI visits); and how we measured “dose,” “reach,” and fidelity of the intervention [41].

Comparator

During control steps of the study design, health workers continued providing antenatal and postnatal care as usually implemented within routinely available resources.

Participants

Clusters

Clusters were defined as described earlier. ANC health workers in clusters participated in CQI based on availability and ability to commit to CQI ideally for the entire study period. The CQI mentors tried to recruit health workers in leadership roles (e.g., operational managers, professional nurses) to clinic CQI teams to increase the likelihood that CQI activities would continue after the end of the intervention.

Individuals

For the primary endpoints, all women aged ≥18 years were eligible for recruitment at delivery if they were resident in the PIPSA area during pregnancy and/or had ever attended any of the 7 study clinics for ANC in pregnancy.

Randomisation and blinding

We have described our randomisation procedure in detail elsewhere [30]. Briefly, the unit of randomisation was cluster balanced by patient volume. A senior biostatistician external to the study team performed the randomisation of all clusters during the baseline and before the first intervention step. Investigators and healthcare workers in the clusters were blinded to randomisation until the AHRI Chief Information Officer revealed each randomised cluster to the AHRI study team 2 weeks prior to the scheduled intervention rollover date for each cluster.

Procedures

Endpoints

The pre-specified registered primary endpoints were indicators of quality of care in HIV-related ANC: (i) VL monitoring among pregnant women living with HIV and (ii) repeat HIV testing among pregnant women not living with HIV. We report the intervention impact on both primary endpoints in this manuscript. We will analyse and report the secondary endpoints elsewhere.

Data sources

The data on our primary endpoints were sourced entirely from routine patient medical records (maternity case records) [30], which were photographed at delivery. All clusters provided pre-CQI, CQI implementation, and post-CQI data continuously throughout the study. As the maternity case records were first accessed after delivery, ANC data were captured retrospectively—this extended the baseline observation period by an additional 6 months, resulting in a total data collection period of 19 months and a total observation period of 25 months. The period after all clusters had received the CQI intervention was 4.5 months (Fig 2). We used a Research Electronic Data Capture (REDCap) study database for data entry [42]. We collected outcome data continuously over the study at all 7 primary care clinics participating in this study. We also collected outcome data at Hlabisa Hospital maternity ward, because most women living in the study subdistrict deliver at this hospital.

We also conducted a process evaluation to better understand intervention delivery and explain our primary findings. For this, we sourced field notes and reports by the CRH team collated every 2 months. The reports described actual visit dates and type, results of the root-cause analyses, the improvement interventions (including PDSA cycles), successes and challenges, as well as other observations, including impressions of health worker receptivity to CQI (Table 1). We further conducted semi-structured interviews with consenting health workers on their experiences of implementing CQI. We describe in detail the methods, data, and results of the process evaluation in an upcoming scientific publication.

Statistical methods

Power calculation and sample size

As we describe in our protocol paper [30], we assumed for our baseline power calculation—informed by local routine data—that without CQI 40% of all pregnant women living with HIV would receive a test for VL monitoring and 65% of pregnant women not living with HIV would receive a repeat HIV test. We further assumed that half of all pregnant women would be HIV positive and that pregnant women would make 3 ANC visits. We assumed an intracluster correlation coefficient (ICC) of 0.10, which is a conservative assumption compared to other ICCs measured in similar settings [43]. We assumed missing data from 15% of enrolled women. If we enrolled a total of 1,260 pregnant women (i.e., 630 women living with HIV and 630 women not living with HIV), we estimated 80% power to detect at least a 15-percentage-point increase in our 2 primary endpoints at the 5% significance level [30]. In discussion with local stakeholders, we identified this minimum detectable difference over the course of a pregnancy as relevant for health policy and clinical practice.

Statistical analyses

We performed intention-to-treat (ITT) analyses based on the clinic attended at the first antenatal booking visit—individuals declared their “intention” to attend that same facility for the remainder of pregnancy. Although it is well established that the AHRI surveillance population is mobile [31], ITT assumes exposure to a single clinic for the entire duration of pregnancy regardless of actual attendance elsewhere. All participants were assigned CQI exposure status at each ANC visit (by the actual date of that visit) according to the exposure status of their clinic at that time. Participants whose assigned clinic rolled over to CQI during their ANC thus had 1 or more initial ANC visits that were CQI unexposed and 1 or more later visits that were CQI exposed. The beginning of each step (CQI rollover date) was defined as the date of the first actual CQI intervention visit in the randomised cluster.

The binary VL monitoring endpoint was measured in pregnant women living with HIV and defined as a documented VL test performed at a particular visit. Each ANC visit was eligible for an endpoint assessment on or after the first documented HIV-positive status irrespective of whether ART was initiated or continued in pregnancy and irrespective of actual VL results. This definition accounts for real-life imperfections in adherence to guidelines and documentation of ART prescriptions. Women who seroconverted from HIV-negative to HIV-positive status during the study were not included in the analysis of the VL monitoring endpoint.

The binary repeat HIV testing endpoint was measured in pregnant women not living with HIV and defined as a subsequent documented HIV test at a particular visit. Each ANC visit following the first documented negative HIV test was eligible for assessment of this endpoint. ANC visits among women who subsequently tested HIV positive were not eligible for the repeat HIV testing endpoint after the first documented HIV-positive test. We did not restrict our endpoint definitions by visit number or gestation, to allow for real-life imperfections in adherence to guidelines.

Primary analyses

The 2018 extension of the CONSORT statement for stepped-wedge cluster RCTs states that “in addition to reporting a relative measure of the effect of the intervention, it can be helpful to report an absolute measure of the effect” [33]. The reason for these recommendations are that “relative measures of the effects are often more stable across different populations” [33]. Relative measures are therefore more useful than absolute measures for policy makers considering transferring an intervention from one context to another one. In contrast, “absolute measures of effects are more easily understood” [33]. We follow these recommendations and report and interpret both the relative and absolute effect sizes as our primary analyses.

For the primary analysis, we estimated the relative effect sizes using modified Poisson mixed effects generalised linear regression models. Modified Poisson regression has become a standard for the analysis of RCTs with binary outcomes [44,45] because it has advantages over logistic and log binomial regression models [44,46]. One advantage of modified Poisson regression is that it directly generates risk ratios rather than the odds ratios that logistic regression generates. Risk ratios are easier to interpret than odds ratios and, unlike odds ratios, are collapsible [47–49].

Another advantage of modified Poisson regression is that it does not suffer from the convergence problems that commonly arise with another approach to estimating risk ratios, log-binomial regression [50–52]. In addition, modified Poisson regression models are more robust to model misspecification than log-binomial regression models [53]. A disadvantage of modified Poisson regression is that it can produce predicted probabilities greater than unity. However, several simulation studies have shown that modified Poisson regression provides risk ratio estimates equivalent to regression models that cannot produce such predicted probabilities, such as log-binomial regression [50,54–56]. Modified Poisson regression models are thus a good choice for estimating risk ratios in RCTs [45]. To measure absolute effect sizes, we used mixed effects linear probability regression models.

We used 3 models to estimate the effect of the CQI intervention. First, we used the standard Hussey and Hughes model, which includes time-step fixed effects and clinic random effects (Model 1) [57]. Second, we used an extension to this model with a nested random clinic–time step interaction term (Model 2). This extension is recommended by Hemming and colleagues [58], because it allows secular time trends to vary randomly by clinic. Third, we extended the standard Hussey and Hughes model with individual random effects nested within clinic random effects (Model 3). This model accounts not only for clustering of outcomes by clinic but also for clustering of outcomes within individual women across time steps. In all models, we used cluster robust standard errors to further adjust for clustering and model misspecification [44,45].

In addition to the per-visit effect sizes described earlier, we also computed the cumulative absolute probabilities of attaining our endpoints. For this purpose, we used the per-visit absolute effect sizes measured in each model and applied the exponential formula to estimate the cumulative probabilities across the median number of visits during a pregnancy [59].

Sensitivity analyses

For both of our primary analyses—estimating relative and absolute risk—we also ran regressions with additional control variables. We added (i) maternal age and parity; (ii) gestation at each ANC visit and total number of ANC visits attended in pregnancy; and (iii) maternal age, parity, gestation at each ANC visit, and total number of ANC visits.

Analysis of effect heterogeneity

We measured CQI effect heterogeneity by duration of CQI exposure, using the same mixed effects regression models as in the primary analyses but replacing the fixed effect for overall CQI exposure with fixed effects for CQI exposure for each time step since rollover to CQI.

We used Stata version 15.0 (StataCorp LLC, College Station, TX) for all statistical analyses.

Ethics approval for the study was obtained from the University of KwaZulu-Natal Biomedical Research Ethics Committee (reference BE209/14). The ethics approval included a waiver of the requirement for individual consent to access routine clinical data from maternity case records, excluding labour and delivery clinical notes. Engagement meetings were held with subdistrict and district-level DoH partners prior to study commencement to share our study objectives and introduce the intervention. Standard DoH approvals for commencing the study were also obtained as part of a Memorandum of Understanding between AHRI and the DoH. Following the analyses of our results and prior to the publication of this paper, we held engagement workshops with the sub-district and district-level DoH partners, as well as with the primary funders of this study (the Delegation of the European Commission to South Africa). During these workshops, we jointly interpreted our findings and derived policy recommendations.

Although this is a low-risk health systems implementation trial, an independent Data Safety and Monitoring Board (DSMB) annually reviewed study progress. No adverse event data were formally collected, because the intervention targeted health workers in clinical facilities.

Results

All 7 primary care clinics and Hlabisa Hospital maternity ward agreed to participate in the study. Maternity case records from 2,498 women who delivered between 15 July 2015 and 30 January 2017 were accessed. The clinic nurses and local DoH staff were continuous partners in this intervention throughout the study duration. No adverse events, complaints, or other deleterious effects of the CQI intervention were reported. The actual intervention rollover dates of clusters are shown in S1 Table.

The health workers appreciated the CQI intervention. For instance, the CRH team observed that the clinic health workers who participated in the CQI intervention were enthusiastic about their increased capacity to improve quality of services. Several clinic health workers expressed an interest in further training and mentoring on quality improvement tools and approaches. Finally, clinic health workers urged the CRH team to continue to work with them on quality improvement beyond the end of this study. We will report further details on these findings in an upcoming publication on the process evaluation that accompanied this trial. Preliminary results of situational analyses (conducted prior to intervention rollover) identified staffing shortages as well as lack of formal tracking systems for patients—and for test results—as potential impediments to the intervention activities described in Table 1.

Most maternity case records contained ANC clinical information. Among the 2,498 women recruited to this study (Fig 3), 5 had blank maternity case records because their previous records had been lost, and a further 19 had blank records because they never attended ANC. Of all women recorded in our study database, 2,160 participants were eligible for our primary ITT analysis (contributing 13,212 observations) during pregnancy: 338 women were excluded from our primary analysis because their HIV status at delivery was unknown, their first ANC clinic was not a study clinic, or their first ANC clinic was unknown (Fig 3).

We estimated the total number of women living with HIV who attended a study clinic at their first ANC visit to be 1,026 (1,011 with documented HIV status plus 15 women with unknown HIV status by delivery), of whom 99% were diagnosed HIV positive—and could therefore start ART.

Median age at first booking was 25 (interquartile range [IQR] 21 to 30), and median gestation at first booking was 19 weeks (IQR 15 to 24). Median parity was 1 (IQR 0 to 2). Median number of antenatal visits was 6 (IQR 4 to 8), and 86.9% of women (95% CI 82.5% to 90.3%) attended at least 4 ANC visits during pregnancy. Around half the women (50.3%, 95% CI 45.1% to 55.4%) attended their first ANC visit before 20 weeks’ gestation. HIV prevalence at first ANC visit was 47.4% (95% CI 42.3% to 52.7%) and varied by age group (S1 Fig). There were no major differences in individual participant baseline variables by arm (Table 2), i.e., the 2 arms were balanced on observed baseline characteristics. Among all eligible participants 822 of 2,160 (38.1%) were never exposed to CQI during their pregnancy (i.e., during all of their ANC clinic visits the clinic[s] had not yet received the CQI intervention), 870 of 2,160 (40.3%) were fully exposed to CQI (i.e., during all of their ANC clinic visits the clinic[s] had already received the CQI intervention), and the remaining participants were partially exposed (i.e., during initial ANC clinic visits the clinic[s] had not yet received the CQI intervention, while during later ANC clinic visits the clinic[s] had received the intervention).

Pregnant women living with HIV

We analysed 1,011 women whose first documented HIV status was HIV positive. At any time in pregnancy, 93.8% of women living with HIV had a documented ART prescription, and 56.3% had received a VL test. Among those women who had received a VL test, only 52.4% had a documented result of which 85.2% were <200 copies/mL (Table 3). ART prescriptions, VL monitoring, and suppression in late pregnancy are presented in Table 3.

Pregnant women not living with HIV

Among the 1,149 women with an initial negative HIV test, 768 of 1,149 (66.8%) had at least one repeat HIV screen during pregnancy. Repeat HIV testing within 3 months prior to delivery is presented in Table 3. Twelve women (1.0%) seroconverted during pregnancy.

Effect of CQI on HIV care tests

Primary analyses—Pregnant women living with HIV

Women exposed to CQI were significantly more likely to receive a VL test: the relative effect sizes (relative risks [RRs]) in our 3 primary analyses ranged from 1.38 to 1.39 (Fig 4, Table 4) and were all highly significant (all p < 0.001). The absolute effect sizes of receiving a VL test per ANC visit in our 3 primary analyses ranged from 3.9 to 4.1 percentage points higher (Table 5) in the intervention compared to the control arm. With a median of 6 ANC visits per pregnancy in this study, the absolute effect size per visit translates to a cumulative probability of receiving a VL test in pregnancy of 58% in the intervention arm compared to 46% in the control arm. With the recent WHO recommendations for at least 8 ANC visits [60], these values are equivalent to a cumulative probability of receiving a VL test of 69% in the intervention arm compared to 57% in the control arm.

Primary analyses—Pregnant women not living with HIV

CQI did not have a statistically significant effect on repeat HIV testing in any of our 3 primary analyses. The RRs ranged from 1.00 to 1.01, and the corresponding p-values ranged from 0.877 to 0.958 (Table 4, Fig 4). These numbers are equivalent to an absolute change in the probability of receiving a repeat HIV test per ANC visit of 0.0 to 0.3 percentage points (all p ≥ 0.758, Table 5). With a median of 6 ANC visits in this study, this effect size per visit translates to a 59% to 60% cumulative probability of receiving a repeat HIV test during pregnancy in both the intervention and the control arm. With the WHO-recommended 8 ANC visits, these numbers translate to 69% to 70% cumulative probability of receiving a repeat HIV test in both arms.

Sensitivity analyses

When we adjusted for additional covariates, all effect size estimates remained essentially the same as in the primary analyses. RRs for HIV VL monitoring ranged from 1.36 to 1.37 (all p < 0.001) with adjustments for maternal age and parity, 1.39 to 1.40 (p-values <0.001 to 0.001) with adjustments for gestation at each visit and total number of ANC visits, and 1.35 to 1.36 (p-values 0.005 to 0.009) with adjustments for all 4 covariates. The absolute effect sizes for VL monitoring per ANC visit were 3.7 to 4.0 percentage points higher (all p < 0.001) with adjustments for maternal age and parity, 4.4 to 4.6 percentage points (all p < 0.001) with adjustments for gestation at each visit and total number of ANC visits, and 4.2 to 4.4 percentage points (all p ≤ 0.008) with adjustments for all 4 covariates.

RRs for repeat HIV testing ranged from 1.00 to 1.01 (all p ≥ 0.871) with adjustments for maternal age and parity, 1.08 (all p ≥ 0.525) with adjustments for gestation at each visit and total number of ANC visits, and 1.08 (all p ≥ 0.504) with adjustments for all 4 covariates. The absolute effect sizes for repeat HIV testing per ANC visit ranged from 0.0 to 0.3 percentage points (all p ≥ 0.757) with adjustments for maternal age and parity, 1.1 to 1.3 percentage points (all p ≥ 0.457) with adjustments for gestation at each visit and total number of ANC visits, and 1.1 to 1.3 percentage points (all p ≥ 0.445) with adjustments for all 4 covariates.

CQI effect heterogeneity

There was variability in CQI effect on VL monitoring with time since rollover to the intervention, in all 3 mixed effects models. Immediately after rolling over to the CQI intervention, the relative effect size was 1.17 (p = 0.071) in Model 1. After 2 months of CQI exposure (lag of 1 step) and 4 months (lag of 2 steps), the relative effect sizes were 1.42 (p = 0.021) and 1.45 (p = 0.015), respectively. There was no significant CQI effect beyond 6 months of exposure (Table 6). Similarly, the absolute effect sizes varied with the same durations of exposure to CQI in Model 1: 1.7 percentage points (p = 0.070) with immediate exposure, 4.4 percentage points (p = 0.038) after 2 months of exposure, 4.9 percentage points (p = 0.035) after 4 months of exposure, and no significant change beyond 6 months of exposure (Table 6). The results were essentially the same with Models 2 and 3 for relative and absolute effect sizes. There was no heterogeneity in CQI effect on repeat HIV testing by duration of CQI exposure in either the relative or absolute measures (Table 6).

Discussion

Our stepped-wedge cluster RCT conducted under real-life conditions in rural South Africa showed that CQI improved antenatal HIV VL monitoring but did not improve repeat HIV testing. To our knowledge, this is the first RCT to show that CQI can improve quality of primary care in sub-Saharan Africa.

The significant improvement in the first of our 2 primary endpoints—VL monitoring—suggests that CQI may indeed be a good approach to improve quality of care in primary care. This finding is important for policy: CQI is an attractive approach for resource-poor settings because it works with local health workers and within existing resources. Moreover, CQI is already commonly used to improve quality of care in many resource-poor countries and communities, including sub-Saharan Africa [7–11]. Our trial provides evidence for these policies.

Both relative and absolute effect sizes are important for policy. We found a large relative effect of the CQI intervention on VL monitoring: CQI caused an almost 40% increase in VL monitoring. Relative effect sizes are commonly more stable across populations with different baseline attainments of an endpoint [61]. VL monitoring varies widely across ANC and HIV treatment programmes in sub-Saharan Africa [62,63]. Policy makers should therefore consider our relative effect size in the context of their health system’s achievement in measuring this critical indicator of success in interrupting transmission of HIV from mother to child. In absolute terms, our effect size was relatively small on a per-visit basis, about 4 percentage points per visit. However, it is well recognised that multiple ANC visits are required for maternal and neonatal health [60]. For the median number of 6 ANC visits in this community, we estimate a cumulative absolute effect size of about 12 percentage points. This effect size is close to the threshold of 15 percentage points for clinically meaningful effects that we used in our power calculations.

We also found heterogeneous effects of CQI on VL monitoring by duration of the cluster’s exposure to CQI: effects reached their maximum 2 to 4 months after intervention rollover and waned thereafter. The time pattern we see here can be explained by a “natural course” of innovation—initially health workers need to learn to implement innovations to maximum effectiveness; over time, the need for the innovations (e.g., novel processes supporting VL monitoring) declines as more and more patients have benefited. At the same time, the waning effects following a maximum could also indicate health worker fatigue in implementing an innovation that has now become routine. This latter explanation would argue for implementation research on approaches to maintain long-term innovation effects.

However, our findings also emphasise that it is important to better understand the contextual determinants and potential mechanisms of CQI effects, because CQI did not have an effect on our second primary endpoint, repeat HIV testing. There are several plausible reasons for the difference in CQI effects. The first reason is perceptions of clinical impact. For VL monitoring, impact is likely very salient—lower maternal mortality, better maternal health, and reduced MTCT. In contrast, for repeat HIV testing, impact may be perceived as more “distal” to clinical outcomes and thus less important. CQI activities aimed at increasing repeat HIV testing may therefore not have resonated as strongly with health workers as did CQI activities aimed at improving VL monitoring. A related issue is patient perceptions or behaviours. For example, women not living with HIV may have less perceived risk of infection [64] or concerns about negative partner attitudes if they received a positive HIV diagnosis [65]. Presenting late to ANC for the first time may also have been a barrier to receiving a second HIV test for some women, because the guidelines stipulate an interval of 3 months between HIV tests.

The second potential reason is the differential baseline attainment of our 2 primary endpoints—about one-third for VL monitoring and about two-thirds for repeat HIV testing. While our power calculations indicated that our trial was sufficiently powered to detect significant CQI effects on both primary endpoints, it might have been practically easier to identify and address problems in achieving high levels of coverage with VL monitoring compared to repeat HIV testing, because VL monitoring involves more processes and complexity than repeat HIV testing—thus, CQI that addresses process quality had fewer opportunities to change practice in repeat HIV testing. For instance, VL monitoring involved identifying women eligible for the test, phlebotomy, following up and documenting results, and retrospectively capturing routine ART clinic data onto the national monitoring and evaluation database (TIER.Net)—CQI could address flaws in most of these processes. During each visit, women living with HIV may have interacted with different staff cadres in the clinic—nurses for clinical management or lay counsellors for adherence counselling—with additional opportunities to identify eligible women for testing during the clinic appointment. Other staff (e.g., data capturers) may have also identified women eligible for VL monitoring while filing previous VL results or capturing routine data onto TIER.Net after the visit. Conversely, repeat HIV testing involved a simpler workflow: identifying eligible women, conducting the rapid test, and immediately documenting the result, with only one type of staff cadre involved (usually a lay counsellor).

The final potential reason for the difference in CQI effects is that, during the study period, many of the HIV lay counsellors, who were carrying out HIV testing in primary care clinics, lost their employment in the public-sector health system in South Africa because of a lay counsellor re-deployment policy [66]. The human resources for HIV testing in primary care were thus diminished over the study period. In contrast, nurses carried out VL monitoring in the South African health system, and their numbers remained stable over the study period. The differential CQI effectiveness on our 2 primary endpoints may thus have been caused by underlying differences in the availability of the human resources needed to effectively implement endpoint-specific CQI activities.

The 2 other trials of CQI in primary care in sub-Saharan Africa did not find significant effects, likely due to selection of endpoints that were relatively distal to CQI activities. As alluded to previously, the variability of these endpoints—neonatal and perinatal mortality in Malawi [13] as well as a distal healthcare utilisation outcome, postpartum retention, in Nigeria [14]—may be a function of many factors that CQI cannot be expected to influence. Our endpoints were more proximate to CQI thereby making it easier to demonstrate a direct CQI effect.

In general, however, the 2 prior studies and ours jointly point towards the need to ensure sufficient resources and sufficiently good underlying health systems processes for CQI to be effective. For instance, staff turnover or redeployment during CQI initiatives are likely to impede CQI effectiveness, whereas clinic health worker willingness to engage with CQI, sufficient protected time for clinic health workers to work with CQI mentors, and relevance of CQI initiatives to local needs are likely enablers of CQI effectiveness [13, 14, 67].

CQI implementation and resource investments

We observed that health workers maintained positive attitudes towards CQI and our CQI mentors during the study. In particular, health workers were enthusiastic to learn CQI tools and improve quality of services and were keen to continue working with the CRH team.

CQI is a systematic process for identifying problems and devising, testing, and revising potential solutions to quality shortcomings. The different management techniques used during this process are highly standardised and time-tested, including process mapping, fishbone diagrams, and PDSA cycles. Deviations from the overall process and these individual techniques—by the CQI mentors implementing them or by clinic health workers utilising them—would imply low fidelity. Based on our observations, CQI mentors implemented the CQI intervention with high fidelity. However, clinic health workers’ implementation of the CQI process and techniques was of somewhat lower fidelity. Clinic health workers could not always participate in CQI meetings and failed to complete some of the CQI tasks, despite their general enthusiasm for the intervention. The reasons for these deviations were mostly rooted in structural capacity constraints in this rural South African health system, including staffing shortages, rapid staff turnover, and competing clinical commitments [68].

The CQI process and management techniques are intended to be the same across clinics and communities; the solutions that they identify, test, and revise are intended to be tailored to each clinic and will therefore differ across contexts. The solutions that each clinic identified and implemented did indeed differ, even though all participating clinics were located in the same district in rural KwaZulu-Natal. We will report further details on these results in an upcoming publication on the process evaluation that accompanied this trial.

The key resource needed to implement CQI is CQI mentors’ and clinic health workers’ time. CQI mentors typically spent several hours per week training and supporting clinic health workers during the intensive phase of CQI; during the maintenance phase, the mentors spent substantially less time interacting with the clinic health workers, typically several hours per month. The training sessions at the very beginning of each step and the action learning sessions each required an entire day. CQI activities required time to (i) identify drivers of poor VL monitoring and repeat HIV testing, (ii) conduct group meetings to design clinic-specific solutions, (iii) implement those solutions (e.g., different methods to track women eligible for HIV care tests), and (iv) use clinic-based real-time data sources to test whether particular solutions improved testing rates.

CQI mentors who are adaptable and relate professionally and culturally to health workers at target health facilities may be more successful at engaging health workers in CQI. Mentors must be adept at generating descriptive data and, ideally, be familiar with implementing the desired change in practice. Obtaining basic CQI certification may take between around 18 hours (online) [69] to 3 days [70] of coursework.

In addition to the CQI mentors’ time and the health workers’ time, a so-called improvement advisor—a more highly qualified health professional than the CQI mentors with the ability to coach and lead the CQI mentors—is an integral part of the team to oversee CQI activities. In our experience, the improvement advisor would need to invest around 20% of their time for successful CQI support; in other settings, this would depend on the needs of the CQI mentors and participating clinics. Given these CQI resource needs, the time—and financial—costs of the intervention were overall low. Feasibility and scalability of CQI will, however, not only depend on resource requirements but also on clinic health workers’ motivation and mindset. CQI requires changes in individual behaviours and institutional practices, which in turn are determined by job satisfaction, ambition, experience, and local workplace culture and constraints [40].

In our study, the CQI mentors typically held CQI meetings 2 to 3 times per week during the intensive 2-month step. For routine implementation at larger scale, alternative meeting frequencies may also work. For instance, it may be more practicable to conduct visits less frequently (e.g., once per week) but over a longer period of time (e.g., half a year). The optimal frequency of mentor visits will likely depend on how many clinic health workers are available for CQI activities and whether the clinic activities that the CQI team chose for improvement require additional support.

Strengths and limitations

Our pragmatic cluster-randomised stepped-wedge design rigorously tested the effectiveness of CQI in a real-world setting in rural South Africa while enabling all clinics to eventually receive the intervention. Our registered primary endpoints were not only important process indicators of HIV care quality expected to respond to CQI, but they were also essential clinical practice standards described in the national South African guidelines for HIV treatment and ANC. The intervention was implemented flexibly by adapting to local clinic needs, and individual participant inclusion criteria were broad. Our CQI implementation team of local CQI mentors, improvement advisor, data manager, and scientific partners were from the University of KwaZulu-Natal, had extensive experience implementing CQI elsewhere in South Africa, and spoke the vernacular language, isiZulu. The research was co-led by South African researchers, while most researchers in the wider scientific team have worked in rural South Africa for many years.

Our study had several limitations. First, 2-month intervention steps may have been too short to catalyse the changes required to improve outcomes given that clinic resources in routine care are already overstretched. However, the monthly maintenance visits following the intervention steps may have compensated for the relatively short initial implementation period. Second, we measured our endpoints as documented in the maternity case records. Although most maternity case records contained clinical information, the poor documentation of VL results among women who had a VL test performed raises concerns about data quality overall. It is thus possible that actual repeat HIV testing and VL monitoring rates were higher than measured in our study albeit not documented. In this case, we would also expect that the effect sizes that we have measured are underestimates of the true effect sizes. Third, we were unable to measure long-term sustainability of CQI over several years, because our study duration was limited by our research budget.

Other policy and future research implications

In addition to the general policy relevance of our findings for health policies aiming to improve the quality of primary care in sub-Saharan Africa, our findings are of specific relevance to the goal of eMTCT. Universal coverage of the activities captured by our 2 primary endpoints—VL monitoring and repeat HIV testing in pregnancy—will be critical to achieve this goal [24]. In particular, VL monitoring close to delivery is key to ensuring maternal virologic suppression in the peripartum period [20], although VL monitoring and repeat HIV testing are crucial throughout pregnancy and breastfeeding to enable timely interventions for mother and baby. According to our findings, CQI could be a good addition to the set of interventions driving eMTCT in HIV hyperendemic settings.

Improving clinical processes to achieve desired health systems and health outcomes requires changes in practice (behavioural changes), which depend on individual motivation, job satisfaction, and clinical knowledge and context [40]. It is also unclear as to how far clinic health workers “normalise” CQI into their practice once the CQI mentors have completed an engagement, particularly given the human resource shortages and high staff turnover in the local health system. Knowledge of time to assimilation of CQI could inform future study designs that account for a suitable time lag before outcomes can be expected to change.

An important question for future research is whether this CQI intervention “spilled over” to ANC activities outside HIV care and to primary care outside ANC. Such spill-over effects could be positive or negative. On the one hand, the clinic health workers trained in CQI may have applied their new skills to other clinical services, resulting in further quality improvements. On the other hand, health workers invested time and energy in this CQI and, as a result, may have neglected clinical services outside the scope of this intervention. In future research, we plan to measure the effects of CQI on a broader set of endpoints, which will capture the quality of a wide range of ANC functions.

Outcome measures that lie on plausible causal pathways from an exposure to an outcome can be used to empirically confirm mechanistic hypotheses. CQI is an intervention that can work its way to outcomes through multiple pathways, because it empowers local health workers to identify and implement those health systems improvements and innovations that are most promising in the particular contexts in which they work. In this case—and other complex interventions—local health workers can identify likely mechanisms connecting an intervention to outcomes. In the case of HIV VL monitoring and repeat HIV testing, proximate endpoints that lie on the causal pathways from CQI could include availability of test kits, presence of systems for patient tracking and tracing, health worker motivation to test, and patient demand for testing.

While our quality of care measures are good choices to test the effectiveness of a CQI intervention, the ultimate purpose of healthcare is to reduce morbidity and mortality. Future mathematical modelling studies should estimate the impact of CQI on health outcomes that could be achieved given the effect sizes we measured in this study and local estimates of VL monitoring in different communities in sub-Saharan Africa.

Our study findings are likely transferable to other resource-poor communities in southern and sub-Saharan Africa, where nurses are the primary providers of ANC and HIV treatment. Given the resource shortages and increasing demand on HIV services with rollout of universal ART for all people living with HIV, patient and laboratory results tracking are likely to become more challenging with paper-based methods. Conversely, access to electronic laboratory results would increase healthcare provider availability for direct clinical care while maximising opportunities for detecting and managing maternal HIV viraemia. It is thus plausible that technological innovations in clinic data and management systems could contribute to CQI effectiveness in improving HIV care.

Conclusions

We showed that CQI in primary care can be effective in rural South Africa. We found a significant CQI effect on one of our 2 primary endpoints, VL monitoring among pregnant women living with HIV. However, the fact that CQI failed to improve our second primary endpoint indicates that CQI success is sensitive to the particular health system processes it is intended to address. CQI should be considered as an intervention to improve quality of primary care in rural African communities. Its use should be closely monitored to ensure that we further improve our understanding of factors influencing its success.

Preliminary results of this trial were presented at the Conference on Retroviruses and Opportunistic Infections (CROI) in March 2018, Boston, MA, USA.

Supporting information

S1 CONSORT Checklist [docx]
CONSORT checklist for stepped-wedge trials.

S1 Text [docx]
Additional methods.

S1 Table [docx]
Characteristics of participating primary care clinics.

S2 Table [docx]
Summary of clinic staffing and recruitment to clinic CQI team.

S1 Fig [tif]
HIV prevalence by maternal age group at delivery.