The value of confirmatory testing in early infant HIV diagnosis programmes in South Africa: A cost-effectiveness analysis

Summary

Lorna Dunning examine the cost-effectiveness of confirmatory testing in early infant HIV diagnosis programmes in South Africa.

Introduction

Despite the success of programmes to prevent mother-to-child transmission (MTCT) of HIV, paediatric HIV remains a substantial burden in sub-Saharan Africa, with 170,000 infants infected with HIV in 2015 [1]. Perinatally infected, untreated infants are at highest risk for rapid disease progression and mortality, with 1 in 2 untreated HIV-infected infants dying before their second birthday [2–5]. The World Health Organization (WHO) recommends testing HIV-exposed infants by 6 weeks of life and immediately referring those who test positive for initiation of HIV care and antiretroviral therapy (ART) [6].

Inexpensive serological antibody assays routinely used for diagnosis in adults cannot be easily interpreted for HIV-exposed infants (those born to HIV-infected women), because transplacental transfer of maternal antibodies leads infants to be seropositive for as long as 18 months [7,8]. Infants therefore require virological tests (nucleic acid amplification tests [NAATs] detecting HIV RNA or DNA) to diagnose HIV infection [9]. Despite reported specificities of >99%, NAATs still have the possibility for false-positive diagnoses [10,11]. As the incidence of paediatric HIV falls with improved access to ART for pregnant/breastfeeding women, the positive predictive value (PPV) of diagnostic assays also decreases, leading to a greater proportion of uninfected infants receiving false-positive diagnoses and starting ART. After ART is initiated, it may be impossible to distinguish truly infected infants from uninfected infants, because effective ART may lead to undetectable HIV RNA in HIV-infected infants whilst also preventing the development of endogenous anti-HIV antibody after maternal antibody fades from the circulation. Because truly uninfected infants may therefore have identical laboratory results to treated infected infants, they may face many years or even a lifetime of incorrect diagnosis and ART [7,12,13].

Current WHO and South African guidelines strongly recommend the use of confirmatory NAAT testing with ART initiation at the first positive result as part of early infant diagnosis (EID) programmes [6,14]. However, a recent policy survey by WHO demonstrated that despite confirmatory testing being used routinely for diagnosis of HIV in adults and across most domains of adult and paediatric medicine, the uptake of confirmatory testing within EID programmes remains limited, with 38% (8/21) of high-burden countries not including confirmatory testing for infants in their guidelines [15–17]. Even where guidelines recommend confirmatory testing, it is rarely implemented [18]. Cost is often cited as a key barrier; many low-resource countries struggle to implement EID programmes due to the high costs of the NAATs required for infant diagnosis. We used a computer simulation model of paediatric HIV infection, diagnosis, and treatment to examine the clinical and economic outcomes of EID programmes without and with confirmatory testing in South Africa.

Methods

Overview

We used the Cost-Effectiveness of Preventing AIDS Complications (CEPAC)–Pediatric model to simulate a cohort of HIV-exposed infants in South Africa undergoing 2 EID algorithms: 6-week EID testing without confirmatory testing and 6-week EID testing with confirmatory testing. The CEPAC–Pediatric model is a first-order Monte Carlo simulation model of paediatric HIV infection, disease progression, diagnosis, and treatment [19–21]. For this analysis, we simulated HIV-exposed infants (born to women living with HIV) from birth through death. Risk of intrauterine or intrapartum HIV infection was modelled as a 1-time risk, based on 3 key maternal characteristics: the probability a mother was aware of her HIV diagnosis during pregnancy; the probability that she received ART during pregnancy, reflecting prevention of MTCT (PMTCT) coverage; and maternal CD4 count for women not receiving ART, reflecting disease stage. Uninfected infants faced a monthly risk of postpartum transmission based on these same characteristics until complete cessation of breastfeeding. All simulated patients faced monthly risks of non-HIV-related mortality. After HIV infection occurred, patients faced additional risks of opportunistic infections (OIs) and HIV-related mortality based on their age, CD4 percent (age < 5 years) or CD4 count (age ≥ 5 years), retention in care, and ART use. Full details of the CEPAC–Pediatric model structure are available in S1 Text and S2 Table and at http://web2.research.partners.org/cepac/model.html.

This work was approved by the Partners Human Research Committee, Boston, MA, US.

Modelled population

In the base-case analysis, we simulated HIV-exposed South African children presenting to care at 6 weeks of age. EID in South Africa is currently directed at infants with known HIV exposure [14]; we therefore included only infants born to women identified as living with HIV during pregnancy. South African guidelines currently recommend ‘Option B+’, or lifelong ART for all pregnant/breastfeeding women identified as living with HIV [14]. We assumed that 90% of women had access to ART during pregnancy and breastfeeding [1]. Based on early data after the release of new infant feeding guidelines in South Africa, we assumed 80% of the cohort was breastfed, for a mean duration of 12 months [6,14,22].

Model outcomes

The model records true infection status for all infants, as well as the results of each administered assay, allowing the direct reporting of true-positive, true-negative, false-positive, and false-negative diagnoses. The primary model outcomes were the number of infants with false-positive diagnoses linked to care as a proportion of total ART initiations, PPV (defined as the proportion of positive test results due to truly HIV-infected infants), life expectancy (LE, in years), and average per-person lifetime HIV-related healthcare costs, from the perspective of the healthcare provider. We projected survival, LE, and costs separately for HIV-infected infants and for the complete birth cohort of HIV-exposed infants, which included both HIV-infected and HIV-uninfected infants. Costs are presented in 2013 US dollars; costs and life expectancies were modelled both undiscounted and discounted at a rate of 3%/year [23]. We first calculated per-person outcomes, then translated these to population outcomes for all 350,000 HIV-exposed infants born in South Africa in 1 year [24,25]. Where clinical outcomes were equal for both strategies, calculation of incremental cost-effectiveness was not necessary; we considered alternative strategies as either cost-saving or more costly in these cases.

Modelled strategies

The strategy ‘without confirmatory testing’ simulated all HIV-exposed infants receiving a NAAT at 6 weeks of age; in this strategy, all infants who received positive results initiated ART upon result return (mean turnaround time: 1 month). The strategy ‘with confirmatory testing’ simulated all HIV-exposed infants receiving a NAAT at 6 weeks of age; all infants who received positive results on the first test initiated ART at result return, but a second blood sample was drawn before ART initiation and sent for a confirmatory NAAT [6,14]. We assumed conditional independence of the primary and confirmatory NAAT results, because WHO recommends that a second specimen be used for confirmatory testing, and most false-positive NAAT results are likely consequences of specimen handling error rather than biological phenomena [6,26,27]. Infants with negative confirmatory tests underwent a third test per WHO guidelines before HIV infection was ruled out and ART stopped. For infants who were truly uninfected but initiated ART and therefore entered HIV care after a false-positive diagnosis (both strategies), we assigned costs for 10 years of routine HIV care, ART, and laboratory monitoring. We varied this duration widely in sensitivity analysis. For truly infected infants diagnosed and linked to care, the model includes lifetime costs for routine care, ART, and laboratory monitoring, as well as care for OIs. We conservatively excluded clinical harms from incorrect ART initiations, such as medication toxicity and stigma.

Input data

We modelled cohort characteristics, PMTCT coverage and MTCT risks, disease progression, and ART outcomes using data from published trials and cohort studies in sub-Saharan Africa (Table 1; Table A in S1 Text; S2 Table) [28–32]. The specificity of NAATs was modelled as 99.6%, from a 2015 WHO meta-analysis [33]. We modelled NAAT sensitivity as a function of time since infection, reaching 100% among infants infected during pregnancy or at least 4 weeks prior to testing (Table 1) [34]. In the base case, laboratory-based NAAT cost was US$25, which included assays, reagents, and human personnel resources associated with specimen processing and result return. To describe the full potential impact of each strategy, in the base-case analysis we modelled guideline-concordant (100%) probabilities of presentation to testing, result return (first NAAT: 1 month), and ART initiation after an HIV diagnosis; we varied these widely in sensitivity and scenario analyses to reflect implementation across a range of settings [1,35–37].

We incorporated published South African healthcare costs for HIV-infected children less than 5 years of age. Costs associated with OI care for children 5 years of age and older were calculated from South African adult resource utilisation (outpatient visits, inpatient days, and laboratory testing) multiplied by South African unit costs [38,39]. Routine HIV care costs (all ages) ranged by age from US$20 to US$165 per month [40,41]. First-line ART costs were from Clinton Health Access Initiative price lists and WHO weight-based dosing recommendations, ranging by age and weight from US$7 to US$40 per month (Table 1) [42,43].

Sensitivity analyses

Following ISPOR–SMDM Good Research Practices Task Force (S1 Table) guidelines on uncertainty in model-based analyses, we conducted extensive univariate and multivariate sensitivity analyses (Tables B and D in S1 Text) [71]. We first conducted univariate sensitivity analyses, in which we varied NAAT specificity and sensitivity, infant HIV prevalence via PMTCT coverage and MTCT risks, the probability of initiating ART after a positive NAAT result, the amount of time infants with a false-positive diagnosis spent on ART, the costs of NAATs, routine HIV care and OI care costs, and ART treatment costs (S1 Table; Task Force recommendation VI-9). Holding all other parameters at their base-case values, we identified the threshold value for each parameter at which the comparison between with and without confirmatory testing would change (S1 Table; Task Force recommendation VI-12). We next performed multivariate sensitivity analyses to evaluate the impact of simultaneous variation in multiple parameters (S1 Table; Task Force recommendation VI-10). We also examined the assumption of conditional independence of the first and confirmatory assays by varying their specificities simultaneously, and assigned a combination of assay cost, sensitivity, specificity, result return time, and result return probability to reflect emerging point-of-care (POC) EID assays (Table 1).

Implementation scenario analyses

We conducted 3 scenario analyses to examine important issues in EID implementation (S1 Table; Task Force recommendation VI-10). First, we simulated lower rates of implementation of 6-week EID testing incorporating input parameters from current testing and treatment cascades in resource-limited countries. We modelled (a) probability of presenting to testing as 73% and probability of ART initiation as 71% of infants with a positive test result and (b) probability of presenting to testing as 95% (2016 UNAIDS estimates for EID coverage in South Africa), probability of result return to caregiver as 80%, and probability of ART initiation as 71% [1,35–37]. Second, we examined programmes offering routine EID testing at both birth and 6 weeks of age to all HIV-exposed infants. Birth tests were modelled as standard laboratory-based NAAT tests with a 1-month result return time (Table 1). Third, WHO strongly recommends prompt ART initiation after a first positive test result (as in our base-case analysis) to avoid delays during a period of high mortality without treatment, but that NAAT testing be repeated on a second specimen drawn prior to ART initiation, to confirm the initial diagnosis [6,7]. Because some providers may be reluctant to initiate ART before receiving a confirmatory result, we also examined the impact of postponing ART until return of the confirmatory test result (1–2 months).

Uncertainty analyses

The univariate sensitivity analyses described above reveal the sensitivity of policy conclusions to variations in key parameters through a wide range of plausible values. To reflect the uncertainty in the primary data estimates, we also varied key parameters through reported 95% confidence intervals (or range or interquartile ranges, if 95% confidence intervals were not available; S1 Table; Task Force recommendations VI-6 and VI-7). For most parameters, this interval fell well within the range examined in sensitivity analyses. We used the results of previously reported model calibration and validation analyses [19,20] to examine the impact of parameter and model structural uncertainty on the comparison between without and with confirmatory testing (S1 Table; Task Force recommendation VI-14).

Results

Base-case results: Clinical outcomes

In the base-case analysis, the birth cohort was projected to include 1.8% infants with intrauterine HIV infection, 1.2% with intrapartum infection, and 1.9% with postpartum infection (cumulative MTCT risk 4.9%), with 95.1% of the cohort HIV-exposed but uninfected (Table 2). In both 6-week EID algorithms (without and with confirmatory testing), HIV-infected infants had a projected 1-year survival of 75.7% and LE of 26.2 years; there was also no clinical difference between the 2 strategies for the entire birth cohort (1-year survival 93.3%, LE 61.4 years). Without confirmatory testing, 128 infants of every 1,000 who initiated ART were truly uninfected, reflecting false-positive diagnosis; this led to a PPV of 87.2%. With confirmatory testing, this proportion fell to only 1 in 1,000 ART initiations, for a PPV of 99.9%.

Base-case results: Costs and cost-savings

Lifetime costs of HIV-exposed infants were US$1,830/infant without confirmatory testing and US$1,790/infant with confirmatory testing (Table 2; Fig 1). The approach of using confirmatory testing was therefore equally effective and was cost-saving; it became cost-saving 3 months after ART initiation (Fig A in S1 Text). Without confirmatory testing, 2.1% of total lifetime HIV-related costs for the birth cohort were accrued by truly uninfected infants after false-positive diagnoses, compared to only 0.01% of lifetime costs with confirmatory testing (Fig 1, orange). If 6-week EID programmes were available for the 350,000 HIV-exposed infants born annually in South Africa, 11,000 infants would require a second NAAT to confirm a first positive result. The cost of these confirmatory NAATs would be approximately US$260,000, but they would avert unnecessary HIV care and ART for 1,400 infants. By averting unnecessary ART and HIV costs for uninfected infants, confirmatory testing would save over US$1,050,000 in the first year and US$13,860,000 in lifetime costs for a South African birth cohort, compared to the approach of not using confirmatory testing [24,25].

Univariate sensitivity analyses: Costs

The use of confirmatory testing remained cost-saving even with wide variations in model parameters, including the cost, specificity, and sensitivity of the NAAT; the probability of presentation at each step in the EID cascade; and the costs of routine care and OI care. Robustness of model results to these key parameters is shown in Fig 2. However, there were 3 key exceptions: confirmatory testing was no longer cost-saving if the duration spent by HIV-uninfected infants in care and on ART after false-positive diagnosis was <3 months (Fig A in S1 Text), if the first and confirmatory assays were no longer considered to be independent (specificity of confirmatory assay <15%; Tables D and E in S1 Text), or if the cost of the NAAT was 16-fold higher than in the base case (>US$400).

We repeated the base-case analysis using a NAAT specificity of 98.8%, as reported by WHO in a previous meta-analysis [33]. The cost-savings associated with confirmatory testing were greater than in the base case (US$1,900/infant without confirmatory testing; US$1,790/infant with confirmatory testing; Fig 1; Table C in S1 Text). With this lower specificity, false-positive diagnoses accounted for 6.0% of total lifetime costs without confirmatory testing, compared to 0.1% with confirmatory testing. After false-positive diagnosis in this scenario, if the HIV status of infants incorrectly identified as infected could be ascertained and their ART interrupted within 2 months of starting ART, confirmatory testing would no longer be cost-saving.

Univariate sensitivity analyses: Clinical outcomes

Several variations in model parameters changed the proportion of ART initiations due to false-positive diagnosis (Table D in S1 Text). Increases in specificity, such as those recently described for novel POC EID assays, reduced the number of false-positive diagnoses in the approach without confirmatory testing to 69/1,000 ART initiations (Table D in S1 Text; Fig 3) [65,66]. A reduction in NAAT sensitivity of 5% only minimally increased the number of infants initiating ART after false-positive diagnosis without confirmatory testing (from 128/1,000 ART initiations in the base case to 133/1,000), but LE for HIV-infected infants was reduced to 25.9 years without confirmatory testing and to 25.7 years with confirmatory testing (Table D footnote in S1 Text). Reductions in the number of infected infants successfully undergoing EID testing, receiving test results, and initiating ART had minimal effects on the proportion of ART initiations due to false-positive diagnoses, but reduced the LE for HIV-infected infants: if only 50% of infected infants completed the EID cascade, LE for HIV-infected infants was reduced to 23.6 years (Table D in S1 Text).

Multivariate sensitivity analyses

In multivariate sensitivity analyses, results were also sensitive to reductions in NAAT specificity and infant HIV prevalence (lower MTCT risks), as depicted in Fig 4. When MTCT risks were <1.3% and NAAT specificity was 98.0%, 749/1,000 infants initiating ART without confirmatory testing were truly uninfected (Fig 4; Table F in S1 Text). With a very high MTCT risk of 9.6% and NAAT specificity of 99.8%, this fell to 34/1,000 ART initiations without confirmatory testing. In all multivariate sensitivity analyses varying specificity (<100%) and infant HIV prevalence, confirmatory testing remained cost-saving (Table F in S1 Text).

Implementation scenario analyses

Incorporating input parameters from current testing and treatment cascades reduced the LE of HIV-infected infants, but the number of infants with a false-positive diagnosis initiating ART remained 128/1,000 ART initiations (Table 3). For programmes offering EID at both birth and 6 weeks of age, the proportion of infants initiating ART after false-positive diagnosis without confirmatory testing increased to 213/1,000 ART initiations, accounting for 4.0% of total costs (Table D and Fig B in S1 Text). When ART was not initiated until the return of the confirmatory NAAT result, mortality during the delay to ART initiation reduced the projected 1-year survival (74.8%) and LE (25.9 years) for HIV-infected infants with confirmatory testing; results for the approach without confirmatory testing (1-year survival 75.7%; LE 26.2 years) were unchanged (Table D in S1 Text). HIV-related healthcare costs remained lower with confirmatory testing than without confirmatory testing in all 3 scenarios.

Uncertainty analyses

The impact of varying key parameters through the confidence intervals around base-case estimates is shown in Fig 2. In all parameter sets derived from formal calibration analyses, confirmatory testing remained cost-saving (Fig C in S1 Text).

Discussion

We simulated EID strategies without and with confirmatory testing for HIV-exposed infants in South Africa. The primary finding was that, when confirmatory testing was not used in the model, more than 10% of infants who initiated ART reflected false-positive diagnoses. These model results are comparable to empirical data from Africa: when records of infants receiving positive EID results and/or initiating ART were reviewed in detail, the proportion found to be truly HIV-uninfected was 2.5% in Kenya, 6.3% in South Africa, 14.6% in Malawi, and 16% in Côte d’Ivoire and Burkina Faso [72–75]. HIV-uninfected infants incorrectly initiating ART not only receive unnecessary medication exposure and treatment costs, but may also experience long-term medication toxicities and the substantial stigma associated with HIV diagnosis [7]. To remain conservative, we did not include these detriments in our analysis. Confirmatory testing would therefore prevent a broader set of adverse outcomes than those described here in a substantial proportion of HIV-uninfected infants.

False-positive results occur for a variety of reasons but are likely the result of stretched human resources compromising test specificity and sensitivity via specimen handling errors such as mislabelling of specimens, incorrect specimen placement in the device, or inadequate quality control resulting in contamination between specimens during PCR amplification [27,76]. WHO recommends the use of a second specimen for a confirmatory test, ensuring the independence of the 2 assays and reducing the likelihood of a second false-positive result. Ensuring that an initial HIV diagnosis is correct is essential because identifying a misdiagnosed, truly HIV-uninfected infant remains extremely difficult after ART is initiated [6]. Lack of detectable anti-HIV antibody, HIV RNA, or HIV DNA may reflect either absence of true infection or the impact of effective ART, which makes withdrawal of treatment the only mechanism available for clinicians wishing to determine the true HIV status of an infant on ART. However, treatment interruptions are not currently recommended, due to concerns about viral rebound and disease progression [77,78]. Unconfirmed EID test results therefore cause diagnostic dilemmas for the provider, whilst families may experience confusion and uncertainty about the health system, discouraging future engagement in care [79].

The second key finding from this analysis is that including confirmatory testing in EID programmes is cost-saving. We found that confirmatory testing reduced the cost per HIV-exposed infant tested from US$1,830 to US$1,790. Although concerns have been raised about the capacity of laboratories to conduct the additional assays needed for confirmatory testing, we found that the number of additional tests was relatively small, as they are required by only 3% of HIV-exposed infants [9]. The costs of these additional tests to confirm HIV infection would be offset by the reduction in false-positive diagnoses and their associated unnecessary HIV care, ART, and ART toxicity costs. Infants with a false-positive diagnosis only had to remain in care and on ART for longer than 3 months for confirmatory testing to become cost-saving; a shorter duration on ART would be extremely unlikely due to the difficulty of identifying uninfected infants after ART initiation. Confirmatory testing remained cost-saving despite wide variations in the costs of clinical care and ART, and at any plausible NAAT cost. In addition, our findings demonstrate that confirmatory testing is cost-saving even with increases in the probability of becoming lost to follow-up between presenting to EID testing, receiving test results, and initiating ART, but lowering the number of HIV-infected infants initiating ART reduces LE for truly infected infants.

A third key finding of this analysis is that when confirmatory testing is done, ART should be initiated after the first positive result, as WHO recommends. If ART was delayed until the result of the confirmatory assay was available, short-term survival and LE were markedly reduced. Waiting even 1 month to initiate ART until the return of the confirmatory result can reduce 1-year survival for HIV-infected infants substantially (from 75.8% to 74.5%) and overall LE for HIV-infected infants (from 26.2 years to 25.9 years). Novel POC EID assays, which offer in-clinic testing and same-day result return, have been proposed as an approach to improving timely ART initiation. Based on preliminary published values for POC NAAT sensitivity (95.5%), specificity (99.8%), and cost (US$30), our analyses suggest that confirmatory testing would likely remain cost-saving in EID programmes using POC assays in place of traditional laboratory assays [65,66].

In South Africa, access to ART during pregnancy has steadily increased, leading to a 76% reduction in new HIV infections among children [1]. If PMTCT programmes continue their success, MTCT rates will fall below 2%, as they have in countries such as Thailand and Cuba [80,81]. As expected, we found that confirmatory testing becomes increasingly critical to reduce false-positive diagnoses when MTCT risks are very low, due to the reduced PPV of diagnostic assays at low disease prevalence (Fig 3). This remained true even when NAAT specificity was extremely high (99.9%). In addition to disease prevalence, assay sensitivity also contributes to PPV. We found that small reductions in NAAT sensitivity, a theoretical result of maternal and infant ART for treatment and MTCT prevention [82], would only minimally increase the number of infants initiating ART after false-positive diagnosis, although lower sensitivity would lead to fewer infected infants being identified and would reduce LE for HIV-infected infants. Finally, opportunities for false-positive results increase when testing is done twice; the inclusion of confirmatory testing within the EID cascade is even more important when programmes consider the addition of birth testing to a 6-week testing programme.

There were 3 main limitations of the analysis. First, uncertainty exists in all long-term projections; although our model was calibrated to ensure results match current survival and OI data, developments in treatment and technology will undoubtedly occur in the coming years [19,20]. Our model structure does not currently permit formal probabilistic sensitivity analyses; however, we performed extensive univariate and multivariate sensitivity analyses to assess the impact of uncertainty around all key input parameters, in keeping with international guidelines [71] (Table G in S1 Text). Confirmatory testing remained cost-saving in a wide range of evaluated scenarios, unless the unlikely thresholds shown in Fig 2 were met or surpassed. Second, while we included all relevant costs for HIV-uninfected infants on ART, we did not include negative clinical impacts for these patients, such as stigma, morbidity or mortality related to ART toxicity, or reduced quality of life after false-positive diagnosis. Including such harms from false-positive diagnosis would further improve the value of confirmatory testing. Finally, this work primarily addresses HIV-exposed infants in South Africa, although our findings may be generalisable to a range of settings: sensitivity analyses demonstrated that confirmatory testing remained cost-saving even with wide variations in MTCT risk, NAAT specificity and sensitivity, NAAT costs, and costs for HIV care and ART.

In summary, we find that use of a second NAAT for confirmatory testing in EID programmes substantially reduces the proportion of infants incorrectly diagnosed as HIV-infected and initiated on ART. While projected cost differences are small, confirmatory testing is cost-saving under a wide range of scenarios in South Africa. Confirmatory testing with ART initiation at the first positive result, as recommended by WHO for EID, should be implemented in settings using NAAT for EID.

Supporting Information

Attachment 1

Attachment 2

Attachment 3

Attachment 4