A novel electronic algorithm using host biomarker point-of-care tests for the management of febrile illnesses in Tanzanian children (e-POCT): A randomized, controlled non-inferiority trial

Summary

In a randomized controlled non-inferiority trial, Kristina Keitel and colleagues compare the use of an electronic algorithm using host biomarker point-of-care-tests to routine care for the management of febrile illnesses in Tanzanian children.

Introduction

Febrile illnesses comprise the vast majority of pediatric outpatient consultations in resource-poor settings [1]. Only a small percentage of these children require antibiotic treatment or referral for hospital-based supportive care, such as oxygen therapy [2]. However, correct identification of this minority of children is pivotal; substandard management of children with infections has led to 2 major public health challenges: first, persistent high mortality from common childhood infections [3] and, second, the tremendous overprescription of antibiotics at the peripheral healthcare level [4], which contributes to spreading antimicrobial resistance [5].

Though childhood infections are common, their management requires integration of a multitude of information such as epidemiological, demographic, clinical, and laboratory data. It also necessitates the consideration of multiple diagnoses at once, as children often present with several concurrent complaints and symptoms [6]. Such an integrated approach for the classification and treatment of childhood infections is reflected in the current World Health Organization (WHO) strategy, Integrated Management of Childhood Illness (IMCI) [7]. Though a positive impact on child mortality could be shown [8], its implementation has faced major challenges through a spectrum of obstacles within the health system, from the macro (policy) to the micro (patient–provider interaction) level. At the micro level, the cornerstone of the IMCI strategy remains a set of paper-based algorithms that recommends presumptive treatment based on clinical signs and symptoms (except for the malaria rapid diagnostic test [mRDT] that was introduced in the 2014 version) [7]. Adherence to the IMCI guidelines is low across geographical settings [9–11]. Electronic IMCI versions (e-IMCI) provide a more user-friendly format and may thereby increase algorithm adherence and the consistency of clinical assessments [2,12]. Beyond e-IMCI, electronic algorithms also have the potential to integrate more complex information while maintaining a simple user interface.

In addition to facing format-related barriers, IMCI implementation also faces content-related challenges. The algorithm lacks guidance for a significant proportion of febrile children, such as children without localizing symptoms (fever without source [FWS]). For such children, IMCI instructs clinicians to “give appropriate antibiotic treatment for an identified bacterial cause of fever” but provides no guidance on how to identify a bacterial cause of fever [7]. Before introduction of mRDT-based treatment, IMCI presumptively classified all children with fever as having malaria. Now that mRDTs are used, and lacking adequate diagnostics for bacterial infections, clinicians have practically exchanged antimalarial against nonselective antibiotic treatment for patients with negative mRDTs [13]. This has led to a tremendous overuse of antibiotics at the peripheral level [4]. To address this challenge of diagnosing bacterial infections in children with FWS, a revised IMCI-based algorithm was developed (ALMANACH; S2 Fig). It includes urine dipstick testing and a clinical predictor or rapid test for typhoid [14]. However, these 2 diseases represent only a fraction of bacterial infections that need to be considered [15]. A second content-related challenge is that the IMCI algorithm relies on clinical symptoms alone, which inherently lack diagnostic accuracy in identifying children in need of antibiotic treatment or referral for hospital-based supportive care [16]. Host biomarkers that can help identify children with bacterial infections, such as C-reactive protein (CRP) and procalcitonin (PCT), have not been considered within the IMCI strategy [17]. The safety of using CRP or PCT cutoffs to decide on antibiotic prescription in children has never been evaluated. Besides 1 trial in Vietnam that evaluated using CRP without provision of clinical guidance to guide antibiotic prescription in mild respiratory infections [18], all studies of CRP and PCT in children have focused on analytical performance; none have assessed whether using these tests would change patient outcome. Furthermore, some point-of-care tests (POCTs) may help detect disease in children with severe symptoms where clinical signs lack diagnostic accuracy: for example hemoglobin (Hb) testing can identify children with severe anemia in need of blood transfusions. Lastly, the diagnostic value of the clinical signs that were included in IMCI (based on expert opinion and small derivation studies) has changed. The epidemiology context of infections has indeed shifted away from bacterial and parasitic infections towards viral infections [15]. Additionally, since development of IMCI 30 years ago, a considerable amount of novel evidence on such clinical signs has emerged, which has not been integrated into IMCI thus far [16].

Based on the challenges and opportunities identified, we constructed a novel electronic patient management algorithm, e-POCT, on the IMCI backbone. e-POCT is derived from the latest evidence of pediatric fever management based on studies from both low- and high-income settings.

e-POCT is built into an Android application, which guides the clinician through the entire consultation and recommends management based on a few clinical elements as well as POCTs. The POCTs used are aimed at triaging children with severe disease who require referral to a higher level of care (oxygen saturation [SaO2], heart rate, blood glucose, and Hb), detecting malaria infection (mRDT), and distinguishing between bacterial and viral diseases (CRP and PCT). Given the innovative approach of the algorithm, we first sought to assess its safety when applied to children presenting with febrile illnesses in a low-resource setting. Hence, the objectives of this study were to determine whether e-POCT was non-inferior in terms of clinical outcome to a validated electronic algorithm derived from IMCI (ALMANACH) when managing febrile illness in children under 5 years and to compare the proportion of antibiotic prescriptions and severe adverse events (deaths and secondary hospitalizations) between the 2 arms.

Methods

Ethics

The study protocol and related documents were approved by the institutional review boards of the Ifakara Health Institute and the National Institute for Medical Research in Tanzania, by the Ethikkommission Beider Basel in Switzerland, and the Boston Children’s Hospital ethical review board. An independent data and safety monitoring board oversaw the study. The trial was registered in ClinicalTrials.gov, identifier NCT02225769.

Study design

This was a randomized (at patient level), open controlled trial to investigate whether a novel electronic algorithm using point-of-care (POC) testing, e-POCT, was not inferior in terms of clinical outcome to a validated electronic algorithm derived from IMCI (ALMANACH) when treating febrile infections in children under 5 years of age. In parallel to the randomized study, a small cohort of children managed per routine care was observed, and their clinical outcomes were compared to those of the children treated using e-POCT. The protocol for this trial, the statistical analysis plan, and supporting CONSORT checklist are available as supporting information (S1 and S2 Texts; S1 Table).

Participants

This study was conducted in the city of Dar es Salaam, Tanzania. Malaria endemicity in this region is relatively low, with about 10% of fever patients positive for malaria; transmission is perennial with a peak in the post-rainy season [15]. Consecutive patients presenting for acute care during normal business hours at the outpatient departments of 3 district hospitals and at 6 health centers in Dar es Salaam were screened for eligibility. Recruitment sites were chosen to represent the pediatric outpatient population in Dar es Salaam. Inclusion criteria were age 2 to 59 months, history of fever for 7 days or less, and axillary temperature ≥ 37.5°C at presentation. Exclusion criteria were weight less than 2.5 kg, main complaint being an injury or acute poisoning, or previous medical care for the present illness. Children satisfying the inclusion and exclusion criteria were enrolled if the parent or guardian had received full information on the study and signed written informed consent.

Randomization

For the main comparison between e-POCT and ALMANACH, patients were enrolled by the study clinicians and then randomized to 1 of the 2 management arms. They were individually randomized in blocks of 4 according to a computer-generated randomization list provided by an independent, off-site researcher. Sealed, opaque forms were used for allocation concealment and were opened only after the patient’s enrollment. Patients in the routine care cohort were enrolled by a research assistant and directed to a routine clinician at the corresponding health center.

Interventions and study procedures

The intervention consisted in having study clinicians use the e-POCT algorithm (e-POCT arm) or ALMANACH algorithm (control arm) during the consultation to manage the patient. The development, rationale, and content of the e-POCT algorithm are detailed in S1 Text. In brief, we performed a structured literature review focusing on (i) the identification of children with severe infections requiring referral, (ii) the identification of children with serious bacterial infections, including using CRP and PCT to predict the need for antibiotic treatment, and (iii) the identification of children with dehydration (S2 Table; S1 Fig). The evidence retrieved is described in detail in S3 Text. It was used to design the novel e-POCT electronic algorithm (Figs 1 and 2).

e-POCT differs from the 2014 version of IMCI in the following ways: (i) use of pulse oximetry to identify children with hypoxemia and severe tachycardia, (ii) use of Hb testing to detect children with severe anemia, (iii) construction of a “severe respiratory distress” classification, (iv) refinement of criteria for severe malnutrition, (v) use of a 2-step approach including temperature- and age-corrected respiratory rate and CRP for diagnosing bacterial pneumonia, and (vi) use of CRP and PCT to decide on antibiotic prescription for children with fever without localizing symptoms. The main differences between the e-POCT and ALMANACH algorithms, as well as IMCI, are summarized in Table 1. We constructed a novel electronic algorithm, e-POCT, that was programmed into an Android-based mobile tool. The electronic version allowed integrating a greater amount of data, more elaborate calculations, and direct connection to the oximeter, without increasing the complexity of the consultation process for the clinician.

Children enrolled in the intervention arm were assigned to study clinicians using e-POCT (Figs 1, 2 and S1) during 2 weeks, while children enrolled in the control arm were managed by other study clinicians using ALMANACH (S2 Fig). In order to minimize a cluster effect at clinician level, clinicians then switched arms, and thus algorithms, every 2 weeks. Based on their assignment, enrolled children were directed either to the e-POCT or ALMANACH clinician.

Compared to the IMCI-based algorithm (ALMANACH) used by study clinicians in the control arm, e-POCT uses fewer clinical symptoms and signs. Rather, it relies on signs that can be measured objectively: hypoxemia and severe tachycardia (oximeter), severe anemia (POC hemoglobinometer), hypoglycemia (POC glucometer), as well as host biomarkers of inflammation predictive of bacterial infection (elevated CRP using a rapid semi-quantitative lateral-flow test and elevated PCT using a POC immunoassay system).

We chose ALMANACH, instead of the paper IMCI, as a control group, since ALMANACH is also built into an Android support tool: we were interested in comparing the impact related to the content of the algorithms rather than the format and technological features. In addition, since our goal was to reduce antibiotic prescription while ensuring optimal clinical patient outcome, and since a reduction in both antibiotic prescriptions and clinical failures using ALMANACH versus routine care has already been demonstrated, we regarded ALMANACH as the current gold standard in terms of antibiotic prescription and used it for the reference control arm [2]. In order to monitor routine care practice during the study, children were included in a routine care cohort in 2 participating health centers. After enrollment, patients were managed by routine clinicians. There was no intervention done, but we assured that essential laboratory tests and medicines were available at the health center.

Rapid diagnostic testing for malaria was done for all patients (including in the routine care cohort) using either the SD BIOLINE Malaria Ag P.f/Pan (Standard Diagnostics) or CareStart Malaria HRP2 (Access Bio) assay. Other POCTs were performed on site as recommended by the algorithms (Figs 1, 2 and S2; Table 1). Following Tanzanian national guidelines, voluntary screening for human immunodeficiency virus (HIV) antibodies using the Determine HIV-1/2 (Alere) was offered to all patients when HIV test kits were available at the health facilities. In the routine care cohort, voluntary screening was offered at the routine clinician’s discretion per standard practice. In the e-POCT arm, Hb measurement (HemoCue 201+ photometer) and oximetry (NONIN XPod with pediatric probe) were done in all patients. Children with clinical signs of lower respiratory tract infection (LRTI) (Table 1) underwent CRP testing to decide on antibiotic prescription for pneumonia. For children with FWS, the e-POCT algorithm uses combined CRP and PCT testing (Table 1). For CRP testing, we used a POC semi-quantitative assay (bioNexia CRPplus, Biomérieux). PCT values were determined on site using the B.R.A.H.M.S PCT assay on the miniVIDAS platform (Biomérieux, Thermo Scientific). Using ALMANACH, children less than 2 years with FWS underwent urine dipstick testing, as well as older children with dysuria (Table 1). Children 2 years or older with FWS were tested for typhoid using the Typhidot assay (Reszon Diagnostics International).

Follow-up

All caregivers were asked to return with the child for scheduled visits on days 3 and 7, or at any time if the parent was concerned about the child’s condition. Patients cured at day 3 were followed up by phone only on day 7. Field workers traced patients missing the day 7 follow-up. For admitted patients, the scheduled visits were done in the hospital. Patients not cured (see definition below under “Outcomes”) before day 7 were treated again per the assigned algorithm, i.e., the e-POCT algorithm if they were part of the e-POCT arm or ALMANACH if in the control arm. Patients not cured before day 7 in the routine care cohort were treated at the routine clinician’s discretion. Patients not cured at day 7 were treated per the study clinician’s judgment, and another follow-up visit was performed at day 14 to assure that the child was cured. All patients were called by phone at day 30 to assess for severe adverse events (secondary outcome measure, see below). When the algorithm recommended referral (or a routine clinician decided to refer a patient), a field worker escorted the patient to the nearest referral hospital. Patients were then admitted (or discharged home) and managed at the discretion of the responsible medical doctor in the referral hospital.

Outcomes

The primary outcome measure was the risk of clinical failure (Table 2) by day 7. At follow-up, clinicians recorded the variables that were used to calculate the criteria for clinical failure per Table 2 (the variables were either already part of the electronic algorithm assessment or otherwise recorded on paper forms). However, the clinicians were unaware of the study criteria for clinical failure and how the variables recorded were used to calculate study outcomes. The study outcomes were not used to decide on patient management. To guarantee equal assessments of the primary outcome in both arms, the following additional definition was applied: Patients were considered “not cured” and were treated again using the respective algorithm (or per the routine clinician) if either (i) the caregiver considered that the child was still ill or (ii) the child still had fever when assessed by trained field workers who did not know the content of the algorithms nor the criteria for clinical failure. The secondary outcome measures were the proportion of antibiotic prescriptions at day 0 and between day 1 and day 6, the proportion of primary referrals at day 0, and the proportion of severe adverse events (secondary hospitalizations and death) by day 30.

Sample size and statistical analyses

The sample size was computed for the primary analysis based on a 97.5% (1-sided) confidence interval (CI). To prove non-inferiority, the upper limit of this CI was to be within 3%. This non-inferiority margin was chosen because 3% was considered a clinically meaningful difference in clinical failure by day 7. The proportion of clinical failures by day 7 was estimated to be 10% in both arms based on prior studies using ALMANACH in the same area [2]. Assuming 80% statistical power, 3,140 patients were needed to show whether the difference in clinical failure by day 7 between the e-POCT and ALMANACH arms was within 3%.

Interim analyses of clinical failure rates were performed after inclusion of the first 200 and 1,000 patients. A stopping rule was predefined for an absolute difference in clinical failure by day 7 of more than 5% between e-POCT and ALMANACH. Both intention-to-treat (ITT) and per-protocol (PP) study populations were defined. The ITT population comprised all randomized patients (or patients recruited into the routine care cohort); per definition, patients who were lost to follow-up were treated as clinical failures. The PP population included all randomized patients (or patients recruited into the routine care cohort) who received the intervention (or were attended by the routine clinician) and completed the day 7 assessment (Fig 3).

Since this was a non-inferiority trial, and bias towards the null would tend to favor non-inferiority, we used a PP analysis as our primary analysis. Accordingly, all results are displayed according to PP analyses if not stated otherwise. Risk difference (RD) and risk ratio (RR) values with 95% CIs were calculated to estimate the intervention effects on the main study outcomes using the Stata cs procedure; associations between the interventions and outcomes were checked using the chi-squared test. Stratified analyses with Mantel–Haenszel estimates for RR were performed to explore the statistical heterogeneity of effect between health centers and clinicians [21]. For the primary outcome, mixed effects logistic regression was used to adjust for possible confounding covariates. Health center was modeled as a random effect, and clinician as a fixed effect. Additional predictors (age in months, weight-for-age z-score, body temperature, respiratory rate, heart rate, past medical history, and maternal education) were chosen based on clinical reasoning and were introduced into the model in a stepwise forward selection process. Predictors that were either a confounder or significantly related to the outcome were kept in the final model. Changes in odds ratio (OR) were used as approximated changes in RR since the primary outcome was rare. Kaplan–Meier survival analysis was used to compare the duration of fever between the 2 study arms.

Results

Recruitment and follow-up

Overall, between December 2014 to February 2016, 4,729 patients were screened, and 3,192 randomized (ITT population; Fig 3) into the main trial. Eight hundred fifty-eight (27%) patients were recruited in the rainy season, 736 (23%) in the post-rainy season, and the remaining 1,598 (50%) in the dry season. The PP population consisted of 3,169 patients (1,586 in the e-POCT arm and 1,583 in the control arm); 4 patients in the e-POCT arm and 5 in the control arm withdrew consent after randomization. There was only 1 algorithm deviation in the e-POCT arm and none in the ALMANACH arm. In all, 5 and 8 patients were lost to follow-up for the day 7 outcome assessment in the e-POCT and control arm, respectively. For the routine care cohort, 547 patients were recruited into the cohort between January and December 2015 (ITT population; Fig 3); 1 refused participation after consent, and 2 were lost to follow-up for the day 7 outcome assessment, leaving 544 for the PP analysis (Fig 3).

The day 30 phone follow-up could not be completed in 20/1,586 patients (1.3%) in the e-POCT arm, 25/1,583 patients (1.6%) in the control arm, and 6/544 patients (1.1%) in the routine care cohort. For the e-POCT and ALMANACH arms as well as the routine cohort, follow-up intervals for the day 3 and day 7 outcome assessments were 3 (IQR 3–3, range 2–5) and 7 (IQR 7–7, range 6–12) days, respectively. The distribution of follow-up intervals did not differ between the e-POCT and ALMANACH arms (Wilcoxon rank-sum p = 0.54 for day 3, p = 0.96 for day 7), nor between the e-POCT arm and the routine care cohort (Wilcoxon rank-sum p = 0.60 for day 3, p = 07 for day 7). Baseline characteristics did not differ between the e-POCT and ALMANACH arms (Table 3). Median age of children enrolled into the trial was 13 months (IQR 9–22); 54% (2,008/3,713) of the children were male (Table 3). In the routine care cohort, fewer patients were reported to have IMCI danger signs by the routine clinician than by clinicians in the other 2 arms (Table 3).

Clinical failure and severe adverse events in the randomized study

Table 4 shows the primary and secondary study outcomes for the randomized study (PP analysis). The results of the ITT analysis are summarized in S3 Table. In the PP population, 2.3% (37/1,586) of patients experienced clinical failure by day 7 in the e-POCT arm versus 4.1% (65/1,583) of patients in the ALMANACH arm (RD −1.7, 95% CI −3.0, −0.5; RR 0.57, 95% CI 0.38, 0.85). There was a 43% reduction in relative risk for clinical failure in the e-POCT arm compared to the ALMANACH control arm. The non-inferiority plot of clinical failure in the ITT and PP populations is displayed in Fig 4.

As depicted in Table 5, the main reduction in clinical failure between the 2 algorithms occurred by day 3 (RR 0.39, 95% CI 0.22, 0.67)—there was no significant difference in clinical failure for days 4–7 (RR 0.93, 95% CI 0.51, 1.72). The crude OR (0.56, 95% CI 0.37, 0.84) for clinical failure by day 7 was similar to the adjusted OR (0.55, 95% CI 0.36, 0.84), adjusted for significant covariates and random effects (age, axillary temperature, weight-for-age z-score [22], respiratory rate, clinician, and health center; S4 Table). We did not note statistical heterogeneity between health centers or clinicians for either the primary and the secondary outcome measures (S5 Table). There was a 68% reduction in the relative risk of severe adverse events (secondary hospitalizations and deaths) in the e-POCT arm (0.6%, 10/1,586) compared to the control arm (1.5%, 24/1,583) (RR 0.42, 95% CI 0.20, 0.87). In all, 0.5% (8/1,586) of patients in the e-POCT arm versus 1.4% (22/1,583) of children in the ALMANACH arm developed severe symptoms during follow-up (RR 0.36, 95% CI 0.16, 0.81; Table 5). Within both algorithms, having a severe classification at day 0 was associated with a higher, and having a likely viral infection classification with a lower, risk of clinical failure (Table 6). Within the ALMANACH algorithm, patients with a clinical pneumonia classification also had a higher risk of clinical failure (Table 6).

Clinical failure and severe adverse events: Comparison of the e-POCT arm with routine care

Table 7 shows the primary and secondary study outcomes for the comparison between the e-POCT arm and the routine care cohort (PP analysis). The results of the ITT analysis are provided in S6 Table. In the PP population, 4.6% (25/544) of patients treated by routine clinicians experienced clinical failure by day 7. There was a 49% lower relative risk of clinical failure in the e-POCT arm compared to routine care (RR 0.51, 95% CI 0.31, 0.84; RD −2.3, 95% CI −4.2, −0.4). Though there were fewer adverse events in the e-POCT arm (0.6%, 10/1,586) compared to routine care (1.1%, 6/544), this difference was not statistically significant (Table 7).

The main reduction in clinical failure between the e-POCT arm and routine care occurred by day 3 (RR 0.34, 95% CI 0.18, 0.67; Tables 5 and 8)—there was no significant difference in clinical failure for days 4–7 (RR 0.84, 95% CI 0.37, 1.90). We found no association between the diagnosis given by the routine clinician on day 0 and clinical failure (Table 9). Time to resolution of fever was not statistically different between the e-POCT arm and the routine care cohort (S3 Fig).

Identification of patients with severe disease at inclusion

The results of the POCTs are summarized in Table 10. The algorithm classifications are provided in Table 11. Compared to the control ALMANACH algorithm, e-POCT categorized around twice as many patients as having severe disease (and recommended referral) at day 0: 6% (103/1,586) versus 3% (55/1,583) (Table 11). A large proportion of this difference was attributable to the severe malnutrition and severe anemia classifications: e-POCT identified 3.5% (55/1,586) of patients as having severe malnutrition, while ALMANACH identified 0.1% (1/1,583; RD 3.4, 95% CI 2.5, 4.3). With e-POCT, 1.3% (20/1,586) were classified as having severe anemia using Hb testing, versus 0.3% (5/1,583) based on clinical symptoms in the ALMANACH arm (RD 0.9, 95% CI 0.3, 1.6). In the routine care cohort, only 0.4% (2/544) of patients were given a severe diagnosis by the routine clinician (Table 9).

Antibiotic prescription

In the e-POCT arm, 11.5% (182/1,586) of participants were prescribed an antibiotic treatment at day 0, versus 29.7% (470/1,583) in the ALMANACH control arm (RR 0.39, 95% CI 0.33, 0.45; Table 4). An additional 4% (53/1,404) and 4% (49/1,113) of patients who had not been prescribed an antibiotic at day 0 received antibiotics between day 1 and day 6 in the e-POCT and ALMANACH control arms, respectively. Out of these antibiotic prescriptions after the initial consultation, 47% (25/53) in the e-POCT arm and 65% (32/49) in the ALMANACH arm received antibiotics through consultation with non-study physicians.

The majority (57%, 103/182) of antibiotic prescriptions in the e-POCT arm resulted from severe classification (Fig 6). In the ALMANACH control arm, (non-severe) clinical pneumonia (see Table 1 for definition) was the leading classification for antibiotic treatment (70%, 330/470), while a severe classification was the reason for 12% (55/470) of these prescriptions (Fig 6). In contrast to that, bacterial LRTI (see Table 1 for definition) was the classification for only 6% (10/182) of antibiotic prescriptions in the e-POCT arm (Fig 6). Importantly, among patients with non-severe classification, antibiotic prescription was associated with clinical failure in the ALMANACH arm, but not in the e-POCT arm (Table 6). In the routine care cohort, upper respiratory tract infection was the most common diagnosis reported by the routine clinician resulting in antibiotic prescription (47%, 245/516 prescriptions), followed by diarrhea (17%, 88/518 prescriptions) and uncomplicated malaria (12.5%, 65/516 prescriptions).

Use of point-of-care tests in the e-POCT arm

In all, 0.3% (4/1,591) of children were identified as having hypoxemia, 0.4% (6/1,591) severe tachycardia, and 1.3% (20/1,591) severe anemia (Table 10). Among children with non-severe signs and symptoms, 2.3% (19/1,591) met the CRP cutoff criterion (≥80 mg/l) for antibiotic treatment: out of the 19 children with high CRP (≥80 mg/l), 53% (10) presented with respiratory symptoms, and 47% (9/19) had fever without localizing signs. Another 2.9% (46/1,591) of patients with fever without localizing symptoms and low CRP values had PCT values of ≥4.0 μg/l, hence meeting the criterion for antibiotic treatment. In sum, a total of 13% (52/401) of children with fever without localizing symptoms were classified as having a likely bacterial infection using PCT and CRP testing in e-POCT, compared to 17% (70/424) in the ALMANACH arm using clinical signs, urine dipstick testing, and a rapid typhoid diagnostic test (Fig 6). There was no statistically significant difference in the risk of clinical failure in patients without localizing symptoms between the 2 arms: 0.7% (3/401) for e-POCT versus 1.4% (6/415) for ALMANACH (RR 0.52, 95% CI 0.13, 2.05).

Discussion

In this multicenter, randomized, controlled non-inferiority trial including 3,739 children with febrile illnesses in Dar es Salam, Tanzania, we showed that e-POCT, a novel patient management algorithm using host biomarker POCTs, was non-inferior to a reference electronic IMCI-based algorithm (ALMANACH) in terms of clinical outcome. e-POCT actually achieved a reduction of 43% in the proportion of clinical failures by day 7 and a reduction of 58% in the proportion of severe adverse events compared to ALMANACH, while substantially lowering the proportion of antibiotic prescriptions from 30% to 11%. e-POCT categorized more patients as having severe disease on day 0. Using e-POCT, the pattern of antibiotic prescription was shifted away from non-severe respiratory infections towards patients with severe disease when compared to the IMCI-based control algorithm. We also observed routine care among children in Dar es Salaam. Compared to children treated per routine care, e-POCT resulted in a 49% reduction in relative risk of clinical failure. In the routine care cohort, 95% of children were prescribed an antibiotic at day 0. The high proportion of antibiotic prescriptions is in line with previous studies in Dar es Salaam [2].

Though the study was designed as a non-inferiority trial, the very low attrition rate makes a type I error unlikely when interpreting the results towards superiority. One could argue that a superiority trial would have been more appropriate from the start. However, we opted for a non-inferiority design, since it had already been demonstrated that ALMANACH was superior to routine care in terms of clinical outcome [2]. In our study, e-POCT also achieved superior clinical outcome compared to routine care. We hence did not expect an additional benefit in terms of clinical outcome. Rather, given the innovative features of e-POCT, we wanted to assess its safety and evaluate whether e-POCT would provide additional benefits in terms of detection of severe illness in children and reduction of antibiotic prescription compared to ALMANACH. We used an individual randomization scheme for the main comparison between e-POCT and ALMANACH, instead of cluster randomization, since the available clusters in the study region were limited. We were concerned about bias from a strong inter-cluster correlation. Given that study clinicians swapped algorithms every 2 weeks, and given that the electronic format guaranteed full algorithm adherence, the overall concern about a bias towards the null was low. The criteria used to define clinical failure were based on previous pneumonia management trials [23,24]. The validity of some of the criteria used for non-severe disease classifications should certainly be reevaluated for future trials. For example, one could argue that having persistent “clinical pneumonia” on day 3 actually does not represent clinical failure but may rather be the natural course of illness of a viral respiratory tract infection [25]. Similarly, patients with persistent fever after several days have in fact a high likelihood of having a viral infection, such as Epstein–Barr virus or Cytomegalovirus infection [26]. As expected, “negative outcome from an infection (clinical failure),” as well as severe adverse events (secondary hospitalizations and deaths) were rare in this study conducted in the primary care setting. Nevertheless, the potential implications for health outcomes for children with infections are substantial given the high burden of disease from infections: the number needed to treat to prevent 1 more clinical failure for e-POCT compared to ALMANACH in this study was 57; given that acute febrile illnesses represent the vast majority of consultations in the pediatric outpatient setting in Tanzania, this would correspond to about 1 averted clinical failure per health center per day. We included febrile children only, instead of all children presenting with acute medical complaints, since children without fever have a very low risk of having an infection that requires antibiotics [2]. Consequently, the results should be generalizable even more to non-febrile children.

In terms of using host biomarkers of inflammation for the management of febrile illness, this was the first trial to our knowledge that assessed the safety of using CRP and/or PCT testing to decide on antibiotic prescription in children in a resource-limited country. The use of CRP and PCT testing within an electronic algorithm in our study even improved clinical outcome. One recent open, randomized trial evaluated the impact of using a CRP POC testing in Vietnam among patients with respiratory symptoms, including 287 children less than 6 years of age [18]. This trial did not find a difference in clinical outcome, but was underpowered to detect such a difference because sample size calculations were done based on reduction of antibiotic prescription. Several randomized trials have assessed CRP and PCT for deciding on antibiotic prescription for respiratory infections in adults [27–30]. A series of studies assessed the accuracy of CRP and PCT in diagnosing bacterial infections in children in an outpatient setting in well-resourced settings [17,31]. These studies found moderate diagnostic accuracy for both CRP and PCT. However, one has to bear in mind that diagnostic gold standards are often imperfect. For example, for pneumonia, the most recognized current gold standard is “WHO endpoint pneumonia,” i.e., consolidation on chest radiograph [32]. However, only a (undefined) proportion of WHO endpoint pneumonia is bacterial in origin. Besides the 1 trial in Vietnam that included both children and adults, all studies of CRP and PCT in children have focused on analytical performance; none have assessed whether using these tests would change patient outcome. Though the POCTs used by e-POCT are used widely in pediatric practice, they are not available in routine care in Tanzania as of yet. Having the POCTs available, as on a single lateral-flow platform, would be ideal. This will be part of future efforts to make e-POCT scalable.

In terms of impact on antibiotic prescription when using host biomarker POCTs, a much smaller effect in terms of reduction in antibiotic prescription can be expected when these POCTs are provided without guidance (i.e., performed on all patients). For example, in the Vietnam trial, the reduction of antibiotic prescription observed was from 74% to 68% by day 14 [18]. Such a small reduction in prescription rate could be achieved on clinical grounds alone: in our study 43% (392/920) of patients in the e-POCT arm with cough did not meet clinical criteria for CRP testing and were categorized as having an upper respiratory infection. This reduction in antibiotic prescriptions achievable by the use of clinical signs alone was also demonstrated in a cluster-randomized trial assessing the use of ALMANACH [2]. Similarly, a recent cluster-randomized trial in pediatric primary care in Belgium assessing the use of CRP as a screening tool to rule out serious bacterial infection concluded that only children at higher risk for serious bacterial infection after clinical assessment should be tested using CRP [33].

To make the best use of CRP and PCT testing, we decided to include them in a patient management tool. It guided clinicians on which patients to select for testing and how to use the result in the clinical context. We also employed higher cutoffs (80 mg/l for CRP and 4 ug/l for PCT) to rule in patients likely require antibiotic treatment (instead of ruling out patients with a very low probability of having a bacterial infection using a lower CRP cutoff, such as 10 mg/l in the Vietnam trial [18]). This approach was based on analyses of host biomarker results from a Tanzanian study of the etiology of fever [32] and on the fact that patients in outpatient settings have a very low pre-test probability of having a bacterial infection [15]. e-POCT indeed selected few patients as requiring antibiotic treatment with this strategy: 1.2% (10/833) of patients with non-severe respiratory classifications and 13% (53/402) of patients with fever without localizing symptoms.

e-POCT also uses POCTs for identifying children with severe disease: an oximeter to detect hypoxemia and severe tachycardia, and a POC hemoglobinometer to detect severe anemia. e-POCT indeed classified around twice as many children as having severe disease than the control algorithm. However, this was only partially due to the POCTs employed: only very few children with hypoxemia and/or severe tachycardia were identified. The use of oximeters as part of IMCI has been advocated. However, their utility at the peripheral healthcare level should be assessed further [34]. e-POCT diagnosed 4 times as many patients with severe anemia than the IMCI-based control algorithm (and only 1/5 patients given a severe anemia diagnosis using the control algorithm actually had a low Hb value). This confirms findings from previous studies that severe anemia cannot be detected using clinical signs [35]. However, severe anemia, in turn, is an important risk factor for death from severe infections and is associated with severe infections [36,37]. Given that children cannot be preselected for Hb testing based on clinical elements, all children would have to undergo Hb testing to detect the few children with severe anemia. This strategy will become feasible only once low-cost Hb tests can be deployed to the peripheral healthcare level—if possible, tests that can be directly connected to tablets. e-POCT also detected and referred more than 20 times as many children with a severe malnutrition classification using combined weight-for-age and mid-upper arm circumference testing (compared to clinical signs in the control arm). Severe malnutrition is recognized as an important risk factor for severe outcome for infections [38–40]. Our findings corroborate the 2014 IMCI recommendations (which had not been implemented in Tanzania at the time of the study) to use anthropometric criteria in addition to clinical signs to detect children with severe malnutrition. Overall, the adequacy of the referral criteria could not be assessed in this study, which is certainly a limitation. This is because decisions to admit a child were made by providers outside of the study, and no predefined criteria were used. Hospital-based care is often low in quality and not standardized. As a result, neither admission decisions nor admission diagnoses could be used as a diagnostic gold standard [41]. In terms of routine care, the comparison between the e-POCT arm and routine care had an important limitation in that we did not take any measures in the routine care cohort (such as provision of guidelines and additional training). In contrast, study clinicians in the e-POCT arm were asked to adhere to the e-POCT algorithm for treating patients. This comparison, however, may give an impression of e-POCT’s maximum achievable public health benefit in terms of clinical outcome and antibiotic prescription.

Overall, our data provide evidence that e-POCT not only reduced the proportion of antibiotic prescriptions but also increased the targeting of children in need for antibiotic treatment compared to ALMANACH. Antibiotic treatment was indeed shifted away from non-severe respiratory infections towards severe disease classifications. Furthermore, in addition to the overall improved clinical outcome using e-POCT, antibiotic prescription was associated with clinical failure in the ALMANACH control arm, but not in the e-POCT arm. This may be because children who received unnecessary antibiotic prescription in the ALMANACH control arm experienced antibiotic side effects, or the neglect of other important supportive treatments for viral infection (such as bronchodilator treatment or rehydration). This interpretation is further underlined by our observation that time to resolution of fever did not differ between the 2 arms, while clinical failure did. The vast majority of children in our study likely had viral infections. Since we did not use any antiviral agents in the study, the overall duration of the illness could not be altered using either electronic algorithm. However, better supportive care (such as provision of hydration and bronchodilator treatment) in the e-POCT arm likely resulted in a lower rate of complications from viral infections (such as dehydration or severe respiratory distress).

Our study has several limitations that need to be addressed in further studies. First, this was a multicenter but single region study, which limits the generalizability of our findings. Given the innovative character of e-POCT, we opted for a study setting where good oversight and pediatric backup could be guaranteed. Key algorithm components should be reassessed in different geographical settings and populations with higher HIV and malnutrition rates. Second, there are concerns about using CRP testing to detect bacterial infections in patients with malaria (malaria infection in itself leads to high levels of CRP and PCT [31,42]). However, antibiotic prescription is generally not indicated in patients with uncomplicated malaria. Bacterial co-infections are nevertheless common in patients with severe malaria [43]. Using enhanced severity criteria, such as Hb testing, may actually improve detection of severe malaria and may thus help to identify children in need of concomitant antimalarial and antibiotic treatment. Third, equipment-based treatment, rather than treatment based on clinical grounds alone, is faced with challenges in terms of supply chain, and possibly cost. Compared to ALMANACH and to routine care, the additional components required for running e-POCT are an oximeter, POC Hb testing, as well as POC CRP and PCT testing in selected subgroups. On the other hand, e-POCT does not require urine testing like ALMANACH. Tablets should also be provided for both e-POCT and ALMANACH; they are now becoming available for other tasks in many health facilities. However, the costs associated with such additional components may be outweighed by the costs that are associated with antibiotic overprescription on an individual and societal level [44]. Finally, the true utility of e-POCT can only be evaluated through open implementation studies since adherence to the algorithm will be an important factor in making use of its advantages in terms of clinical outcome and antibiotic prescription.

Conclusion

e-POCT, an innovative electronic algorithm using host biomarker POCTs, has the potential to improve clinical outcome for children with febrile illnesses in low-resource settings while reducing antibiotic use through improved identification of children with severe infections and increased targeting of children in need of antibiotic prescriptions. Using CRP and PCT cutoffs, integrated into an overall disease management algorithm, for the management of children with respiratory infections and FWS was safe in terms of clinical outcome. Notwithstanding the need for replication of these findings in other geographical settings and further implementation studies, our results provide first evidence that such an innovative patient management approach is beneficial. Electronic algorithms in general are important tools to increase compliance with IMCI—the integration of POCTs would make even better use of such technologies. A key advantage of using host biomarker tests, as compared with a series of disease etiology tests, is that such an approach is likely more robust to seasonal and geographical variations in disease etiology. POCTs should include tests both for identification of patients with severe disease (e.g., severe anemia) and for detection of bacterial infections (such as CRP and PCT). To make the best use of these POCTs, they should be integrated into a patient management tool that helps clinicians not only select patient subgroups for which testing is useful, but also interpret results within an overall patient assessment. This will also allow the continuation of an integrated approach to the treatment of childhood infections as advocated by IMCI.

Supporting Information

Attachment 1

Attachment 2

Attachment 4

Attachment 5

Attachment 6

Attachment 7

Attachment 8

Attachment 9

Attachment 10

Attachment 11

Attachment 12