The use of validated and nonvalidated surrogate endpoints in two European Medicines Agency expedited approval pathways: A cross-sectional study of products authorised 2011–2018

Summary

Background

In situations of unmet medical need or in the interests of public health, expedited approval pathways, including conditional marketing authorisation (CMA) and accelerated assessment (AA), speed up European Medicines Agency (EMA) marketing authorisation recommendations for medicinal products. CMAs are based on incomplete benefit-risk assessment data and authorisation remains conditional until regulator-imposed confirmatory postmarketing measures are fulfilled. For products undergoing AA, complete safety and efficacy data should be available, and postauthorisation measures may include only standard requirements of risk management and pharmacovigilance plans. In the pivotal trials supporting products assessed by expedited pathways, surrogate endpoints reduce drug development time compared with waiting for the intended clinical outcomes. Whether surrogate endpoints supporting products authorised through CMA and AA pathways reliably predict clinical benefits of therapy has not been studied systematically. Our objectives were to determine the extent to which surrogate endpoints are used and to assess whether their validity had been confirmed according to published hierarchies.

Methods and findings

We used European Public Assessment Reports (EPARs) to identify the primary endpoints in the pivotal trials supporting products authorised through CMA or AA pathways during January 1, 2011 to December 31, 2018. We excluded products that were vaccines, topical, reversal, or bleeding prophylactic agents or withdrawn within the study time frame. Where pivotal trials reported surrogate endpoints, we conducted PubMed searches for evidence of validity for predicting clinical outcomes. We used 2 published hierarchies to assess validity level. Surrogates with randomised controlled trials supporting the surrogate-clinical outcome relationship were rated as ‘validated’. Fifty-one products met the inclusion criteria; 26 underwent CMAs, and 25 underwent AAs. Overall, 26 products were for oncology indications, 10 for infections, 8 for genetic disorders, and 7 for other systems disorders. Five products (10%), all AAs, were authorised based on pivotal trials reporting clinical outcomes, and 46 (90%) were authorised based on surrogate endpoints. No studies were identified that validated the surrogate endpoints. Among a total of 49 products with surrogate endpoints reported, most were rated according to the published hierarchies as being ‘reasonably likely’ (n = 30; 61%) or of having ‘biological plausibility’ (n = 46; 94%) to predict clinical outcomes. EPARs did not consistently explain the nature of the pivotal trial endpoints supporting authorisations, whether surrogate endpoints were validated or not, or describe the endpoints to be reported in the confirmatory postmarketing studies. Our study has limitations: we may have overlooked relevant validation studies; the findings apply to 2 expedited pathways and may not be generalisable to products authorised through the standard assessment pathway.

Conclusions

The pivotal trial evidence supporting marketing authorisations for products granted CMA or AA was based dominantly on nonvalidated surrogate endpoints. EPARs and summary product characteristic documents, including patient information leaflets, need to state consistently the nature and limitations of endpoints in pivotal trials supporting expedited authorisations so that prescribers and patients appreciate shortcomings in the evidence about actual clinical benefit. For products supported by nonvalidated surrogate endpoints, postauthorisation measures to confirm clinical benefit need to be imposed by the regulator on the marketing authorisation holders.

Keywords:

Research and analysis methods – Social sciences – Sociology – Communications – People and places – Geographical locations – Europe – Medicine and health sciences – pharmacology – Clinical medicine – Gastroenterology and hepatology – liver diseases – oncology – Cancer treatment – Cancers and neoplasms – Hematologic cancers and related disorders – Myelomas and lymphoproliferative diseases – Myelomas – Multiple myeloma – clinical trials – Phase III clinical investigation – Phase II clinical investigation – Drug research and development – Hematology – Plasma cell disorders – Chronic liver disease – Chronic hepatitis – Marketing

Introduction

The European Medicines Agency (EMA) is the central regulatory body providing recommendations on the authorisation (approval) of new medicinal products in the European Union (EU) [1]. In situations of ‘unmet medical need’ and/or ‘in the interests of public health’ [2], conditional marketing authorisation (CMA) and accelerated assessment (AA) pathways provide expedited routes to EMA authorisation recommendations.

CMAs are based on incomplete benefit-risk assessment data, and the authorisation is conditional on completing regulator-imposed postmarketing measures ‘with a view to providing comprehensive data confirming that the benefit-risk balance is positive’ [3,4]. Products undergoing AA have a shortened regulatory assessment period compared with standard assessments. They require complete safety and efficacy data to be available and may have postauthorisation conditions or restrictions, including obligations to conduct postauthorisation safety studies (PASSs) described in their risk management plans (RMPs) (Table 1) [5].

Pivotal trials or ‘main studies/trials’ that provide benefit-risk data to support authorisation recommendations may be based on ‘clinically meaningful’/‘clinical’ endpoints (or outcomes) or surrogate endpoints. A clinical endpoint/outcome is described as ‘a characteristic or variable that reflects how patients feel, function or how long they survive’ [7,8]. Surrogate endpoints are biomarkers or intermediate endpoints intended to substitute for and predict a clinical outcome [7,8]. They may be validated or nonvalidated. A validated surrogate endpoint is one in which the treatment effect on the surrogate corresponds to the effect on the intended clinical outcome and hierarchies of validity are described in the literature [7,8]. For example, blood pressure has been shown to predict mortality from cardiovascular disease. Nonvalidated surrogate endpoints are not supported by evidence demonstrating that they reliably predict clinical outcomes. For example, despite its widespread use in studies of oncology therapies, the surrogate endpoint of progression-free survival (PFS), a composite of time to tumour progression or death, has been found to have highly variable correlation with the desired clinical outcome of overall survival [9–11]. Consequently, products recommended for authorisation based on nonvalidated surrogate endpoints may not reliably provide the intended clinical benefits for patients [8,12–14].

Both validated and nonvalidated surrogate endpoints have been accepted in place of clinical endpoints by medicines regulators, including for products recommended for approval through expedited pathways. Surrogate endpoints reduce drug development time, permitting new discovery benefits to reach patients faster than waiting on clinical endpoints, and they may be practically and/or ethically preferable [6]. EMA has reported CMA as advancing authorisation by an average of 4 years, but in the United States, regulatory acceptance of surrogate outcomes in oncology trials has been estimated to save just 11 months of drug development time, on average, compared with waiting on meaningful clinical outcomes [15,16]. The ‘Accelerated Approval Pathway’ enables products to enter US markets based on clinical trials with surrogate endpoints that are only required to be ‘reasonably likely’ to predict clinical outcomes [17,18].

In the EU, pivotal trials supporting marketing authorisation recommendations through CMA and AA pathways use both clinical and surrogate endpoints. In both cases and in common with other approval pathways, postmarketing requirements include risk management and pharmacovigilance plans. Products granted CMA are subject to additional specific obligations imposed by the regulator and described in the summary of product characteristics (SpC) document, Section E of Annex II, requiring the marketing authorisation holder to collect additional data to complete the benefit-risk profile. Where a product authorisation recommendation has been based on a surrogate endpoint, specific obligations could potentially require postmarketing studies demonstrating the intended clinical benefit or validating the endpoints in the case of nonvalidated surrogates. For products undergoing AA on the basis of surrogate endpoints, the true clinical benefit of the products approved for marketing may never be established because there is no consistent requirement for confirmatory postmarketing studies.

The reliance on surrogate endpoints and the extent to which they are validated have not been studied systematically. In this cross-sectional study, our objective was to examine whether authorisations granted in Europe through 2 expedited pathways, CMA and AA, were based on clinical or surrogate endpoints. Where surrogate endpoints were used, we assessed whether they were validated or nonvalidated and determined whether postauthorisation measures to confirm clinical outcomes had been required by the regulator.

Methods

We used the EMA website search tool to identify products granted CMA between January 1, 2011, and December 31, 2018 [19]. We obtained the names of products authorised via the AA pathway during January 1, 2011, and December 31, 2017, through a request (via Regulation [EC] No. 1049/2001) to EMA; for January 1, 2018–December 31, 2018, we obtained the names from the EMA annual report for 2018 [20]. We excluded the following products: vaccines; reversal agents, e.g., idarucizimub (Praxbind, AA), for anticoagulant-associated bleeding; topical agents, e.g., cenegermin (Oxervate, AA), for ophthalmic keratitis; those without pivotal trials, e.g., ketoconazole (Ketoconazole HRA, AA), for Cushing’s disease; bleeding prophylaxis agents, e.g., human coagulation Factor X (Coagadex, AA), for Factor X deficiency-associated bleeding prevention; and those no longer authorised at time of data collection, e.g., boceprevir (Victrelis), for chronic hepatitis C infection, withdrawn during 2018 at the request of the marketing authorisation holder. We also excluded products authorised via a third expedited pathway, exceptional circumstances, products for which comprehensive data on efficacy and safety cannot be provided ‘because the condition to be treated is rare or because collection of full information is not possible or is unethical’ [21]. These products are unlikely ever to have a full dataset as is expected for products granted CMA or AA.

Noting that Banzi and colleagues had examined CMAs for the period from January 2006 to June 2015, we opted to minimise overlap by studying approvals from 2011 onwards because, by our study commencement, many products granted CMA prior to 2011 had achieved full approval owing to completion of specific obligations [6]. Our study protocol was prespecified in alignment with the objectives and comprised the steps described in the subsequent paragraphs (S1 Text).

We used European Public Assessment Reports (EPARs) to determine the endpoint(s) used in the pivotal trial(s) supporting product authorisations, the rationale for the endpoint choice, and specific obligations in the case of CMAs and postauthorisation conditions or restrictions in the case of products undergoing AA. We focused on the pivotal (or main) trials described in EPAR Section 2.5.2 because these had provided the main evidence basis for authorisation. We checked whether surrogate endpoints had an EMA qualification opinion on their acceptability or context of use for regulatory decision-making [22].

For products granted CMA, the authorisation date on the search tool and in the EPAR did not always concur; we used the search tool date. We used the SpCs to cross-check (December 2018) authorisation status and specific obligations (Annex ll, Section E) of products initially granted CMA and postauthorisation requirements or restrictions applying to products that had AA (Annex ll, Section D). Products undergoing CMA that were also granted AA are reported here as CMA because a full dataset was unlikely to exist at the time of authorisation.

We used the description of Fleming and Powers (F&P) as the basis for identifying validation studies: ‘validating a surrogate endpoint requires providing an evidence-based justification, often from randomised controlled clinical trials, that achievement of substantial effects on the surrogate endpoint reliably predicts achievement of clinically important effects on a clinically meaningful endpoint’ in conjunction with the description of Ciani and colleagues (‘Ciani’) [7,8].

We conducted PubMed searches to investigate whether surrogate endpoints had validation studies. The PubMed search terms were as follows: [‘endpoint’] and [validat* surrogate outcome OR validat* surrogate endpoint OR validat* surrogate end-point] and [‘indication’] (i.e., the ‘therapeutic indication’ described in ‘product information’ provided on the EMA website). Following reviewer advice, we repeated the searches using [valid*] instead of [validat*]. We applied filters for ‘past 10 years’ and ‘humans’. We excluded studies if they reported on an indication that was different from that for which authorisation was granted. We supplemented the PubMed searches with additional internet searches using Google and Google Scholar. Two investigators independently reviewed the search results. For discrepancies, we consulted a third investigator to reach consensus.

We applied F&P and Ciani hierarchies to categorise surrogate endpoints reported in the pivotal trials (Table 2) [7,8]. We carried out the first stage of the Ciani methods and omitted the steps that seek to establish a correlation coefficient for the surrogate–endpoint relationship. We did not include this step because our interest in the current study was to establish whether a surrogate endpoint had been validated or not as predictive of the clinical outcome, not to conduct a validation process. Moreover, correlations between surrogate endpoints and clinical outcomes may not reliably predict clinical efficacy [8,10–12,23]. Three authors categorised the pivotal trial endpoints independently according to both F&P and Ciani hierarchies and based on evidence from the literature searches. Where individual categorisations differed, we discussed to reach consensus. Surrogates of F&P ‘Level 2’ and Ciani ‘Level 1’ were considered validated. Where clinical outcomes were reported, these were categorised as Level 1 according to the F&P hierarchy. The Ciani hierarchy applies to surrogate endpoints only.

We further categorised pivotal trial endpoints according to whether they were single, composite, or multiple. Single outcomes were those for which the pivotal trial reported a single clinical or surrogate endpoint. A composite endpoint was that for which the pivotal trial reported an endpoint comprising more than one element and for which the individual elements could not be distinguished, for example, PFS, defined as the time to disease progression or death, whichever occurred first; because the numbers of deaths were not reported separately, the clinical component (death) could not be distinguished from the surrogate component, disease progression. Multiple endpoints arose when there was more than one pivotal trial supporting authorisation and these reported different primary endpoints or when a single trial reported more than one primary endpoint; for each endpoint, we recorded whether it was clinical or surrogate.

This study did not require ethics approval.

Results

Between January 2011 and December 2018, 26 products granted CMA and 25 granted AA met the inclusion criteria (Table 3). Among the CMA products, 18 were to treat malignancy, 3 were for genetic, and 2 were for infection indications, and one each was for neurological, gastrointestinal, and endocrine disorders (Table 4). Among the products granted AA, 8 were to treat malignancy, 8 were for infection, 5 were for genetic disorders, 2 were for respiratory conditions, and 1 each was for a gastrointestinal disorder and a lymphoproliferative disorder (Table 4). There were 33 pivotal trials for CMA and 58 for AA products. Among the 51 products, 5 had pivotal trials that reported clinical outcomes, and 49 had trials with surrogate endpoints. The pivotal trial endpoints are described in detail in Table 4. No studies were identified that confirmed the validity of the surrogate endpoints as predicting the intended clinical outcomes (F&P Level 2 or Ciani Level 1).

CMAs

No product was granted CMA on the basis of a trial that reported a clinical endpoint. The pivotal trials reported a single nonvalidated surrogate endpoint for 22 CMA products and a composite nonvalidated endpoint for 4.

Of the 18 oncology products, the CMAs for 14 were based on pivotal trials reporting single surrogate endpoints. These included ‘objective response rate’ (reported for crizotinib, Xalkori; brentuximab vedotin, Adcetris; vismodegib, Erivedge; osimertinib, Tagrisso; alectinib, Alecensa; avelumab, Bavencio; rucaparib, Rubraca), overall response rate (ceritinib, Zykadia; daratumumab, Darzalex; venetoclax, Venclyxto), ‘complete response’ (pixantrone dimaleate, Pixuvri), ‘complete remission’ (blinatumomab, Blincyto), and ‘major cytogenic response’ (bosutinib, Bosulif) (Table 4). For 4 oncology products, the pivotal trials reported the composite endpoint of PFS: vandetanib (Caprelsa) and cabozantinib (Cometriq) for thyroid medullary cancer, olaratumab (Lartruvo) for soft tissue sarcoma, and ixazomib (Ninlaro) for multiple myeloma (Table 4).

Both fampridine (Fampyra) for multiple sclerosis and ataluren (Translarna) for Duchenne muscular dystrophy were granted CMA based on timed walking tests. The pivotal trial endpoints for 2 products indicated for multidrug-resistant tuberculosis—bedaquiline (Sirturo) and delamanid (Deltyba)—were based on sputum culture tests. For everolimus (Votubia), indicated for tuberous sclerosis-associated angiomyolipoma, the change in astrocytoma volume was reported, and for burosomab (Crysvita) (indicated for X-linked hypophosphotaemia), the rickets severity score based on knee and hand/wrist radiographs was reported. The pivotal trials for both obeticholic acid (Ocaliva) for primary biliary cirrhosis and parathyroid hormone (Natpar) for hypoparathyroidism reported biomarker endpoints.

In the F&P hierarchy, the endpoints in the pivotal trials for all 26 CMA products were categorised as Level 3 (the surrogate is established to be reasonably likely to predict clinical benefit for a specific disease setting and class of intervention) or Level 4 (the correlate is established to be a measure of biological activity but not established to be at a higher level) [7] (Table 4). In the Ciani hierarchy, all of the endpoints were categorised as Level 3, meaning there was biological plausibility of a relation between the surrogate and the clinical outcome [8].

Eight products were supported by Phase III trials, 11 by Phase II trials, 5 by Phase I/ II trials and 2 by Phase I trials alone (Table 4). By December 2018, 9 products granted CMA during the study period had been switched to full authorisation (Table 4 and S1 Table).

Three CMA products were also granted an AA—osimertinib mesylate (Tagrisso) indicated for non-small cell lung cancer, daratumumab (Darzalex) for multiple myeloma, and olaratumab (Lartruvo) for sarcoma. Osimertinib mesylate and daratumumab were fully authorised by December 2018.

AAs

The pivotal trials for 5 of the 25 products granted AA reported clinical endpoints. For 2 products, this was a single endpoint, overall survival—abiraterone (Zytiga), for prostate cancer, and lanadelumab (Takhzyro), for hereditary angioedema (Tables 3 and 4). For 3 products—vemurafenib (Zelboraf) for melanoma, sebelioase alfa (Kanuma) for lysosomal acid lipase deficiency, and nusinersen (Spinraza) for spinal muscular atrophy—multiple different endpoints were reported in more than one trial, and for all 3, these included clinical endpoints as well as nonvalidated surrogate endpoints (Tables 3 and 4).

For 20 products, authorisation via AA was granted on the basis of trials reporting single nonvalidated surrogate endpoints (Tables 3 and 4). Eight of the products were for treatment of chronic hepatitis C virus (HCV) infection, and the pivotal trials were based on a biomarker, sustained virological response at 12 weeks following the end of treatment (SVR12). Among 7 oncology products, 5 had pivotal supporting trials that reported the composite outcome of PFS. The endpoints in the pivotal trials for 2 respiratory products, nintedanib (Ofev) for idiopathic pulmonary fibrosis and ivacaftor (Kalydeco) for cystic fibrosis, were forced vital capacity (FVC) and forced expiratory volume, respectively. The trials for 2 products for hereditary transthyretin amyloidosis, inotersen (Tegsedi) and patisiran (Onpattro), reported changes from baseline in neuropathy impairment scores. The pivotal trial endpoint for budesonide (Jorveza), indicated for eosinophilic esophagitis, was a composite of histology and symptom scores (Table 4).

The pivotal trial endpoints were categorised as F&P Level 1 for the 5 AA products where the trials reported clinical outcomes (Table 4). (The Ciani hierarchy does not apply for clinical endpoints.) The endpoints of forced expiratory volume in 1 second (FEV1) change reported for ivacaftor (Kalydeco) for cystic fibrosis and the neuropathy impairment scores reported for inotersen (Tegsedi) and patisiran (Onpattro) for hereditary transthyretin amyloidosis were categorised as F&P Level 3 and Ciani Level 2. Among the remaining products, most endpoints were categorised as F&P Level 3 and as Ciani Level 3 (Table 4).

Twenty products were supported by Phase III trials, one by a Phase II/III trial and 4 by Phase II trials (Table 4).

EPAR explanation of rationale for pivotal trial endpoints

In some cases, EPARs mentioned that the pivotal trial endpoints evaluated to support authorisation recommendations were not clinical outcomes but did not systematically indicate whether surrogate endpoints were validated or nonvalidated. S1 Table summarises the information provided in each product EPAR on the rationale for the pivotal trial endpoint(s). Among CMA products, 18 of the 26 EPARs (69%) made no mention of whether the surrogate endpoint was validated or not. Others acknowledged uncertainties or highlighted difficulties in endpoint choice. For example, for multidrug-resistant tuberculosis, in the case of bedaquiline (Sirturo), the ‘time to sputum culture conversion’ (SCC) endpoint was acknowledged as a surrogate, but there was no mention of its validity in predicting the desired clinical outcome, while for delamanid (Deltyba), its association with SCC at 24 months and with mortality were discussed (S1 Table). For some products, preference was expressed for a different outcome, e.g., pixantrone (Pixuvri) for non-Hodgkin lymphoma, where it was noted that PFS or overall survival would have been preferred over the ‘complete response’ endpoint reported.

Among products granted AA, the EPARs for 13 of the 20 products (65%) with pivotal trial surrogate endpoints had no mention of the endpoint validity (S1 Table). In some cases, the rationale for the reported endpoints was provided. For example, in the case of ivacaftor (Kalydeco) for cystic fibrosis, the endpoint ‘change in percent predicted FEV1’ was described as ‘accepted’, and FEV1 rate of decline was described as ‘demonstrated to correlate with survival and to be the strongest clinical predictor of mortality’, but no supporting reference was provided (S1 Table). For the 9 products indicated for treatment of chronic HCV infection, the endpoint in the pivotal trials was SVR12, and the EPARs noted variously that this was accepted or followed established principles but did not explain details or describe limitations. For example, with sofosbuvir/ledipasvir (Harvoni), it was noted that SVR12 ‘in practice is equivalent to cure’, but no explanation was provided to support this view.

In contrast, for budesonide (Jorveza), indicated for eosinophilic esophagitis, the rationale for the pivotal trial endpoint was clearly stated: ‘At the time of conduct of trial, no obviously “standardised” method to assess the course of the symptoms was available, and therefore the chosen parameters appear to be adequate’; a prior EMA Scientific Advice that had contributed to endpoint discussions and choice was acknowledged (S1 Table).

Postauthorisation measures

Among CMAs, Section 4 of the EPARs (and Annex ll, Section E of the SpCs) summarised the specific obligations required to complete postauthorisation measures, but these did not consistently specify whether the measures were intended to confirm clinical outcomes (S1 Table). For 4 of the 26 CMA products (15%), the required postauthorisation studies had a stated clinical endpoint, crizotinib (Xalkori) for non-small cell lung cancer, brentuximab vedotin (Adcetris) for myeloid leukaemia, ixazomib citrate (Ninlaro) for multiple myeloma, and obeticholic acid (Ocaliva) for primary biliary cirrhosis.

For 2 products, everolimus (Votubia) for tuberose sclerosis and vandetanib (Caprelsa) for thyroid neoplasm, the required postauthorisation studies had nonvalidated surrogate endpoints.

The endpoint that was to be reported in the required postauthorisation studies was not stated in the EPAR Section 4 information for 19 products granted CMA (73%). For example, in the case of daratumumab (Darzalex) for multiple myeloma, the specific obligation was written thus: “In order to address the uncertainties related to the single arm design of the pivotal study supporting the approval of Darzalex, the MAH [marketing authorisation holder] should submit the results of study MMY3003, a phase III randomised study investigating lenalidomide and dexamethasone with or without daratumumab in patients with previously treated multiple myeloma.” There was a similarly worded obligation to also provide the results of study MMY3004.

For the 25 products granted AA, Section 4 of the EPARs (and Annex ll, Section D of the SpCs) likewise indicated whether there were obligations to conduct postauthorisation measures (S1 Table)—these were obligations additional to the periodic safety update report (PSUR) and the activities and interventions detailed in the RMP. Two of the 5 (40%) products whose pivotal trials reported clinical outcomes had obligations to conduct postauthorisation studies, nusinersen (Spinraza) for spinal muscular atrophy and sebelipase alfa (Kanuma) for lysosomal acid lipase deficiency (S1 Table).

The pivotal trials for 20 of the 25 AA products (80%) reported surrogate endpoints of which 10 (50%) had postauthorisation obligations. For the 8 direct-acting antiviral products indicated for chronic HCV infection, the obligation was to conduct a prospective PASS to evaluate the recurrence of hepatocellular carcinoma associated with use of the products, a clinical safety outcome, not a clinical efficacy/effectiveness outcome. Studies with clinical outcomes were required for 2 products: ivacaftor (Kalydeco) for cystic fibrosis and siltuximab (Sylvant) for Castleman disease. The remaining 10 (50%) products had no obligations beyond their PSUR and RMP requirements. In total, 18 (90%) of the 20 products granted AA based on pivotal trials reporting surrogate endpoints had no requirement to conduct postauthorisation measures to confirm clinical efficacy/effectiveness outcomes.

Discussion

In this study, we found that most of the marketing authorisations issued between January 1, 2011, and December 31, 2018, for products assessed through 2 EMA expedited pathways, CMA and AA, were based on pivotal trials that reported nonvalidated surrogate endpoints. This information was not systematically or explicitly included in the product EPARs, including in the SpCs, the definitive, regulator-approved information for prescribers (SpC Annex 1) and patients (SpC Annex IIIB, Package Leaflet). Prescribers and patients may therefore not be aware that there is only limited evidence that the products concerned provide clinical benefit. Moreover, clinical benefit may never be established because in most cases, there was also no requirement for marketing authorisation holders to conduct confirmatory postauthorisation studies.

Other studies, mainly in the US, have highlighted the use of surrogate endpoints in supporting expedited authorisations of oncology products and the subsequent failure to confirm clinical outcomes [24–28]. This study demonstrates that a similar situation applies for non-oncology products as well as for oncology products authorised in Europe through expedited pathways.

For products granted CMA based on pivotal trials reporting surrogate endpoints, the EPARs and SpCs (including the prescriber and patient information sections) typically provided no indication of whether the surrogates were validated or not as being predictive of the intended clinical outcomes. While it was always clear that more evidence was pending and postauthorisation monitoring was being undertaken, EPARs did not consistently say whether the associated specific obligations were intended to confirm clinical benefits. The required postauthorisation studies were described in Section 4 of the EPARs, but their outcomes, in many, were not stated. Highlighting the necessity of ensuring that clinical benefits are achieved from products authorised using surrogate endpoints, olaratumab (Lartruvo), granted CMA for the treatment of soft tissue sarcoma on the basis of PFS, a nonvalidated surrogate endpoint, failed to show a survival benefit in the postauthorisation study imposed as a specific obligation [29]. (The EPAR did not state the outcome to be evaluated in this study.) In April 2019, EMA recommended withdrawal of the marketing authorisation for olaratumab, noting that this was first revocation of a CMA [30].

While it was unsurprising that none of the products granted CMA were authorised on the basis of clinical outcomes, it was astonishing that just 20% of the products granted AA were supported by pivotal trials reporting clinical outcomes. For products authorised via the AA pathway, postauthorisation requirements may relate only to safety update reporting and to the RMP, and this was the case for half of the AA products recommended for authorisation on the basis of pivotal trials reporting surrogate endpoints. The EPARs and SpCs did not explain that clinical benefits might never be confirmed. For AA products for which postauthorisation measures were required, it was clear that ‘additional monitoring’ was being undertaken, but the documents did not distinguish whether this was to confirm clinical efficacy/effectiveness or evaluate safety outcomes.

Categorisation of surrogate endpoints and regulator views on endpoints

We assessed surrogate endpoint validity according to the hierarchies of F&P and Ciani [7,8]. We judged surrogates of F&P Level 2 or Ciani Level 1 as validated. We did not find evidence at these levels for the endpoints reported in the pivotal trials of the CMA and AA products examined. In most cases, there was biological plausibility of a relation between the surrogate and the intended clinical outcome or a ‘reasonably likely’ relationship (Level 3 in both hierarchies).

None of the surrogate endpoints had an EMA qualification opinion [22], but for several conditions, we found that EMA guidance on treatment endpoints existed though it was not consistently mentioned in the corresponding product EPARs. For instance, among direct-acting antiviral products for chronic HCV infection, the primary endpoint in the pivotal trials, SVR12, was a nonvalidated surrogate endpoint, F&P Level 3 and Ciani Level 3 according to our methods. EMA guidance (2016) recommended SVR12 as the primary efficacy endpoint ‘for studies aiming at defining cure rate’ but offered no evidence to support its validity for this purpose and appears to recognise the limitations of SVR in specifying that ‘a representative subset’ of patients achieving SVR12 should be monitored for 12 months from end of treatment to assess durability, while those not achieving SVR12 should be monitored for 3 years [31]. Whilst SVR is a widely used endpoint in chronic HCV treatment trials and observational data from a single US-based population describe an association with mortality outcomes, there is debate about whether it reflects long-term clinical outcomes [7, 32–35]. A comprehensive review of randomised trials of direct-acting antivirals for chronic HCV infection concluded that ‘SVR is still an outcome that needs proper validation in randomised clinical trials’ [32].

In the case of cystic fibrosis, the primary endpoint in the pivotal trial for ivacaftor (Kalydeco) was the rate of decline in FEV1, a surrogate endpoint associated with morbidity and mortality and categorised as F&P Level 3 and Ciani Level 2 according to our methods. Challenges in its use are well-acknowledged, and an EMA-hosted cystic fibrosis stakeholder workshop in 2012 reported that ‘FEV1, despite its major limitations, still remains an important outcome measure for clinical efficacy’ [36, 37]. The workshop was not mentioned in the EPAR or SpC.

Surrogate categorisations: Evidence from the literature

The surrogate endpoint of SCC within 2 months, reported in the pivotal studies for 2 products granted CMA to treat multidrug-resistant tuberculosis, bedaquiline fumarate (Sirturo) and delamanid (Deltyba), has been shown to correlate well with recurrence of infection in drug-sensitive tuberculosis, but data on its reliability in multidrug-resistant tuberculosis were unavailable, hence we categorised it as F&P Level 3, Ciani Level 3 [38].

Nintedanib (Ofev), granted AA for treatment of idiopathic pulmonary fibrosis, was similarly categorised. The pivotal trials used the endpoint ‘annual rate of decline in FVC’. Discussions in the literature consider options for analysing FVC changes; although FVC of itself has been shown to predict mortality, its annual rate of decline has not been shown to predict disease progression [39, 40].

For the neuropathy impairment scores reported for inotersen (Tegsedi) and patisiran (Onpattro) for hereditary transthyretin amyloidosis, the Norfolk quality of life score demonstrated correlation with clinical outcomes in an observational study, while in a subset of patients recruited to a randomised trial, it was shown that clinicians could be trained to accurately assess neuropathy signs using a modified neuropathy impairment score (mNIS+7_Ionis) [41, 42]. We categorised these endpoints as F&P Level 3 and Ciani Level 2.

Among the 17 CMA and 8 AA oncology products, most pivotal trials reported surrogate endpoints of F&P Level 4 and Ciani Level 3 according to our methods. Correlations between surrogate endpoints and clinical outcomes have been investigated quite extensively in oncology treatments, especially between the composite surrogate endpoint, PFS, and overall survival, and found wanting [10, 24, 25, 43, 44]. Heterogeneous correlations are reported across studies, even where the same endpoint was investigated in the same cancer type [10]. Using a validated framework—the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS)—to assess the ‘meaningful clinical benefit’ of 38 oncology treatments authorised for 70 indications between 2011 and 2016, the threshold for benefit was achieved in only 21% of the indications [45]. When a more rigorous but nonvalidated version of the framework was applied, just 11% met the threshold. The endpoints in the trials for 8 CMA products and 3 AA products were assessed according to the Revised Evaluation Criteria in Solid Tumours (RECIST), a set of rules for reporting of treatment responses for solid tumours [46]. RECIST aims to ensure consistency in measuring progression of disease but does not assess surrogate endpoint validity.

EMA guidance on the evaluation of oncology products states that there should be ‘sufficient evidence available demonstrating that the chosen primary endpoint can provide a valid and reliable measure of clinical benefit’ noting that ‘convincingly demonstrated favourable effects on survival’ were ‘the most persuasive outcome of a clinical trial’ [47]. ‘Acceptable’ primary endpoints in Phase III confirmatory trials were cure rate, overall survival, and PFS/disease-free survival, but we found no discussion on the validity of PFS in reflecting clinical outcomes. Objective response rate, the pivotal trial endpoint for 7 CMA products, was judged suitable in Phase II evaluation of activity studies, but its validity as a surrogate was not discussed. Neither overall response nor complete response/remission, the pivotal trial endpoints for 5 products, was mentioned [47].

Thus, where product authorisation was supported by pivotal trials reporting surrogate endpoints, the major issue of concern highlighted by our findings is that the EPARs, including the SpCs, did not clearly describe whether the surrogates were validated as reliably reflecting clinical outcomes. This is an omission that could feasibly be addressed in both documents. Our findings do not mean that the appropriateness of pivotal trial endpoints was not discussed during regulatory evaluations, but if it was discussed, the reasoning was not systematically recorded in the main public assessment report or in the SpC.

Surrogate endpoint validity is a matter of concern for decision-makers other than medicines regulators. For example, the National Institute for Health and Care Excellence (NICE) in the United Kingdom recommends that ‘when the use of “final” clinical endpoints is not possible and “surrogate” data on other outcomes are used to infer the effect of treatment on mortality and health-related quality of life, evidence in support of the surrogate-to-final endpoint outcome relationship must be provided together with an explanation of how the relationship is quantified’ [48]. EUnetHTA, the European network of health technology assessment agencies, has provided guidance on the use of surrogate endpoints in relative effectiveness assessment of pharmaceuticals [49]. In Germany, the Institute for Quality and Efficiency in Healthcare has published recommendations for the validation of surrogate endpoints in oncology, has documented the limited benefits of new medicines entering the German healthcare system, and called for EU and national action to define public health goals and to revise the legal and regulatory framework to properly meet patient needs [50, 51].

Benefits and drawbacks of expedited authorisations

Studies investigating the additional therapeutic value of products authorised through expedited pathways have variable findings. A study of expedited drug development and approval programs in the US, 1987–2014, criticised the regulator’s oversight of applications to the programs after finding that increasing numbers were driven by drugs not first in class and therefore not likely to provide noticeable clinical advances over existing products [17]. Quality adjusted life year gains have been reported with some expedited authorisations compared with standard approvals in US healthcare settings, but safety problems leading to market withdrawal of products have also emerged [26, 27, 50]. Although a study of products authorised via 2 EMA expedited pathways compared with standard authorisations, 1999–2009, reported no increase in postmarketing safety alerts or withdrawal, in terms of clinical value, it appears that most new cancer drugs authorised on the basis of surrogate outcomes provided little or no survival or quality of life benefit [27, 28, 52, 53].

Postauthorisation requirements in expedited authorisations

A recurring criticism of expedited authorisations is the failure by marketing authorisation holders to undertake and/or complete postmarketing obligations in a timely and rigorous manner, both in the EU and the US [5,14–18,27, 54–56]. Notably, completed postapproval studies of oncology products granted accelerated approval in the US (2009–2013) mainly reported surrogate endpoints, not clinical outcomes, highlighting that without ensuring that surrogates are validated, clinical benefit may never be established [24, 25, 27]. In our study, which examined products for oncology and beyond, the EPARs in most cases did not state the endpoint for the postmarketing studies imposed as specific obligations on products granted CMA. Of the 6 products for which the endpoints were stated, these were clinical outcomes for 4 products, but nonvalidated surrogate endpoints were required for 2 products.

Products granted AA may have no postmarketing requirements beyond PSUR and RMP activities. Of the 20 out of 25 AA products in our study that had pivotal trials reporting surrogate endpoints, 18 (90%) had no requirement to undertake postmarketing measures to confirm clinical efficacy/effectiveness (rather than safety) outcomes. Thus, their clinical benefits may never be known.

Limitations

The main limitation of the study is that the findings apply to products recommended for authorisation through 2 expedited assessment pathways and may not be generalisable to products authorised through the standard pathway. For the CMA and AA products examined, the EPAR documentation is extensive, and it is possible we overlooked some information (or products) during data collection. We tried to minimise subjectivity in our decisions on whether surrogate endpoints were validated or nonvalidated by searching the literature widely for supporting validation studies, but we may not have found all of the relevant studies. We further sought to minimise subjectivity by applying methods from 2 independent systems to categorise surrogate endpoints. Where individual author categorisations differed, we reached agreement by consensus but acknowledge the final categorisation assumed that our searches had found all of the relevant studies of endpoints. We searched for EMA qualification opinions or other guidance relating to the endpoints reported in pivotal trials. We found guidance in 2 cases that helped to explain the endpoint choice although neither was mentioned in the associated product EPARs, but we may have overlooked other cases.

We focused on the primary endpoints reported in the pivotal supporting trials and did not collect information on secondary endpoints or on outcomes of nonpivotal trials. These may have reported some clinical outcomes but were obviously not adequate as primary evidence; otherwise, they would have been considered as such.

Conclusion

This is, to our knowledge, the first systematic study of the use of surrogate endpoints to support marketing authorisation of products assessed through both CMA and AA expedited regulatory pathways in the EU. The extensive use of nonvalidated surrogate endpoints is concerning because the likelihood that treatment will provide the intended clinical benefit is unknown. In the current study, it was not clear from the publicly available information about the products whether surrogate endpoints were validated or nonvalidated. Neither was it clear whether postauthorisation measures—when they were required—would confirm clinical benefits. Because products authorised through these pathways are intended to satisfy unmet need or are in the public interest, the marketing authorisation holders should ultimately be required to demonstrate that they fulfil these goals.

It would be helpful for patients, prescribers, and healthcare providers broadly if the regulator ensured that EPARs, including SpC Annex I (Prescriber information) and SpC Annex III (Package leaflet), as well as the website ‘authorisation details’ summary, consistently provide explicit information on the nature of the pivotal study endpoints supporting marketing authorisations. When surrogate endpoints are used, this needs to indicate their level of validity, the rationale for their acceptance (including whether there was a qualification opinion, whether scientific advice had been provided, or whether EMA had otherwise provided guidance), and their limitations in reflecting intended clinical outcomes. If surrogate endpoints have supported the authorisation, then postauthorisation measures to be completed in a reasonable time frame are needed, including for rare disease treatments, to confirm that the clinical outcomes are achieved. When such measures fail to confirm clinical benefit, then—as in the case of olaratumab—the regulator should consider withdrawal of the marketing authorisation.

Supporting information

S1 Text [docx]
Study protocol.

S1 Table [xlsx]
Summary information on products granted CMA or undergoing AA (January 1, 2011–December 31, 2018).