The Imperative to Share Clinical Study Reports: Recommendations from the Tamiflu Experience

article has not abstract

Published in the journal: . PLoS Med 9(4): e32767. doi:10.1371/journal.pmed.1001201
Category: Policy Forum
doi: https://doi.org/10.1371/journal.pmed.1001201

Summary

article has not abstract

Summary Points

Systematic reviews of published randomized clinical trials (RCTs) are considered the gold standard source of synthesized evidence for interventions, but their conclusions are vulnerable to distortion when trial sponsors have strong interests that might benefit from suppressing or promoting selected data.
More reliable evidence synthesis would result from systematic reviewing of clinical study reports—standardized documents representing the most complete record of the planning, execution, and results of clinical trials, which are submitted by industry to government drug regulators.
Unfortunately, industry and regulators have historically treated clinical study reports as confidential documents, impeding additional scrutiny by independent researchers.
We propose clinical study reports become available to such scrutiny, and describe one manufacturer's unconvincing reasons for refusing to provide us access to full clinical study reports. We challenge industry to either provide open access to clinical study reports or publically defend their current position of RCT data secrecy.

Regulatory approval of new drugs is assumed to reflect a judgment that a medication's benefits outweigh its harms. Despite this, controversy over approved drugs is common. If sales can be considered a proxy for utility, the controversies surrounding even the most successful drugs (such as blockbuster drugs) seem all the more paradoxical, and have revealed the extent to which the success of many drugs has been driven by sophisticated marketing rather than verifiable evidence [1],[2]. But even among institutions that aim to provide the least biased, objective assessments of a drug's effects, determining “the truth” can be extremely difficult.

Consider the case of the influenza antiviral Tamiflu (oseltamivir). Prior to the global outbreak of H1N1 influenza in 2009, the United States alone had stockpiled nearly US$1.5 billion dollars worth of the antiviral [3]. As the only drug in its class (neuraminidase inhibitors) available in oral form, Tamiflu was heralded as the key pharmacologic intervention for use during the early days of an influenza pandemic when a vaccine was yet to be produced. It would cut hospitalizations and save lives, said the US Department of Health and Human Services (HHS) [4]. The Advisory Committee on Immunization Practices (ACIP, the group the US Centers for Disease Control and Prevention [CDC] uses to form national influenza control policy) said it would reduce the chances of developing complications from influenza [5]. So, too, did the Australian Therapeutic Goods Administration [6] and the European Medicines Agency (EMA) [7].

Most (perhaps all) of these claims can be traced back to a single source: a meta-analysis published in 2003 that combined ten randomized clinical trials conducted during the late 1990s by the manufacturer prior to US registration of the drug [8]. This analysis, conducted by Kaiser and colleagues, proposed that oseltamivir treatment of influenza reduced both secondary complications and hospital admission. In contrast, the Food and Drug Administration (FDA), which approved Tamiflu in 1999 and was aware of these same clinical trials, concluded that Tamiflu had not been shown to reduce complications, and required an explicit statement in the drug's label to that effect [9]. FDA even cited Roche, Tamiflu's manufacturer, for violation of the law for claims made to the contrary [10].

Nor did the FDA approve an indication for Tamiflu in the prevention of transmission of influenza [9],[11]. This assumption was at the heart of the World Health Organization's (WHO) proposed plan to suppress an emergent pandemic through mass prophylaxis [12]. While the WHO recently added Tamiflu to its Essential Medicines list, if FDA is right, the drug's effectiveness may be no better than aspirin or acetaminophen (paracetemol). The FDA has never clarified the many discrepancies in claims made over the effects of Tamiflu. Although it may have limited approval indications accordingly, the FDA has never challenged the US HHS or the US CDC for making far more ambitious claims. This means that critical analysis by an independent group such as a Cochrane review group is essential.

But which data should be used? In updating our Cochrane review of neuraminidase inhibitors, we have become convinced that the answer lies in analyzing clinical study reports rather than the traditional published trials appearing in biomedical journals [13]. Clinical study reports contain the same information as journal papers (usually in standardized sections, including an introduction, methods, results, and conclusion [14]), but have far more detail: the study protocol, analysis plan, numerous tables, listings, and figures, among others. They are far larger (hundreds or thousands of pages), and represent the most complete synthesis of the planning, execution, and results of a clinical trial. Journal publications of clinical trials may generate media attention [2], propel researchers' careers, and generate some journals a revenue stream [15]. However, when regulators decide whether to register a new drug in a manufacturer's application, they review the trial's clinical study report.

In 2010, we began our Cochrane review update using clinical study reports rather than published papers [16]. We obtained some sections of these clinical study reports for the ten trials appearing in the Kaiser 2003 meta-analysis from Tamiflu's manufacturer, Roche—around 3,200 pages in total. In 2011, we obtained additional sections of clinical study reports for Tamiflu through a Freedom of Information request to the EMA, amounting to tens of thousands of pages. While extensive and detailed, it is important to note that what we have obtained is just a subset of the full clinical study reports in Roche's possession. Nonetheless, Box 1 provides a list of details we have already discovered—and would have never discovered without access to these documents. This information has turned our understanding of the drug's effects on its head. Other drugs for which previously unpublished, detailed clinical trial data have radically changed public knowledge of safety and efficacy include Avandia, Neurontin, and Vioxx (Table 1).

Box 1. What Is Missed without Access to Tamiflu Clinical Study Reports

Knowledge of the total denominator. (How many trials were conducted on this drug that might fit the systematic review inclusion criteria?) [13]
Realization that serious adverse events (SAEs) occurred in trials for which SAEs were not reported in published papers [13].
Understanding what happened in some trials that were published 10 years after completion [43].
Vital details of trials (content and toxicity profile of placebos, mode of action of drug, description and temporality of adverse events) [11].
Authorship is not consistent with published papers [44] (although, if a review's inclusion criteria include clinical study reports, authorship is not an issue, as the responsibility is clearly the manufacturer's).
Rationale for alternatively classifying outcomes such as pneumonia as a complication or an adverse event [16].
Ability to know whether key subgroup analysis (influenza-infected subjects) is valid [11].
Assessment of validity of previously released information on the drug (articles, reviews, conferences, media, etc.).
Realization that Roche's claim of Tamiflu's mode of action [45] appears inconsistent with the evidence from trials [11],[46].

**Tab. 1. Different sources of data for the uncovering of failures in reporting of safety and effectiveness of some examples of new drugs.**

An Urgent Call for a Debate on the Ethics of Data Secrecy

Taken together, these experiences suggest that any attempt at reliable evidence synthesis must begin with clinical study reports. Yet we are aware of only three other groups of independent researchers conducting a systematic review based entirely (or mostly) on these and other regulatory documents [17]–[19]. We can think of two major reasons this might be. First, outside of regulatory agencies, few researchers have ever heard of clinical study reports. Second, clinical study reports are massive in size and difficult to obtain, traditionally shared only with regulators. The first problem seems tractable, but gaining better access to manufacturers' clinical study reports requires shifting the status quo from a default position of confidentiality to one of disclosure.

In the mid-20th century, regulatory agencies became increasingly responsible to the public for ensuring the safety and effectiveness of approved medicines. This rise paralleled an increase in the number and complexity of clinical trials. Both in the United States and Europe, manufacturers would come to submit trial data to regulators with the assurance that authorities would treat all trade secret data as confidential. Even following passage of the 1966 Freedom of Information Act (FOIA), the FDA maintained that safety and effectiveness test data were not subject to the FOIA [20],[21]. One sign things may be changing came in November 2010, when the EMA announced its intention to make all industry clinical study reports about a drug publicly available as a matter of standard practice after reaching a regulatory decision [22],[23]. The EMA has also already improved its handling of data under Freedom of Information requests. But as we learned was true for Tamiflu, regulators may not be in possession of full trial reports of all studies on a given intervention, implying the necessity of obtaining reports from manufacturers [11]. But at present, industry seems extremely reluctant to make its clinical study reports freely available.

In addition to clinical study reports, we also need access to regulatory information. By the very nature of their professional mandate, regulators may conduct some of the most thorough evaluations of a trial program, and efforts like ours aimed at up-to-date evidence synthesis are seriously deprived without access to their reports. While the FDA has increasingly published its reviews, memos, and other correspondence on its Drugs@FDA website (http://www.accessdata.fda.gov/scripts/cder/drugsatfda/), and additional documents are accessible through FOIA requests, the process can be very time-consuming, taking months or even years. Moreover, regulatory agencies' lack of public inventories of their documentary holdings complicates the retrieval of information. Ideally, we would also have details of the regulators' deliberations, which can serve as signposts to important issues that need investigating.

In December 2009, after we voiced serious concerns in the BMJ about Tamiflu's alleged ability to reduce complications [24], Roche wrote that it was “very happy to have its data reviewed by appropriate authorities or individuals,” and publicly pledged to release “full study reports” for ten trials “within the coming days” [25]. But despite extensive correspondence over the next year and a half, Roche refused to provide any more than portions of the clinical study reports for the ten Kaiser studies (Table 2) and no reports for any of the additional Tamiflu trials we had subsequently identified and requested (Table 3). Reasons for refusing to share the full reports on Tamiflu kept changing, and none seemed credible (Tables 2 and 3).

**Tab. 2. Roche's reasons for not sharing its clinical study reports and the authors' response (for 10 trials in Kaiser meta-analysis <em class="ref">[8]</em>).**

**Tab. 3. Roche's reasons for not sharing its clinical study reports and the authors' response (for other Tamiflu trials).**

There are strong ethical arguments for ensuring that all clinical study reports are publicly accessible. It is the public who take and pay for approved drugs, and therefore the public should have access to complete information about those drugs. We should also not lose sight of the fact that clinical trials are experiments conducted on humans that carry an assumption of contributing to medical knowledge. Non-disclosure of complete trial results undermines the philanthropy of human participants and sets back the pursuit of knowledge.

Potentially valid reasons for restricting the full public release of clinical study reports include: ensuring patient confidentiality (although this could be remedied with redaction); commercial secrets (although these should be clear after drugs have been registered, and we found no commercially sensitive information in the clinical study reports received from EMA with minimal redactions); and finally, industry concerns over adversaries' malicious “cherry picking” over large datasets (which could be reduced by requiring the prospective registration of research protocols). None appear insurmountable.

With the EMA's stated intentions on far wider data disclosure, we hope the debate may soon shift from one of whether to release regulatory data to the specifics of doing so. But until these policies go into effect—and perhaps even after they do—most drugs on the market will remain those approved in an era in which regulators protected industry's data [26]. It is therefore vital to know where industry stands. If drug companies have legitimate reasons for maintaining the status quo of treating all of their data as trade secret, we have yet to hear them. We are all ears.

Supporting Information

Zdroje

1. BrodyHLightDW 2011 The inverse benefit law: how drug marketing undermines patient safety and public health. Am J Public Health 101 3 399 404

2. SmithR 2005 Medical journals are an extension of the marketing arm of pharmaceutical companies. Plos Med 2 5 e138 doi:10.1371/journal.pmed.0020138

3. The New York Times 28 April 2009 Tamiflu (drug). The New York Times. Available: http://topics.nytimes.com/topics/news/health/diseasesconditionsandhealthtopics/tamiflu-drug/index.html. Accessed 6 March 2012

4. U.S. Department of Health and Human Services 2005 HHS pandemic influenza plan. Available: http://www.hhs.gov/pandemicflu/plan/pdf/HHSPandemicInfluenzaPlan.pdf. Accessed 6 March 2012

5. HarperSAFukudaKUyekiTMCoxNJBridgesCB 2004 Prevention and control of influenza: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Recomm Rep 53 RR-6 1 40

6. Roche Products Pty Limited 2011 TAMIFLU® capsules: consumer medicine information. Available: http://www.roche-australia.com/fmfiles/re7229005/downloads/anti-virals/tamiflu-cmi.pdf. Accessed 6 March 2012

7. European Medicines Agency 2011 Summary of product characteristics (Tamiflu 30 mg hard capsule). Available: http://www.ema.europa.eu/docs/en_GB/document_library/EPAR_-_Product_Information/human/000402/WC500033106.pdf. Accessed 5 February 2012

8. KaiserLWatCMillsTMahoneyPWardPHaydenF 2003 Impact of oseltamivir treatment on influenza-related lower respiratory tract complications and hospitalizations. Arch Intern Med 163 14 1667 1672

9. Hoffman-La Roche, Ltd. 2011 Product label. Tamiflu (oseltamivir phosphate) capsules and for oral suspension. Available: http://www.accessdata.fda.gov/drugsatfda_docs/label/2011/021087s056_021246s039lbl.pdf. Accessed 6 March 2012

10. U.S. Food and Drug Administration 14 April 2000 NDA 21-087 TAMIFLU (oseltamivir phosphate) MACMIS ID#8675. Available: http://www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/EnforcementActivitiesbyFDA/WarningLettersandNoticeofViolationLetterstoPharmaceuticalCompanies/UCM166329.pdf. Accessed 6 March 2012

11. JeffersonTJonesMADoshiPDel MarCBHeneghanCJ 2012 Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev 2012 1 CD008965

12. World Health Organization 2007 WHO interim protocol: rapid operations to contain the initial emergence of pandemic influenza. Available: http://www.who.int/influenza/resources/documents/RapidContProtOct15.pdf. Accessed 6 March 2012

13. JeffersonTDoshiPThompsonMHeneghanC 2011 Ensuring safe and effective drugs: who can do what it takes? BMJ 342 c7258

14. International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use 1996 Guideline for industry: structure and content of Clinical Study Reports (ICH E3). Available: http://www.fda.gov/downloads/regulatoryinformation/guidances/ucm129456.pdf. Accessed 6 2012

15. LundhABarbateskovicMHróbjartssonAGøtzschePC 2010 Conflicts of interest at medical journals: the influence of industry-supported randomised trials on journal impact factors and revenue – cohort study. PLoS Med 7 10 e1000354 doi:10.1371/journal.pmed.1000354

16. JeffersonTJonesMADoshiPDel MarCBHeneghanCJ 2011 Cochrane Database Syst Rev Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children - a review of unpublished data. 2011 CD008965 Available: http://doi.wiley.com/10.1002/14651858

17. GøtzschePCJørgensenAW 2011 Opening up data at the European Medicines Agency. BMJ 342 d2686

18. Yale School of Medicine 2011 Yale University Open Data Access Project (YODA Project). Available: http://medicine.yale.edu/core/projects/yodap/index.aspx. Accessed 6 March 2012

19. EydingDLelgemannMGrouvenUHarterMKrompM 2010 Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ 341 c4737

20. HalperinRM 1979 FDA disclosure of safety and effectiveness data: a legal and policy analysis. Duke Law Journal 1979 1 286 326

21. KesselheimASMelloMM 2007 Confidentiality laws and secrecy in medical research: improving public access to data on drug safety. Health Aff (Millwood) 26 2 483 491

22. European Medicines Agency 2010 Output of the European Medicines Agency policy on access to documents related to medicinal products for human and veterinary use. Available: http://www.ema.europa.eu/docs/en_GB/document_library/Regulatory_and_procedural_guideline/2010/11/WC500099472.pdf. Accessed 6 March 2012

23. PottA 2011 EMA's response to articles. BMJ 342 d3838

24. JeffersonTJonesMDoshiPDel MarC 2009 Neuraminidase inhibitors for preventing and treating influenza in healthy adults: systematic review and meta-analysis. BMJ 339 b5106

25. SmithJ on behalf of Roche 2009 Point-by-point response from Roche to BMJ questions. BMJ 339 b5374

26. AbrahamJ 1995 Science, politics and the pharmaceutical industry: controversy and bias in drug regulation. First edition. Routledge

27. CohenD 2010 Rosiglitazone: what went wrong? BMJ 341 c4848

28. RosenCJ 2007 The rosiglitazone story—lessons from an FDA advisory committee meeting. N Engl J Med 357 9 844 846

29. RichterBBandeira-EchtlerEBergerhoffKClarCEbrahimSH 2007 Rosiglitazone for type 2 diabetes mellitus. Cochrane Database Syst Rev 2007 3 CD006063

30. NissenSEWolskiK 2007 Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 356 24 2457 2471

31. LenzerJ 2003 Whistleblower charges drug company with deceptive practices. BMJ 326 7390 620

32. KesselheimASMelloMMStuddertDM 2011 Strategies and practices in off-label marketing of pharmaceuticals: a retrospective analysis of whistleblower complaints. PLoS Med 8 e1000431 doi:10.1371/journal.pmed.1000431

33. SteinmanMABeroLAChrenM-MLandefeldCS 2006 Narrative review: the promotion of gabapentin: an analysis of internal industry documents. Ann Intern Med 145 4 284 293

34. LenzerJ 2004 Pfizer pleads guilty, but drug sales continue to soar. BMJ 328 7450 1217

35. SteinmanMAHarperGMChrenM-MLandefeldCSBeroLA 2007 Characteristics and impact of drug detailing for gabapentin. PLoS Med 4 4 e134 doi:10.1371/journal.pmed.0040134

36. BombardierCLaineLReicinAShapiroDBurgos-VargasR 2000 Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. N Engl J Med 343 21 1520 1528

37. CurfmanGDMorrisseySDrazenJM 2005 Expression of concern: Bombardier et al., “Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis,” N Engl J Med 2000; 343:1520–8. N Engl J Med 353 26 2813 2814

38. WilsonD 22 November 2011 Merck to pay $950 million over Vioxx]. The New York Times. Available: https://www.nytimes.com/2011/11/23/business/merck-agrees-to-pay-950-million-in-vioxx-case.html. Accessed 6 March 2012

39. TopolEJ 2004 Failing the public health–rofecoxib, Merck, and the FDA. N Engl J Med 351 17 1707 1709

40. DoshiP 2009 Neuraminidase inhibitors–the story behind the Cochrane review. BMJ 339 b5164

41. GodleeF 2009 We want raw data, now. BMJ 339 b5405

42. MelvilleNA 20 January 2012 Tamiflu data called inconsistent, underreported. Available: http://www.medscape.com/viewarticle/757226 [requires free registration]

43. DutkowskiRSmithJRDaviesBE 2010 Safety and pharmacokinetics of oseltamivir at standard and high dosages. Int J Antimicrob Agents 35 5 461 467

44. CohenD 2009 Complications: tracking down the data on oseltamivir. BMJ 339 b5387

45. WelliverRMontoASCarewiczOSchattemanEHassmanM 2001 Effectiveness of oseltamivir in preventing influenza in household contacts: a randomized controlled trial. JAMA 285 6 748 754

46. Gilead Sciences 1999 Roche announces new data on recently approved Tamiflu™, first pill to treat the most common strains of influenza (A & B). Available: http://www.gilead.com/pr_942082531. Accessed 6 March 2012