Next Stop, Don't Block the Doors: Opening Up Access to Clinical Trials Results

Summary

article has not abstract

2008 has been a good year for access to research. Effective New Year's Day, both the Canadian Institutes of Health Research [1] and the Howard Hughes Medical Institute [2] require publicly accessible archiving of papers published by their grantees. Also in January, the European Research Council announced its European Union–wide open-access mandate [3]. In February, the Harvard Faculty of Arts and Sciences voted to give the University a worldwide license to exercise copyright in each faculty member's scholarly articles for the purpose of making these articles freely available [4]; Harvard Law School committed to mandatory free access in May [5]. In March, the European University Association endorsed open-access repositories [6], and in April the United States National Institutes of Health Public Access Policy [7] took effect, bringing America's leading sponsor of biomedical research into the impressive circle of agencies that require archiving of papers resulting from the research they fund. Judging by the ever-increasing number of submissions to PLoS journals, authors appear to be voting with their manuscripts for open access to research.

The year is only half over, however, and at least one important milestone is still to come. As of September 27, 2008, the US Food and Drug Administration Amendments Act of 2007 (FDAAA) will require that clinical trials results be made publicly available on the Internet through an expanded “registry and results data bank” [8].

Under FDAAA, enrollment and outcomes data from trials of drugs, biologics, and devices (excluding phase I trials) must appear in an open repository associated with the trial's registration, generally within a year of the trial's completion, whether or not these results have been published. The new law is innovative in bridging the gap between a clinical trial's registration at inception (now an established requirement for publication) and the public archiving of its final peer-reviewed report.

For each trial falling within its scope, the law requires the posting of a table of “demographic and baseline characteristics” of the study participants, as well as a “table of values for each of the primary and secondary outcome measures for each arm of the clinical trial, including the results of scientifically appropriate tests of the statistical significance.” Safety outcomes must be posted as of 2009, and further information may be required in future years. These are not just recommendations; the law imposes fines of up to US$10,000 per day for noncompliance.

PLoS Medicine and the other PLoS journals endorse timely and accessible reporting at all stages of clinical drug and device development. As we now state in our Author Guidelines: “PLoS supports the public disclosure of all clinical trial results, as mandated for example by the FDA Amendments Act, 2007. Prior disclosure of results on a public website such as clinicaltrials. gov will not affect the decision to peer review or acceptance of papers in PLoS journals” [9].

We are not alone in favoring such availability of results. In January 2008, the BMJ published an editorial supporting FDAAA's “great leap forward for public disclosure” and noting that “[t]he BMJ will consider disclosed trials and urges other journals to do the same…” [10]. In May, the Canadian Institutes of Health Research convened a meeting of the PROCTOR group (Public Reporting Of Clinical Trials Outcomes and Results) to launch “an international dialogue of constituencies interested in results reporting” with the goal of “contributing toward the development of international standards for results disclosure” [11]. In June, members of the World Health Organization's Registry Platform Working Group on the Reporting of Findings of Clinical Trials advanced a position that “the findings of all clinical trials must be made publicly available,” but noted that “Although some journal editors have acknowledged the changing climate around results registration and reporting…they may have a conflict of interest in that they will probably want the key (and potentially most exciting) messages from a trial to appear first, and perhaps exclusively, in their publication” [12].

Indeed, one criterion that editors must determine is how much data can be publicly presented without constituting prior publication—standard grounds for disqualifying a manuscript from consideration. In practical terms, this determination indicates the extent of a journal's support for public disclosure. In 2007, prior to the passage of FDAAA, the International Committee of Medical Journal Editors (ICMJE) had announced that its 12 affiliate journals would permit very limited prepublication presentation of results “posted in the same clinical trials registry in which the primary registration resides…if the results are presented in the form of a brief, structured (<500 words) abstract or table,” and had noted that “[r]esearchers should be aware that editors may consider more detailed deposition of trial results in publicly available registries to be prior publication” [13].

How will journals adapt such positions in the wake of FDAAA, and in light of the need for broader international consensus as articulated by PROCTOR? In one scenario, business interests might drive a journal to constrain data disclosure by limiting consideration to authors whose archived results adhere to the narrowest interpretation of the law. Concerns about loss of profit from reprint sales, advertising, or subscription income, or a publisher's desire to control information that might become part of a lucrative proprietary database may come into consideration. But such motivations would be difficult to justify on principle. The withholding of biomedical research results serves neither the public interest nor the advancement of science.

Traditionally, journal editors have assumed substantial responsibility for, and taken a lead in defining, the quality of research reports that do reach the public. The ICMJE's pivotal role in requiring clinical trials registration provides an example of this responsibility, as do journals' rules for disclosure of competing interests by authors and reviewers. Beyond financial concerns, it is therefore appropriate that editors consider the effects that the availability of results outside traditional publication might have. As Zarin and Tse have pointed out [14], FDAAA promotes transparency by outlawing concealment of a trial's existence or results, but does not directly address problems arising from flawed study design, failure to adhere to ethical principles, presentation of fraudulent data, or misrepresentation of actual results. These matters of research quality and interpretation routinely fall to editors and peer reviewers to identify and, when possible, to correct.

Under FDAAA, will the initial reporting instead become the sole province of those with the greatest financial or personal interest in a favorable result, without the benefit of dispassionate evaluation? Will the public find itself beset by press coverage of post-hoc subgroup analyses, overgeneralizations of results, or improper statistical treatments, slickly packaged as medical breakthroughs? Or will immediate and universal access via the Internet to an ever-increasing number of health-savvy readers provide a better level of scrutiny? (These savvy readers include experienced care providers, patient and professional organizations, consumer advocates, specialty bloggers, health reporters, and entire fields of researchers—not just the few selected to perform formal peer review of a given trial.) Given that trials results must now be released irrespective of “formal” publication in a journal, it surely makes sense to ensure that the public dataset for every trial contains sufficient information to permit objective evaluation of the trial's findings for each prespecified study outcome.

In the best case, unfettered access by these parties would provide radical improvements over the current system, in which limited access to data hampers systematic review and abets disingenuous drug marketing. It's not difficult to imagine a vigorous network of skilled evaluators serving as watchdogs over posted data that have been misrepresented or remain unpublished. Perhaps peer-reviewed journals will provide a forum for publishing independent analyses of such datasets. The appearance of such articles would mark a full circle affirming the contribution of formal peer review, but would also demonstrate the value of openly available results.

Constraining prepublication data to protect the public interest seems wholly untenable given the likely benefits of entrusting these data to an informed public. But will individuals with the requisite abilities assume the necessary responsibilities? Will universities, the news media, or the FDA itself see critical evaluation of public data as an effort worthy of professional or financial reward, or will the task of monitoring data quality and interpretation fall entirely to volunteers? The details of how to maximize public benefit while minimizing abuse require careful consideration.

With FDAAA the train leaves the station, perhaps before many have found their seats. We think it's going in the right direction, but would expect some jostling. As we encourage colleagues at other journals to fulfill the promise of trials registration and to support authors who would publicly share the results of their research—even where the law doesn't require it—we also recognize that the debate on how best to publicize these results is still in its early stages. We therefore support PROCTOR's emphasis on global involvement in realizing the long-term benefits of this new opportunity.