NeuroThera Effectiveness and Safety Trial 3
How Do We Align Corporate and Scientific Integrity to Complete and Report Pharma-Sponsored Trials Properly?
See related article, p 3187.
In this issue of Stroke, Hacke et al1 for the NeuroThera Effectiveness and Safety Trial 3 (NEST 3) Committees and Investigators report the results of a phase III, randomized, double-blind clinical trial of transcranial laser therapy for acute ischemic stroke. This was a device trial sponsored and funded by industry via venture capital specifically to PhotoThera, Inc, a privately held medical device company. Once the independent Data Monitoring Committee halted the trial for futility, a little more than halfway through recruitment, a drama full of twists and turns played out like a John Grisham novel.
The authors of this editorial received further information about the history of the NEST trials to help frame a perspective (Dr Dilly, personal email communication through the Stroke editorial office to S.R. Levine, MD, and M.D. Hill, MD, unpublished data, July 28, 2014):
NEST 3 was the third clinical study of transcranial laser therapy for stroke conducted by PhotoThera. NEST 1, a phase II study, gave highly encouraging results including a statistically significant improvement in functional outcome 90 days after stroke. PhotoThera raised money from a syndicate of venture capital investors to conduct NEST 2, intended to be the definitive registration trial for transcranial laser therapy. NEST 2 enrolled 660 patients at a cost of ≈$50 million and failed to meet its primary end point. However, further analysis of the trial strongly suggested that a subgroup of the study population with more superficially placed strokes and mild-moderate baseline symptom severity had benefitted from treatment.
On the basis of these data, PhotoThera raised an additional ≈$60 million from existing and new investors to fund NEST 3, which was aimed specifically at the subgroup. It is important to understand that $10 million was in the form of venture debt—secured against the intellectual property and assets of PhotoThera. [Venture debt is a type of debt financing for emerging venture-backed companies and is a means of financing startups that are in between more traditional venture capital financing rounds. Sometimes companies run short of capital in between funding milestones. Companies who run short of capital are sometimes good candidates for venture debt.] The $60 million raised was projected to be sufficient to fund the NEST 3 study fully. Unfortunately, recruitment into NEST 3 went significantly slower than planned, and site costs were higher than predicted such that PhotoThera was projected to run out of money before the completion of the study. (We are not aware of when they realized this.)
NEST 3 had passed a prespecified futility analysis when data from 300 patients were available, and an additional analysis was planned for 600 patients. Because of the significant over-run in expenditures, and the imminence of the interim analysis, the investor group decided to wait until after the 600 patient futility analysis to complete funding of the study. Given the strong positive data from NEST 1 and the highly suggestive subgroup data from NEST 2, it was a significant surprise to the investigators, company, and investors when the Data Monitoring Committee declared that the NEST 3 study is futile at 566 completed patients.
Because of the negative outcome of NEST 3, the investor group declined further investment in PhotoThera, and the venture debt principal was recalled. The lenders were unwilling to ascribe significant value to PhotoThera’s intellectual property and decided to recoup what they could by liquidating PhotoThera’s assets. The immediate effect was that all PhotoThera employees and consultants were terminated without compensation, all external payments were frozen, and PhotoThera’s assets were sold.
Under these extreme circumstances, the PhotoThera investor group negotiated with the lender to be allowed to put an additional $3 million into the company specifically for the purpose of winding down NEST 3 in compliance with Good Clinical Practice and regulatory requirements and paying as many outstanding invoices as possible. A small group of PhotoThera employees worked without pay to do the best we could to close down the study with the cooperation of the sites and at least allow the data to be published. Although it was not possible to do a perfect job—for instance, we could not afford to retrieve and destroy study instruments so they had to be disabled and left on site—we did manage the process sufficiently well that patient safety was ensured and a reliable database was built.
This is a story in which nobody wins; patients did not benefit from a promising new treatment; investigators and study site organizations were left with significant invoices unpaid; the investors in PhotoThera lost >$100 million; and the PhotoThera staff all lost their jobs without compensation. However, despite these difficult circumstances, the extraordinary efforts of participating investigators, the NEST 3 Steering Committee and Data Monitoring Committee (particularly Drs Hacke and Lees), and the great generosity of Parexel in providing the study database for analysis free of charge meant that a high-quality publication has been generated and an important scientific question on the use of transcranial laser therapy treatment in ischemic stroke has been answered.
The lessons learned from the aftermath of NEST 3 should serve as a catalyst for the development of strategies to avoid similar outcomes in future trials.
There were several international Health Authorities overseeing this trial: The Food and Drug Administration (United States), Federal Office for Safety in Health Care (Austria), Health Canada, National Supervisory Authority for Welfare and Health (Finland), Afssaps—Agence Française de Sécurité Sanitaire des Produits de Santé (Saint-Denis, France), German Institute of Medical Documentation and Information (Germany), Instituto Nacional de Salud (Pera), Medical Products Agency (Sweden), Agencia Española de Medicamentos y Productos Sanitarios (Spain), and Swissmedic (Switzerland). What role did they have, collectively and individually, in guaranteeing compliance with ethical and financial obligations during the planning phase and enrollment, leading up to the futility analyses?
Food and Drug Administration oversight includes2 processes critical to protecting human subjects, maintaining the integrity of study data including monitoring and auditing and compliance with applicable laws and regulations. Internationally, most regulatory bodies either fully subscribe to Good Clinical Practice-International Conference on Harmonisation guidelines for the conduct of clinical research in humans or have policies and guidance, which is similar. Good Clinical Practice guidelines follow the Declaration of Helsinki,3 which explicitly state, “At the conclusion of the study, patients entered into the study are entitled to be informed about the outcome of the study…” Declaration of Helsinki—article 36 states, “Researchers, authors, sponsors, editors and publishers all have ethical obligations with regard to the publication and dissemination of the results of research. Researchers have a duty to make publicly available the results of their research on human subjects and are accountable for the completeness and accuracy of their reports. All parties should adhere to accepted guidelines for ethical reporting. Negative and inconclusive, as well as positive results must be published or otherwise made publicly available.” Thus, both the regulatory authorities and the investigators hold a responsibility to report the data properly and in full. For NEST 3, it was only because of the exceptional sense of duty from the investigators and key employees (who were unpaid) that we have any data to discuss at all. The data were not reported as planned because of a lack of statistical resource. How can we ensure that this situation, which is not unique to stroke trials, does not arise in the future?
We propose that there should be regulatory change. A rigorous, accountable and predefined plan for study close-out in the event of early stopping and at the end of a trial is required. Funds should be escrowed to manage this responsibility. A National Institutes of Health–funded trial stopped abruptly because of futility would have appropriate close-out with the remaining grant funds. It should be a legally binding commitment from the sponsor of the study with relevant and reasonable time limits for reporting. Indeed, the academic community, through public-funding bodies and major medical journals, now demands that clinical trials be registered. Registration, as was done for NEST 3 (NCT01120301),4 implies a commitment to reporting, but accountability for reporting is not enforced. The academic community can and should demand a higher standard.
Conducting clinical trials cost tens of billions of dollars, and pharma companies may spend $100 to $800 million per drug candidate, growing at a rate of 4.6% per year.2 This is an extremely expensive investment. Approximately 15% to 20% of trials never actually enroll a single patient and more than two thirds of trial sites fail to meet their projected enrollment goals for a given study.2 Early stopping occurs in as many as 20% to 30% of trials.5 This results in delays that increase study costs frequently leading to incomplete or even abandoned trials. Presently, selective reporting of clinical trial results is a well-known major issue. Extensive research demonstrates that between 25% and 50% of clinical trial results remain unpublished several years after completion.2 Fifteen years ago at the 24th American Heart Association International Joint Conference on Stroke and Cerebral Circulation (the current American Heart Association/American Stroke Association International Stroke Conference) an investigator of a pharma-funded clinical trial that was peer-reviewed for a podium presentation, announced that he was blocked from presenting the results of the study6 by the sponsor. This led to an editorial also calling for reform of unethical behavior.7
We recommend that the Stroke Treatment Academic Industry Roundtable8,9 visit these issues on a regular basis and collaboratively make new recommendations on the nature of sponsors’ (and their sponsors/funding sources) obligations (both moral/ethical and financial requirements) for supporting clinical trials. We think that these can be incorporated into each study’s contracts with investigators for legally binding responsibilities to ensure proper completion of a trial. Without updated policies that promote transparency in clinical research, the for-profit sector will continue to respond primarily to financial motives at the expense of the integrity of medical science.10 As Lexchin has argued, what may be necessary is to separate the financing of clinical trials from their conduct.11 The solution is conceptually simple but politically unattractive and will require regulation. Complicating this is the fact that pharmaceuticals/health products are the number 1 lobbyist of the US government (>$226 million spent in 2013)12 and funds a good part of the Food and Drug Administration, a classic Grisham twist!
From a social perspective, medical evidence must stand on a stronger statistical base than in business. The randomized clinical trial is one of the highest standards of evidence not typically seen in economics/business—which necessarily rely on retrospective cohort study level data, but nevertheless demands a high level of financial data reporting. The medical community and the sponsors put a great deal of effort into these trials. Devoted patients and their families are cooperative volunteers. Failing to insure mechanisms of high-quality reporting, irrespective of the scientific result, is a failure to finish the job and a breach of an implicit contract with ourselves and our patients. We are calling for policy revision at the Food and Drug Administration and other regulatory bodies, demanding that accountability for registration and reporting be enforced13 and calling on academic colleagues globally, in all disciplines, to exert pressure to ensure that this occurs.
We thank Sarah Zelonis, Steven H. Rudolph, MD, and Adrian Marchidann, MD, for helpful comments and discussions during the writing of this editorial. We also thank Wayne M. Clark, MD, for confirming that the pharma sponsor of the phase III Cervene refused to let him present the trial results.
Dr Levine was a member of the Data Monitoring Committee, NeuroThera Effectiveness and Safety Trial 2 in 2008 and received modest compensation. Dr Hill reports no conflicts.
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
- © 2014 American Heart Association, Inc.
- Hacke W,
- Schellinger PD,
- Albers GW,
- Bornstein NM,
- Dahlof BS,
- Fulton R,
- et al
- Widler BE
- 3.↵WMA Declaration of Helsinki - ethical principles for medical research involving human subjects. http://www.wma.net/en/30publications/10policies/b3/. Accessed July 24, 2014.
- 4.↵NEST 3 Web site. http://clinicaltrials.gov/ct2/show/results/NCT01120301. Accessed July 24,2014.
- Pergolizzi JV
- Clark WM,
- Raps EC,
- Tong DC,
- Kelley RE
- Goldstein LB,
- Brott TG,
- Kothari RU,
- Smith WS
- 8.↵STAIR I: Stroke Treatment Academic Industry Roundtable. Stroke drug development: bridging the gap from animal research to human trials. Stroke. 1999;30:2752–2758.
- 9.↵STAIR VIII: Stroke Treatment Academic Industry Roundtable. Accelerating the Evolution of Stroke Therapy. March 9–10, 2013, Washington, DC. http://www.thestair.org/content/stair-viii. Accessed July 30, 2014.
- Ashcroft R
- 12.↵OpenSecrets.org: Center for Responsive Politics. Influence and Lobbying. Top Industries. https://www.opensecrets.org/lobby/top.php?indexType=i. Accessed July 30, 2014.
- Lewis T,
- Reichman J,
- So A