Wisdom and Determination in the Ongoing Pursuit of the Ever-Elusive Neuroprotective Agent
See related article, pages 1621–1625
As noted by Hill et al,1 the need for neuroprotective agents has never been greater. However, few would describe the trek to discover an efficacious neuroprotective agent as anything other than a frustrating and expensive journey. But as suggested in a quote attributed to Charles Noble, “You must have long-range goals to keep you from being frustrated by short-range failures.” Hill et al not only have a clear focus and determination to achieve their long-term goal but also have been both clever and responsible in their approach to achieve this goal.
Unfortunately, early deaths in the albumin-treated arm of what at that time was the Albumin in Acute Stroke (ALIAS) trial (subsequently called ALIAS 1) led the Data and Safety Monitoring Board (DSMB) to terminate the study for safety reasons.2 With the time it takes to design and mount such a study, coupled with the effort to select and initiate centers, it is natural for investigators to become highly vested in their trials. One can only imagine the agitation and emotion of the investigators after the DSMB announcement of study termination. The ALIAS investigators and their DSMB are to be commended for remaining calm and focused during this period, particularly for their great wisdom for the investigators to remain blinded to efficacy outcome in what subsequently would become ALIAS 1. This separation of safety and efficacy outcomes allowed ALIAS 2 to be designed and mounted without being biased by awareness of the efficacy outcomes by the study investigators. Congratulations for being clever and thoughtful!
The ALIAS investigators also have been responsible by reporting efficacy differences in ALIAS 1 in this issue of Stroke, an analysis that was importantly only conducted after the design of ALIAS 2 was approved and the study was initiated. It is assuring that there is evidence of an efficacy signal among the subgroup of ALIAS 1 participants defined by the eligibility criteria used in ALIAS 2. In essence, ALIAS 1 has become the ultimate pilot study for ALIAS 2 and, with the failure of so many potential neuroprotectant agents, it would be irresponsible to continue with the fiscal commitment required to complete ALIAS 2 without a signal in the “pilot” ALIAS 1. As such, the article in this issue of Stroke represents the fulfillment of the responsibility to ensure the good use of the National Institutes of Health funding underwriting the ALIAS 2.
Whereas these events have been carefully orchestrated to protect the validity of ALIAS 2, it will be important for investigators to address at least 3 issues raised by these unusual maneuvers. First, the operation of most DSMB involves the ongoing monitoring of safety but preplanned assessments of efficacy at fixed times. ALIAS 1 was designed with 3 such assessments of efficacy. Apparently, the timing of the recommendation by the DSMB to stop for safety concerns was during the meeting that coincided with the first of these preplanned efficacy analyses.2 Biases in the magnitude of treatment effects reported by Hill et al1 would be introduced if the unblinded DSMB members (who were at a meeting to review efficacy data), rather than the blinded ALIAS investigators, recommended the “restart” of the study. If a promising (but nonsignificant) efficacy signal potentially seen by the unblinded DSMB was the source of the recommendation to “restart” the study, then the positive efficacy outcome at this particular point in the conduct of ALIAS 1 is directly a product of a selective efficacy assessment. That is, if the DSMB was the source of the recommendation to “restart” the study, then this recommendation would have been less likely if the positive efficacy signal was not seen. This concern is heightened by the observation that the timing of the recommendation to stop the study for safety concerns was at the time of the first interim analysis, when the smaller sample size would permit a greater measurement error in the assessment of the efficacy, making it easier to have a larger spurious positive efficacy treatment effect. Importantly, this concern does not affect the validity of ALIAS 2 (which stands on its own as a frequentist trial) but could potentially result in a falsely positive assessment of efficacy in ALIAS 1 reported by Hill et al and would also result in inappropriately optimistic power calculations for ALIAS 2. It will be important for the (still-blinded) ALIAS investigators to assure readers that they are directing this process rather than the (unblinded) DSMB.
Second, an unfortunate aspect of many stroke trials is the confounding of safety and efficacy outcomes. Subject to this are the ALIAS studies, in which safety concerns leading to the stopping of ALIAS 1 appear to include both death and neurological worsening, but the primary outcome is the modified Rankin scale (mRS), in which death is scored as 6 and neurological worsening is likely closely associated with other high mRS scores (likely 5). The ALIAS investigators implemented modified eligibility criteria in ALIAS 2 (and in the report by Hill et al1) to produce a “safety population” in which death and neurological worsening (ie, mRS scores of 5 or 6) were reduced in the actively treated group. That the age criterion was set at the unusual threshold of 83 years (rather than a more traditional 80 or 85 years) suggests that considerable latitude was provided in this decision. Safety and efficacy become confounded to the extent that the definition of eligibility criteria defined to reduce mRS score of outcomes of 5 or 6 (ie, do not have a safety concern) may make them more likely to have mRS score of 0 or 1 (ie, have a positive efficacy outcome). It will be important for the ALIAS investigators to acknowledge this confounding of safety and efficacy in the reporting of the ALIAS 2 results.
Finally, the stroke community is clearly in the midst of a spirited discussion (debate?) of whether to dichotomize mRS or to consider analytic approaches that capture treatment differences across the entire distribution of mRS scores. ALIAS 1 data presented herein are a great teaching example of the potential challenges of dichotomization as a favorable outcome (mRS score 0 or 1). Specifically, Hill et al1 are correct to celebrate the 36.3% good outcomes in the albumin-treated group, which on a relative basis is 29% (relative risk=1.29) higher than the 28.2% good outcomes in the saline-treated group. However, the article is silent on the 20.1% death or severely disabled outcomes (mRS score 5 or 6) in the albumin-treated group, which is 20% (relative risk=1.20) higher than the 16.8% death or severely disabled outcomes in the saline-treated group. That estimation of the risk of these poor outcomes is likely biased by the investigation of reducing “safety” events in the determination of the eligibility criteria and only heightens this concern. Even if the findings of ALIAS 2 directly reflect these ALIAS 1 findings, it is not clear that a drug resulting in a 29% increase in good outcomes is desirable considering a 20% increase in outcomes of death and severely disabled states. Hence, it is possible that albumin may make it possible to cure more patients (ie, have “good outcomes”) while not making an average patient better (because it results in a near-equal increase in the worst outcomes). It is unfortunate that the Food and Drug Administration apparently sees the additional effort to interpret outcomes that are not focused on dichotomizing the data as a barrier to approve designs focusing on differences in the distribution of outcomes, a position that fails to place any value on the primary assessment of a drug burden of higher numbers of patient deaths and severely disabled states. Although the ALIAS investigators must give first priority to their a priori primary outcome, the distribution of outcomes in the report by Hill et al1 underscores that the decision to incorporate albumin into stroke care must be made regarding the entire distribution of outcomes.
In conclusion, Hill et al1 should be congratulated for having a strong focus on the long-term goal of neuroprotective drug development, and also for not being frustrated by the short-term safety failure of ALIAS 1. We consider these 3 issues to be minor in comparison to the success and nimbleness of these investigators and their efforts to press forward in the investigation of a promising compound. We close where we began, by applauding the cleverness and responsibility of the investigators, and we await the results of ALIAS 2 with great hopes!
The opinions expressed in this editorial are not necessarily those of the editors or of the American Heart Association.
- © 2011 American Heart Association, Inc.
- Hill MD,
- Martin RH,
- Palesch YY,
- Tamariz D,
- Waldman B,
- Ryckborst KJ,
- et al
- Ginsberg MD,
- Palesch YY,
- Martin RH,
- Hill MD,
- Moy CS,
- Waldman BD,
- et al