Dichotomized Efficacy End Points and Global End-Point Analysis Applied to the ECASS Intention-to-Treat Data Set
Post Hoc Analysis of ECASS I
Background and Purpose—It is not yet known which end points are the most suitable for evaluation of the effects of acute stroke intervention. The European Cooperative Acute Stroke Study (ECASS) I study used 2 primary end points. The study was powered to detect a 15% improvement of the median of each primary end point. The study failed to show this effect and was negative in the intention-to-treat analysis. The National Institute of Neurological Disorders and Stroke (NINDS) study used 4 dichotomized end points and applied a global end-point analysis. This study was positive and led to FDA approval of thrombolytic therapy for acute ischemic stroke. This study was undertaken to answer the question of whether a different statistical design may have shown a positive results of the ECASS I trial.
Methods—We performed a retrospective analysis of the ECASS I intention-to-treat data set (615 randomized and treated patients, rtPA treatment versus placebo) and post hoc application of the NINDS trial statistical methodology (global end-point analysis). The scores of the modified Rankin Scale (mRS), Barthel Index (BI), and the National Institutes of Health Stroke Scale (NIHSS) were dichotomized according to the criteria used in the NINDS trial. Favorable outcome was defined as a score of 0 or 1 on mRS, a score of 95 or 100 on BI, and a score of 0 or 1 on NIHSS.
Results—The number of patients reaching favorable outcome were higher in all 3 end points in the rtPA-treated group. The effect sizes were 8% for mRS, 6% for BI, and 14% for NIHSS, respectively. The differences are statistically significant for the mRS (P=0.044; odds ratio [OR], 1.4; 95% confidence interval [CI], 1.0 to 2.0) and the NIHSS (P=0.001; OR, 1.9; 95% CI, 1.4 to 2.8), while for the BI significance was missed (P=0.102; OR, 1.3; 95% CI, 0.9 to 1.8). The global end-point statistics, however, shows a significant increase (P=0.008; OR, 1.5; 95% CI, 1.1 to 2.0) of favorable outcome in the rtPA-treated patient group.
Conclusions—Using the global end-point analysis, ECASS is positive in the intention-to-treat analysis. This may indicate that the time window for thrombolysis may be as long as 6 hours. Looking at the 3 dichotomized end points, the effect sizes for 2 end points, mRS and BI, are smaller in the ECASS 6-hour intention-to-treat population compared with the NINDS trial, whereas the effect size for the NIHSS is larger. While in the NINDS trial all 3 end points reveal statistically significant results, in ECASS only 2 of the 3 corresponding end points, mRS and NIHSS, were statistically significant. This finding underlines an important difference of a global end-point approach: it may show a positive overall result although one of the end points is not positive.
In the fall of 1995, the results of 2 large, placebo-controlled trials1 2 testing the efficacy and safety of intravenous recombinant tissue plasminogen activator (rtPA) in acute ischemic stroke were published. The European Cooperative Acute Stroke Study (ECASS) allowed a 6-hour time window and used 1.1 mg/kg rtPA, whereas the National Institute of Neurological Disorders and Stroke (NINDS) study used 90- and 180-minute time windows to test a lower dose (0.9 mg/kg) of rtPA. One prominent feature of ECASS was the introduction of subtle, predefined CT exclusion criteria, which among others were used to identify the so-called target population.
Regarding the original predefined statistical plan of ECASS,3 the intention-to-treat (ITT) analysis did not show a statistically significant benefit for the rtPA-treated patients in the 2 primary end points, the median of the modified Rankin Scale (mRS) score and the median of the Barthel Index (BI) score. The secondary end points, the combination of BI and mRS, early neurological recovery as assessed by the Scandinavian Stroke Scale (SSS), and duration of in-hospital stay were significantly different in favor of the rtPA-treated patients. In the target population analysis, however, one primary end point (mRS) and all secondary end points except for mortality were statistically significant in favor of the rtPA-treated patients. The 90-day mortality in ECASS was significantly higher among the rtPA-treated patients, presumably in part due to an unexpectedly low mortality among the placebo patients (16%).
In contrast, the NINDS trial showed a statistically significant result in favor of rtPA for the global end-point analysis4 of 4 dichotomized end points—the National Institutes of Health Stroke Scale (NIHSS), mRS, BI, and Glasgow Outcome Scale (GOS) scores—and had a lower 90-day mortality in the rtPA-treated group, with a mortality in the placebo group of 21%. Furthermore, each of the 4 individual end points, if analyzed separately, showed a statistically significant result in favor of the rtPA-treated patients. Consequently, the results of the NINDS trial led to approval of rtPA by the Food and Drug Administration (FDA) in the United States for the treatment of acute ischemic stroke patients within a time window of 3 hours.
Considering that the ECASS and NINDS trials have recruited patients of similar stroke severity, we hypothesized that the divergent results might, at least in part, be related to the different statistical methods applied. We therefore analyzed the ECASS ITT data (6-hour time window) with the same statistical approach used in the NINDS trial. We describe the results for the 3 dichotomized end points mRS, BI, and NIHSS. The GOS was not used in ECASS.
Subjects and Methods
As in the NINDS study, the efficacy end points mRS, BI, and NIHSS were dichotomized, and favorable outcome was defined by the following scores: 0 or 1 on mRS, 95 or 100 on BI, and 0 or 1 on NIHSS. GOS was not used in ECASS.
Imputations for missing values were done as follows: Patients who died before the 3-month assessment were given the worst possible scores for all outcomes. In the case of surviving patients missed for follow-up, the “last observation carried forward” procedure was applied.
The global hypothesis was tested simultaneously with a Wald-type global test statistic derived from a general linear model with logit function, computed with the use of a generalized estimating equations.4
The individual end points were tested by Fisher’s exact test for significant differences if the global test statistics were significant. A 2-tailed probability value of <0.05 was considered significant. In addition, odds ratios (ORs) and 95% confidence intervals (CIs) are given.
Table 1⇓ gives a representation of favorable versus unfavorable outcome for the 3 end points in ECASS of both the placebo- and the rtPA-treated patients. The numbers of patients reaching favorable outcome in each of the 3 end points were always higher in the rtPA-treated group. The global end-point statistics shows a significant increase (P=0.008; OR, 1.5; CI, 1.1 to 2.0) of favorable outcome in the rtPA-treated patient group. The effect sizes, defined as the absolute percentage differences between rtPA and placebo, are 8% (mRS), 6% (BI), and 14% (NIHSS), respectively. The differences are statistically significant for the mRS (P=0.044; OR, 1.4; CI, 1.0 to 2.0) and the NIHSS (P=0.001; OR, 1.9; CI, 1.4 to 2.8). For the BI, significance was missed (P=0.102; OR, 1.3; CI, 0.9 to 1.8).
Fig 1⇓ allows direct comparison of the ECASS I end points and the corresponding NINDS study end points.
ECASS, the first large-scale thrombolysis trial in acute ischemic stroke, is considered a negative trial. The hypotheses were that rtPA treatment leads to an improvement in the median of the BI score by 15 points (15%) or to an improvement in the median of the mRS score by 1 point (14%).3 In ECASS we failed to show statistically significant differences in the ITT analysis for the 2 primary end points.1 Several reasons for this failure, such as the inclusion of patients with major CT protocol violations and the unexpectedly low mortality in the placebo group, have already been discussed. The importance of early infarct signs and other major protocol violations, for example, is underlined by the fact that in the predefined target population (patients treated according protocol) the expected difference in the mRS was found (P=0.035).
Based on part I of the NINDS trial, which looked at very early recovery defined by a reduction of >4 points or a score of 0 on the NIHSS, the NINDS investigators decided to continue with part II of this trial, in which they looked at 4 dichotomized end points: mRS, BI, NIHSS, and GOS. The results showed a statistically significant difference in favor of the rtPA-treated patients in all of the 4 end points individually as well as in the global end-point analysis.
Since ECASS assessed prospectively 3 of the 4 NINDS trial end points, it was feasible to apply the global end point analysis post hoc, although not predefined in the statistical section of the ECASS protocol. Reanalyzing the ECASS ITT data using this statistical methodology reveals that ECASS becomes positive in the ITT analysis. This may indicate that the time window for thrombolysis may be as long as 6 hours. Nevertheless, including patients with major early infarct signs as defined by the ECASS protocol still remains unsafe, even if the results for the 3 dichotomized end points are in favor of rtPA.
Furthermore, looking at the 3 end points individually, the effects are comparable between the NINDS study and ECASS. The effect sizes, defined as the differences (absolute percents) between rtPA and placebo, are for the RS 8% (ECASS) versus 13% (NINDS trial), for the BI 6% (ECASS) versus 12% (NINDS trial), and for the NIHSS 14% (ECASS) versus 11% (NINDS trial). While in the NINDS trial all 3 end points revealed statistically significant results, only 2 of the 3 end points in ECASS are positive. Significance is missed for the BI. This finding underlines the high sensitivity of the global end-point approach: it may show a positive overall result although one of the end points is not positive.
By applying the global end-point statistics to the ECASS data, it becomes clear that the NINDS study methodology is a very useful one. The analysis also shows the importance of the a priori definition of end points and the corresponding statistical strategy-. Retrospectively, the choice of the medians of the mRS and the BI in ECASS as primary end points was not optimal and prevented the trial from being positive in the ITT analysis.
However, looking at patient outcome in a dichotomized fashion does not overcome the methodological problems entirely.
However, a global end-point statistic should also be viewed with a word of caution, as it counts some redundant measures in a multiple way and does not separate the different dimensions in the assessment. It might therefore be of importance for future evaluations to consider in parallel a factorial design to identify the principal component accounting for the majority of the observed variance, followed by an unidimensional analysis helping to interpret the multidimensional result. One important shortcoming of this dichotomized analysis is that unfavorable outcome may be underrepresented because a differentiation of the remaining outcome categories is missing. While mortality is a separate part of the safety assessment, important information concerning different degrees of dependence and handicap after ischemic stroke may be hidden.
Nevertheless, the NINDS trial methodology proved to be a robust one and seems to be useful for stroke trials. We recently presented in abstract form5 an even more complex global end point, where in addition to the mRS, BI, and NIHSS, we also assessed the SSS, infarct size, and duration of in-hospital stay.
In summary, when applying the statistical approach of the NINDS study to the ECASS ITT data set, the outcome of rtPA-treated patients is significantly improved and therefore ECASS is a positive trial. Unfortunately, this is only a post hoc, nonpredefined analysis, but it supports the hypothesis that the time window for thrombolysis maybe longer than 3 hours.
Dr Hacke was chairman of the steering committee of ECASS I and is currently chairman of the steering committee of ECASS II. He is reimbursed for his time devoted to the studies. Dr Bluhmki is employed by Boehringer Ingelheim, the main sponsor of the trial. Dr Meier is employed by Boehringer Ingelheim and is a member of the steering committee of ECASS II.
- Received March 3, 1998.
- Revision received June 11, 1998.
- Accepted June 11, 1998.
- Copyright © 1998 by American Heart Association
Boysen G, Hacke W. European Cooperative Stroke Study (ECASS): study design and progress report. Eur J Neurol. 1995;1:213–219.
Tilley BC, Marler J, Geller NL, Lu M, Legler J, Brott T, Lyden P, Grotta J, for the National Institute of Neurological Disorders and Stroke (NINDS) rt-PA Stroke Trial Study Group. Use of a global test for multiple outcomes in stroke trials with application to the National Institute of Neurological Disorders and Stroke rt-PA Stroke Trial. Stroke. 1996;27:2136–2142.
Steiner T, Höxter G, Bluhmki E, Hacke W. Multidimensional assessment of stroke outcome. Cerebrovasc Dis. 1996;6:54. Abstract.