| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2006;37:2963.)
© 2006 American Heart Association, Inc.
Original Contributions |
From Institute for Clinical Research and Health Policy Studies and Department of Medicine (D.M.K., H.P.S., R.R.), Tufts-New England Medical Center, Tufts University School of Medicine, Boston, Mass; Boehringer Ingelheim (E.B.), Ingelheim, Germany; Department of Neurology (W.H.), University of Heidelberg, Heidelberg, Germany.
Correspondence to David M. Kent, MD, MS, Institute for Clinical Research and Health Policy Studies, Tufts-New England Medical Center, 750 Washington St, #63, Boston, MA 02111. E-mail Dkent1{at}tufts-nemc.org
| Abstract |
|---|
|
|
|---|
Methods We explored outcomes in patients from 5 major randomized clinical trials testing intravenous recombinant tissue plasminogen activator (rt-PA) classified by the Stroke-TPI as "treatment-favorable" or "treatment-unfavorable." We used iterative bootstrap re-sampling to estimate how such a model would perform on independent test data.
Results Among patients treated within the 3- to 6-hour window, 67% of patients were classified by Stroke-TPI predicted outcomes as "treatment-favorable" and 33% were classified as "treatment-unfavorable." Outcomes in the treatment-favorable group demonstrated benefit for thrombolysis (modified Rankin Scale score
1: 44.0% with rt-PA versus 34.2 with placebo, P=0.005), whereas harm was demonstrated in the treatment-unfavorable group (modified Rankin Scale score
1: 31.3% with rt-PA versus 38.3% with placebo; P=0.004). Bootstrap resampling with complete cross-validation showed that the absolute margin of benefit in the treatment-favorable group diminished on average by 36% between derivation and independent validation sets, but still represented a significant tripling of improvement in benefit compared with conventional inclusion criteria (5.2% [interquartile range, 1.7% to 8.6%] versus 1.8% [interquartile range, 0.5 to 4.1], P<0.0001).
Conclusions Such multivariable risk-benefit profiling may be useful in the selection of acute stroke patients for rt-PA therapy even more than 3 hours after symptom onset. Prospective testing is indicated.
Key Words: acute care acute Rx acute stroke emergency medicine outcomes risk factors stroke management thrombolysis thrombolytic Rx treatment predictive modeling
| Introduction |
|---|
|
|
|---|
Previously, we found evidence suggesting the absence of benefit in these trials with patients enrolled beyond 3 hours from symptom onset may obscure treatment-favorable and treatment-unfavorable patients identifiable on the basis of pretreatment clinical characteristics.9 We recently developed a predictive instrument for thrombolysis in stroke (the Stroke-Thrombolytic Predictive Instrument [Stroke-TPI]), which predicts outcomes with and without thrombolysis.10 This studys purpose was to test whether such a prognostic instrument might be useful for the selection of patients with a favorable risk-benefit profile for IV rt-PA when treated after 3 hours of stroke onset.
| Methods |
|---|
|
|
|---|
Because the Stroke-TPI was developed on the combined database used for this analysis, these results are likely to be overly optimistic and might represent the upper limit of model performance. Therefore, we investigated how such a model would perform on independent data. Figure 1 summarizes our approach. First, the combined database was divided into derivation and test subsets. To select the derivation subset, we randomly sampled patients from the combined database with replacement (ie, bootstrap sampling), to reconstitute a derivation database that was the same size as the overall combined database ("0.632 bootstrapping").11 Because patients are sampled with replacement, some patients are drawn repeatedly and some (
37% on average) are excluded. The excluded portion forms the independent test dataset. Stepwise logistic regression was used on the derivation dataset to develop a model predicting the probability of a normal or near-normal outcome (modified Rankin Scale [mRS] score
1), with and without rt-PA, as with the development of the Stroke-TPI.10 The model was then applied to those patients treated after 3 hours in the independent test data, to classify them as "treatment-favorable" or "treatment-unfavorable," based on model predictions. We then examined the actual outcomes in these patients, hypothesizing that "treatment-unfavorable" patients would do worse with rt-PA than with placebo, whereas "treatment-favorable" patients would benefit from rt-PA.
|
Because the 3- to 6-hour subset of the test dataset is not large enough to make reliable treatment-effect estimates in each of the favorable/unfavorable subgroups, and because any given sample might be nonrepresentative, we iterated the bootstrap sampling and modeling procedures 250 times. Each iteration produced a different derivation dataset, a different model (with different variables and coefficients) and a different test dataset. Through this iterative bootstrap sampling, modeling, and testing, we arrived at a stable estimate of the decrement in the degree of benefit in the test dataset compared with the derivation dataset, and also of the "gain" in the degree of benefit expected with the use of such a multivariable model in independent data compared with the conventional selection criteria.
Outcomes
Our primary outcome was the degree of benefit among model-selected "treatment-favorable" patients in the test set across samples, compared with that seen overall in the 3- to 6-hour sample from which the subgroup was drawn, using the mRS
1 outcome. In addition, as a secondary outcome, we examined the proportion of patients who were dead or severely disabled (mRS
5), by treat ment-assignment, in the treatment-favorable and treatment-unfavorable groups.
| Results |
|---|
|
|
|---|
1: 44.0% with rt-PA versus 34.2 with placebo; P<0.005). Outcomes in the 33% classified as treatment-unfavorable showed harm with rt-PA (mRS
1=31.3% versus 38.3% with placebo; P<0.004). The interaction for the treatment-effect between the groups was highly significant (P<0.0001).
Characteristics of High-Benefit Versus Low-Benefit Patients
Table 1 shows the characteristics of patients classified as treatment-favorable versus treatment unfavorable, according to the Stroke-TPI. There was significant overlap in the value of individual variables that modify rt-PA effect, indicating that a combination of factors needs to be examined simultaneously, as by a multivariable model. The outcomes shown in Table 1 indicate that, though the model was developed to predict a normal or near-normal outcome (mRS
1), patients likely to benefit on this outcome also benefitted from thrombolysis across other measures.
|
Iterative Bootstrap Resampling, Modeling and Testing
Across the 250 sampling iterations, each derivation dataset had 2131 patients; each test dataset had on average 784 patients. Table 2 shows the variables tested and the proportion of the 250 models into which they were selected.
|
When the models were applied directly to the dataset on which they were developed (Figure 2), on average, 33% (interquartile range [IQR], 27% to 39%) of patients were classified as "treatment-unfavorable," and 67% (IQR, 61% to 72%) were classified as "treatment-favorable." As predicted, patients in the treatment-unfavorable group, on average, had worse outcomes with rt-PA than with placebo, by an absolute margin of 10.2% (IQR, 6.9% to 14.2%), whereas patients in the treatment-favorable group, on average, had better outcomes with rt-PA than with placebo, by an absolute margin of 8.1% (IQR, 5.8% to 10.7%). When the models are applied to the test datasets (Figure 2), on average, 33% (IQR, 26% to 40%) of patients were classified as "treatment-unfavorable" and 67% (IQR, 61% to 73%) of patients were classified as "treatment-favorable." Within the treatment-unfavorable group, again, outcomes were worse with rt-PA treatment than with placebo, whereas in the treatment-favorable group, on average, outcomes were better with rt-PA therapy. As expected, there was some "shrinkage" of the absolute margin benefit in the "treatment-favorable" group on the test datasets compared with the derivation datasets, but 64% of the margin of benefit was maintained, with an average absolute margin of 5.2% on average (IQR, 1.7% to 8.6%). This compares favorably to the overall 1.8% (IQR, 0.5 to 4.1) average treatment benefit in the 3- to 6-hour window in the test datasets from which these patients were selected (P<0.0001 for the difference in benefit between these 2 samples in the test datasets). If the benefit found using the Stroke-TPI has a similar degree of "shrinkage" when tested on independent data, than the margin of absolute benefit in the treatment-favorable subgroup (9.8% in the combined database) would be expected to be 6.3%.
|
Death or Severe Disability
We also examined the likelihood of severe disability or death (mRS
5) in patients treated in the 3- to 6-hour period, overall, and in the treatment-favorable and treatment-unfavorable subgroups. These results, shown in Figure 3, were virtually identical in the derivation and test sets, suggesting good generalizability, and suggesting that the model might be especially useful for excluding patients who would be harmed by rt-PA.
|
| Discussion |
|---|
|
|
|---|
In developing the Stroke-TPI, we identified 4 variables that diminish the benefits of rt-PA: longer symptom onset to treatment time, male sex, higher systolic blood pressure, and previous stroke. The Stroke-TPIs ability to select treatment-favorable patients in an independent population depends on whether these associations with treatment effect arose by chance in this database, or whether they are true biologically based treatment modifiers. Longer symptom onset to treatment time is well-accepted to modify the treatment effects of thrombolytic therapy, based on clinical trials,5,15 as well as biologic plausibility, animal models of reperfusion16 and by analogy with results in acute myocardial infarction (AMI).17 There is ample evidence from independent data that higher systolic blood pressure is associated with an increased risk of thrombolytic-related intracranial hemorrhage in both AMI1820 and stroke.2123 Previous stroke has also been shown in multiple studies to increase the risk of thrombolytic-related intracranial hemorrhage in AMI,18,20 and presumably would diminish the probability of a normal or near-normal outcome even in patients who do not have hemorrhagic complications of thrombolysis. That male sex would be associated with less benefit from thrombolytics was initially unexpected. However, after the report of the sex-by-treatment interaction in this database,24 confirmatory findings from independent data have been published.25,26,27
There are several limitations to our study. First, a much larger database would have allowed for a single independent test data set sufficient for reliable treatment effect estimates in treatment-favorable and treatment-unfavorable subgroups, without the need for bootstrap re-sampling. However, as we used a complete cross-validation method (ie, new variables selected at each iteration, not just new coefficients) and validated on data excluded from model development, our test set results provide an estimate of the degree of over-optimism in the Stroke-TPI performance. The performance of the models built during the automated procedure likely represents an underestimation of the "gain" expected when using the Stroke-TPI on independent data, because the variableoutcome relationships in these models were not individually checked against clinical reasoning, and results of all models, including those of dubious quality, were averaged. In particular, although the Stroke-TPI includes 4 robust treatment effect interactions on which risk-benefit profiling depend, 56 of the 250 bootstrapped models included
2 treatmenteffect interactions, including 23 that did not even include a treatment-by-time interaction. Additionally, ongoing trials (such as ECASS 3) should provide the opportunity for future independent testing.
The model we present was limited also by the variables collected commonly across trials. In particular, serum markers and perfusion-diffusion magnetic resonance imaging have shown some preliminary promise for risk-benefit profiling, but these data were not included in these trials.
Despite these limitations, the approach presented here could be of substantial use. In the context of a clinical trial, improving the margin of benefit from 1.8% to 5.2% would dramatically diminish the sample size required for an adequately powered clinical trial, from an infeasible >20 000 to a more feasible <3000 patients. (With an expected margin of benefit of 6.3%,
1000 patients would be required for each trial arm for 80% statistical power.) Use of such a model could also have an important effect in clinical care. In the ECASS I and II trials,
80% to 85% of patients were treated after 3 hours. Thus, using estimates from these enrollees, treating even two-thirds of this group of patients could more than triple the number of patients eligible for rt-PA (ie, from 20% of ECASS enrollees to >70%), although other studies suggest that the number of patients otherwise eligible for treatment in this time window may be less.28
In summary, we have demonstrated that there is a great deal of heterogeneity in the treatment effect of rt-PA for acute stroke, and that the Stroke-TPI may be effective in identifying those patients most likely to benefit from treatment in the 3- to 6-hour window. This approach requires testing in a prospective trial.
| Acknowledgments |
|---|
Sources of Funding
This research was supported by grants from the NINDS to Dr Kent (K23NS44929 and R21NS48225).
Disclosures
E.B. is an employee at Boehringer Ingelheim, which has a financial interest in this study. W.H. has received honoraria for speaking at major symposia for Boehringer Ingelheim, and for participating in DSMB and a steering committee for Boehringer Ingelheim. W.H. is also on an advisory board for Boehringer Ingelheim. All other authors report no conflicts of interest.
Received May 19, 2006; accepted June 21, 2006.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
M Uyttenboogaart, R E Stewart, P C Vroomen, G-J Luijckx, and J De Keyser Utility of the stroke-thrombolytic predictive instrument J. Neurol. Neurosurg. Psychiatry, September 1, 2008; 79(9): 1079 - 1081. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. B. Milbrandt, A. Ishizaka, and D. C. Angus Update in Critical Care 2006 Am. J. Respir. Crit. Care Med., April 1, 2007; 175(7): 638 - 648. [Full Text] [PDF] |
||||
![]() |
D. M. Kent, H. P. Selker, R. Ruthazer, E. Bluhmki, and W. Hacke The Stroke-Thrombolytic Predictive Instrument: A Predictive Instrument for Intravenous Thrombolysis in Acute Ischemic Stroke Stroke, December 1, 2006; 37(12): 2957 - 2962. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2006 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |