The Stroke–Thrombolytic Predictive Instrument
A Predictive Instrument for Intravenous Thrombolysis in Acute Ischemic Stroke
Background and Purpose— Many patients with ischemic stroke eligible for recombinant tissue plasminogen activator (rt-PA) are not treated in part because of the risks and benefits perceived by treating physicians. Therefore, we aimed to develop a Stroke-Thrombolytic Predictive Instrument (TPI) to aid physicians considering thrombolysis for stroke.
Methods— Using data from 5 major randomized clinical trials (n=2184) testing rt-PA in the 0- to 6-hour window, we developed logistic regression equations using clinical variables as potential predictors of a good outcome (modified Rankin Scale score ≤1) and of a catastrophic outcome (modified Rankin Scale score ≥5), with and without rt-PA. The models were internally validated using bootstrap re-sampling.
Results— To predict good outcome, in addition to rt-PA treatment, 7 variables significantly affected prognosis and/or the treatment-effect of rt-PA: age, diabetes, stroke severity, sex, previous stroke, systolic blood pressure, and time from symptom onset. To predict catastrophic outcome, only age, stroke severity, and serum glucose were significant; rt-PA treatment was not. For patients treated within 3 hours, the median predicted probability of a good outcome with rt-PA was 42.9% (interquartile range [IQR]=18.6% to 64.7%) versus 25.3% (IQR=9.8% to 46.2%) without rt-PA; the median predicted absolute benefit was 12.5% (IQR=5.1% to 21.0%). The median probability for a catastrophic outcome, with or without, rt-PA was 15.2% (IQR=8.0% to 31.2%). The area under the receiver-operator characteristic curve was 0.788 for the model predicting good outcome and 0.775 for the model predicting bad outcome.
Conclusions— The Stroke-TPI predicts good and bad functional outcomes with and without thrombolysis. Incorporated into a usable tool, it may assist in decision-making.
- acute care
- acute Rx
- acute stroke
- clinical decision support
- emergency medicine
- predictive models
- thrombolytic Rx
The National Institute of Neurological Disorders and Stroke (NINDS) recombinant tissue plasminogen activator (rt-PA) Stroke Study demonstrated that intravenous rt-PA improves outcomes in acute ischemic stroke, when administered within 3 hours of symptom onset.1 The absolute benefit in the likelihood of a normal or near-normal outcome ranged from 11% to 15%, depending on the functional outcome scale. However, registry data from several studies suggests that only ≈2% of patients with acute ischemic stroke receive thrombolysis.2,3
Although many with acute stroke present beyond the 3-hour time window,4 rt-PA remains underused even among eligible patients5 and emergency physicians report reluctance to treat patients with thrombolytics, even in the ideal setting, in part because of the perceived risks and benefits of the treatment.6
Computer-based decision-support tools can improve process-of-care outcomes.7 For acute myocardial infarction, the Thrombolytic Predictive Instrument (TPI),8 which provides clinicians with 0% to 100% probabilities of clinically important outcomes, with and without thrombolysis, has been shown to increase the speed and likelihood of thrombolytic therapy for subgroups of patients that physicians often fail to treat.9 The purpose of this study was to develop a predictive instrument, the Stroke-TPI, that could similarly provide clinicians, at the point-of-care, with the probabilities of important clinical outcomes with, and without, thrombolysis in acute stroke. A companion study addresses whether such an instrument may also be useful in patient-selection beyond the 3-hour time window.10
We combined randomized clinical trial databases, from the NINDS rt-PA Study (Trial 1 and Trial 2),1 ATLANTIS A,11 ATLANTIS B,12 and ECASS 2.13 These trials represent the published clinical trial experience with the current dosage of rt-PA. ECASS 1 was not included because a different dose was used in that trial.14 These trials were detailed in the original articles, and their differences were highlighted in a pooled analysis.15 For modeling, we excluded patients with minor stroke (National Institutes of Health Stroke Scale [NIHSS] ≤4), as well as patients missing 90 day outcome data, NIHSS or time from symptom onset to treatment.
Selection of Prediction Outcomes
We aimed to create multivariate mathematical models to predict both the risks and the benefits of thrombolysis in a way most useful to clinicians. Because the influence of thrombolysis may differ across different outcomes and because these different outcomes may be important for informed decision-making, we developed 2 separate models to capture outcomes at opposite ends of the functional outcome scale (modified Rankin Score [mRS] ≤1 and mRS ≥5). These extreme thresholds were chosen because they divide the Rankin Score into relatively homogenous outcomes in terms of quality of life and patient preferences.16 As the effects of symptomatic intracerebral hemorrhage (sIH) on clinical outcome are captured by these measures, it was felt separate predictions for the rate of sIH would add redundant information that may lead to the “double-counting” of bad outcomes among those treated with thrombolysis.
Variable Selection and Model Building
Besides their presence in each of the component databases, several considerations influenced our selection of predictor variables. First, to reduce the likelihood of model over-fitting, we sought to limit the number of candidate variables tested. Thus, we considered only variables previously demonstrated to be prognostically important, or likely to modify treatment effect.17–21 Second, we considered whether the variables were likely to be easily and reliably obtainable for real-time pretreatment use. Third, because we were especially interested in patient characteristics that might modify the effect of therapy, we permitted all variables to interact with treatment, even when the main effect did not significantly predict outcome. We also included the interaction of stroke severity (NIHSS score) and age, previously shown to be prognostically important.20 Otherwise, no additional interaction variables were tested. Variables and interactions were included in the final model at a threshold probability value of 0.05.
We anticipated that careful reading of the presenting CT scan might be important for patient-selection for thrombolysis. We therefore endeavored to obtain a complete set of CT scan readings using the Alberta Stroke Program Early CT (ASPECT) Score.22 However, we recognized that obtaining real-time ASPECT scores might not be feasible for nonspecialized physicians, so we modeled with and without this variable.
Assessing Model Performance and Statistical Validity
The performance of each model was assessed by the receiver-operator characteristic (ROC) curve area and calibration curves. Because model performance measures on the database on which it was developed may be over-optimistic because of over-fitting, to assess how model performance might degrade when the predictive instrument is applied to independent data drawn from the same population, we internally tested this using 0.632 bootstrapping23 (described in more detail in our companion article10). Using this resampling technique, we created 250 individual development datasets, each with a companion independent test dataset, and used automated procedures to develop and test 250 models. The median ROC and calibration curves for development and test sets were examined.
Although these procedures evaluated model performance of across patient subgroups, because our intention is to support physicians treating individual patients, we subjected model predictions to further scrutiny. Because during stepwise model building, multiple treatment effect interactions were tested, we considered that an identified interaction in the model might arise by chance. Therefore, we assessed the validity of these interactions by examining the direction of the interaction and seeking confirmation from the literature. Isopleths of the relationship of outcome with and without treatment over the entire range of each of the covariates were examined to ensure the model described known or plausible relationships between variables and outcomes, and variables and treatment effects. In addition, interaction terms were tested for possible nonlinear effects. We also checked for consistency of model performance between each of the component database and we examined whether the individual effects of predictor variables were especially influenced by any of the component databases. Finally, the predictive models were incorporated into usable computer-based tools and individual patient predictions were examined for face validity by investigators and outside stroke experts.
Of 2184 patients in the combined database, 2131 were included in the dataset for modeling (after excluding minor stroke [n=29] or those missing data on outcome [n=9], baseline NIHSS [n=2] or symptom onset to treatment [n=13]). Of these, 1062 (50%) received placebo, 1307 (61%) were treated after 3 hours, 773 (36%) had normal or favorable outcomes (mRS ≤1), and 464 (22%) had catastrophic outcomes (mRS ≥5). Patient characteristics are in Table 1.
Predicting Normal/Near-Normal Outcome
The final model predicting mRS ≤1 is shown in Table 2. All variables considered for model inclusion had a significant effect on this outcome, with the exception of sex (which was included in the model because of the significant treatment interaction) and glucose (which showed a strong trend for decreasing the likelihood of a normal/near-normal outcome [P=0.07]). Four variables were found to modify the effect of thrombolysis: being male, having a higher initial systolic blood pressure, a previous stroke, or longer delays from symptom onset to treatment, all predicted less benefit from thrombolysis. There was no evidence for nonlinearity in these interactions. The ROC area was 0.788. Inclusion of baseline ASPECTS did not significantly improve the model.
When 0.632 bootstrapping was performed, we found the median area under the ROC was 0.793 (interquartile range [IQR] 0.786 to 0.799) on the developmental data, which diminished to 0.772 on the independent test data (IQR 0.763 to 0.782). Calibration of the predictions was excellent, shown for the test data in Figure 1A.
Predicting Catastrophic Outcome
The final model predicting mRS ≥5 is in Table 3 (with and without ASPECTS). Of the clinical variables, only age, NIHSS, and serum glucose were predictive of a poor outcome. The baseline ASPECT score (as a linear variable) was also highly predictive of the likelihood of a poor outcome. Importantly, this outcome was independent of thrombolysis, suggesting that treatment does not have a significant positive or negative effect on likelihood of poor outcome in the 0- to 6-hour window. Further, with the possible exception of diabetes, effect of rt-PA was not modified by any other variable. This suggests that the absence of a treatment effect on this outcome is generalizable to all subgroups tested (ie, regardless of stroke severity, ASPECT score, presence or absence of a previous stroke, systolic blood pressure, etc.) In automated stepwise model building, there was a strong interaction between rt-PA treatment and diabetes (P=0.02), suggesting that diabetics may be more likely to be harmed by therapy compared with nondiabetics. However, when this was re-tested on the dataset including patients excluded from the automated procedure because they were missing other variables shown to be nonsignificant in the final model (ie, on those with complete data for age, treatment, NIHSS, glucose, time and diabetes [n=2109]), this interaction effect was substantially mitigated and was no longer statistically significant (P=0.23).
The model had an ROC area of 0.775 (0.789 with ASPECTS). When 0.632 bootstrapping was performed, the average ROC area was 0.784 (IQR, 0.774 to 0.794) on the developmental data, and was 0.762 on the independent test data (IQR, 0.749 to 0.776). Calibration of the predictions was excellent, shown for the test data in Figure 1B.
Individual Patient Predictions
For patients treated within the 3-hour time window, the median predicted probability for a good outcome with rt-PA was 42.9% (IQR 18.6% to 64.7%), versus 25.3% (IQR 9.8% to 46.2%) without; the median predicted absolute benefit was 12.5% (IQR 5.1% to 21.0%). The median predicted probability for a catastrophic outcome was 15.2% (IQR 8.0% to 31.2%), with or without rt-PA.
Examination of individual predictions and isopleths uncovered one relationship that contradicted clinical judgment. There was a very small nonsignificant trend for better outcomes among placebo-treated patients with a previous stroke compared to those without, which was incorporated into the model only because the treatment interaction with prior stroke was significant. Thus, for consistency with clinical judgment, this was adjusted such that previous stroke has no influence on predictions in placebo-treated patients, without changing its strong negative impact with thrombolysis.
The Stroke-TPI provides patient-specific predictions of the probability of a normal or near-normal outcome, and also the probability of a catastrophic outcome (death or very severe disability), if treated and if not treated with thrombolysis. Incorporated into a handheld (shown in Figure 2) or other computer system, this predictive instrument could potentially support physician decision-making and counseling of patients and families.
Use of rt-PA in acute ischemic stroke is associated with a reperfusion rate of ≈50%, depending on the location of the occlusion,24,25 but does so at a risk of parenchymal hematoma of ≈4%.15 Balancing these benefits and risks is difficult and physician uncertainty of the impact of rt-PA in acute stroke is an important barrier to thrombolysis.26 The Stroke-TPI summarizes the impact of reperfusion and hemorrhage on both good and bad functional outcomes, as patient-specific prognoses. Providing such estimates at the point-of-care may mitigate physician uncertainty and improve decision-making.
Table 4 shows 9 randomly selected patients treated in the 0- to 3-hour time window. As can be seen, for each of these patients, the benefits of thrombolysis outweigh its risks. This is true for all but 6.5% of database patients treated in the 0- to 3-hour window. Yet, even in this time-window, there is considerable heterogeneity in patient outcome and treatment benefit. When discussing the outcomes of potential therapies, physicians typically quote average results of clinical trials. However, because of this heterogeneity, adjusting these average results to take into account important individual patient characteristics may be more useful in clinical decision-making. (Interested readers can examine model predictions using an online version of the Stroke-TPI at http://www.nemc.org/icrhps/faculty/fac_respage/KentD_respage.asp, where any relevant updates will also be posted).
Our modeling yielded several important clinical findings. First, we identified 4 variables that diminish the benefits of thrombolysis: longer symptom onset to treatment time; male sex; higher systolic blood pressure, and a history of previous stroke. These treatment-effect interactions are discussed in more detail in a companion article.10 Second, thrombolysis did not appear to have any effect, positive or negative, on the likelihood of severe disability or death, either on average, or for any tested subgroups (eg, those with versus without prior stroke, those with higher versus lower blood pressure, etc.). This suggests that, for this outcome, the increased risk of sIH is approximately balanced by improvement in those patients achieving reperfusion across all these subgroups, and that subgroups at higher risk for thrombolytic-related sIH are also at similarly high risk for poor outcomes without thrombolysis.
Despite the benefits found in clinical trials in the 0- to 3-hour time window, a study of academic medical centers found that only 20% of those eligible for treatment actually received thrombolysis.5 One survey found that 40% of emergency department physicians, 91% of whom were board certified in this specialty, were either uncertain, unlikely, or very unlikely to use rt-PA for stroke in the ideal setting.6 Among the reluctant subset of physicians, the average maximally acceptable rate of thrombolytic-related sIH was 2.1%, even if thrombolysis improves the likelihood of a good outcome by 45%. Such a response suggests a cognitive error. Whereas the impact of both thrombolytic-related reperfusion and thrombolytic-related hemorrhage on patient quality of life are both fully accounted for and summarized in the functional outcome measure (as long as both good and bad functional outcomes are measured), physicians appear to be, in addition, considering the likelihood of sIH as independent information, suggesting that these risks are being double counted. Presenting the likelihoods of both the good and the bad functional outcomes simultaneously, with and without rt-PA, might allow physicians to better understand the trade-offs between reperfusion and hemorrhage and to better summarize these trade-offs when obtaining informed consent.
There are several limitations to the study. First, the predictive equations are based on outcomes achieved in the major European and North American randomized clinical trials, and thus the outcome predictions assume strict adherence to treatment protocols and may apply only to those treated in similar settings. Although outcomes in community practice may vary, outcomes similar to trial results are achievable in routine practice.27–29 Similarly, predictions may not be reliable for patients who are not well-represented in the database (eg, those age >85, those with initial systolic blood pressure >200 mm Hg, or those with pre-existing disability). Second, the variation in treatment effect is dependent on the four treatment effect interactions; because eight different interactions were tested, there is the possibility that one or more of these may have arisen by chance, or that other interactions that may be clinically important are not included because these effects did not reach statistical significance in this particular sample. These considerations are discussed in a separate article dealing with patient selection.10 Finally, the effects of the Stroke-TPI on physician decision-making, process, and clinical outcomes have not been tested, effects that should be studied in a multi-center clinical evaluation.
Despite these limitations, the Stroke-TPI offers promise as a real-time clinical decision aid for an area of clinical medicine that seems especially in need of such support. A similar decision-support instrument improves use of thrombolytics in AMI for subgroups of patients in whom thrombolysis is suboptimal.8,9 Use of a predictive instrument for stroke might have even greater impact because individual physicians who provide acute stroke care generally have extremely limited experience administering thrombolysis for stroke. Further, embedding the predictive instrument in a comprehensive computerized tool that includes other acute stroke support (such as an NIHSS calculator, a dosage calculator, treatment guidelines, etc) presents the opportunity to further improve timeliness and appropriateness of therapy.30 Finally, in acute stroke such a predictive instrument holds promise for the improvement of patient selection, possibly allowing selective treatment beyond the currently approved 3-hour time window for patients with an otherwise favorable risk–benefit treatment profile.31
The authors would like to acknowledge the work of the NINDS, ATLANTIS, and ECASS Trialists, without which this secondary analysis would not be possible. The following stroke experts assisted in review of Stroke-TPI model predictions: Michael D. Hill, MD; Karen C. Johnston, MD; Steven R. Levine, MD; David E. Thaler, MD; and Judith Hinchey, MD.
Sources of Funding
This research was supported by grants from the NINDS to Dr Kent (K23NS44929 and R21NS48225).
E.B. is an employee at Boehringer Ingelheim, which has a financial interest in this study. W.H. has received honoraria for speaking at major symposia for Boehringer Ingelheim, and for participating in DSMB and a steering committee for Boehringer Ingelheim. W.H. is also on an advisory board for Boehringer Ingelheim. All other authors report no conflicts of interest.
- Received March 1, 2006.
- Revision received July 14, 2006.
- Accepted August 2, 2006.
Katzan IL, Furlan AJ, Lloyd LE, Frank JI, Harper DL, Hinchey JA, Hammel JP, Qu A, Sila CA. Use of tissue-type plasminogen activator for acute ischemic stroke: the Cleveland area experience. JAMA. 2000; 28: 1151–1158.
Johnston SC, Fung LH, Gillum LA, Smith WS, Brass LM, Lichtman JH, Brown AN. Utilization of intravenous tissue-type plasminogen activator for ischemic stroke at academic medical centers: the influence of ethnicity. Stroke. 2001; 32: 1061–1068.
Selker HP, Griffith JL, Beshansky JR, Schmid CH, Califf RM, D’Agostino RB, Laks MM, Lee KL, Maynard C, Selvester RH, Wagner GS, Weaver WD. Patient-specific predictions of outcomes in myocardial infarction for real time emergency use: a thrombolytic predictive instrument. Ann Intern Med. 1997; 127: 538–556.
Selker HP, Beshansky JR, Griffith JL. TPI Trial Investigators. Use of the electrocardiograph-based thrombolytic predictive instrument to assist thrombolytic and reperfusion therapy for acute myocardial infarction: a multicenter, randomized, controlled, clinical effectiveness trial. Ann Intern Med. 2002; 137: 87–95.
Kent DM, Selker HP, Ruthazer R, Bluhmki E, Hacke W. Can multivariable risk-benefit profiling be used to select treatment-favorable patients for thrombolysis in stroke in the 3 to 6 hour time window? Stroke. 2006; 37: 2963–2969.
Clark WM, Albers GW, Madden KP, Hamilton S. The rtPA (alteplase) 0- to 6-hour acute stroke trial, part A (A0276g) : results of a double-blind, placebo-controlled, multicenter study. Thromblytic therapy in acute ischemic stroke study investigators. Stroke. 2000; 1: 811–816.
Clark WM, Wissman S, Albers GW, Jhamandas JH, Madden KP, Hamilton S. Recombinant tissue-type plasminogen activator (Alteplase) for ischemic stroke 3 to 5 hours after symptom onset. The ATLANTIS Study: a randomized controlled trial. Alteplase Thrombolysis for Acute Noninterventional Therapy in Ischemic Stroke. JAMA. 1999; 282: 2019–2026.
Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D, Larrue V, Bluhmki E, Davis S, Donnan G, Schneider D, Diez-Tejedor E, Trouillas P. Randomised double-blind placebo-controlled trial of thrombolysis with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet. 1998; 352: 1245–1251.
Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, Boysen G, Bluhmki E, Hoxter G, Mahagne MH, Hennerici M. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS). JAMA. 1995; 274: 1017–1025.
Solomon NA, Glick HA, Russo CJ, Lee J, Schulman KA. Patient preferences for stroke outcomes. Stroke. 1994; 25: 1721–1725.
Counsell C, Dennis M, McDowall M, Warlow C. Predicting outcome after acute and subacute stroke: development and validation of new prognostic models. Stroke. 2002; 33: 1041–1047.
Johnston KC, Connors AF, Wagner DP, Knaus WA, Wang X, Haley EC Jr. A predictive risk model for outcomes of ischemic stroke. Stroke. 2000; 31: 448–455.
The NINDS t-PA Stroke Study Group. Generalized efficacy of t-PA for acute stroke. Subgroup analysis of the NINDS t-PA Stroke Trial. Stroke. 1997; 28: 2119–2125.
Marler JR, Tilley BC, Lu M, Brott TG, Lyden PC, Grotta JC, Broderick JP, Levine SR, Frankel MP, Horowitz SH, Haley EC Jr, Lewandowski CA, Kwiatkowski TP. Early stroke treatment associated with better outcome: the NINDS rt-PA stroke study. Neurology. 2000; 55: 1649–1655.
Pexman JH, Barber PA, Hill MD, Sevick RJ, Demchuk AM, Hudon ME, Hu WY, Buchan AM. Use of the Alberta Stroke Program Early CT Score (ASPECTS) for assessing CT scans in patients with acute stroke. AJNR Am J Neuroradiol. 2001; 22: 1534–1542.
Alexandrov AV, Grotta JC. Arterial reocclusion in stroke patients treated with intravenous tissue plasminogen activator. Neurology. 2002; 59: 862–867.
Kwan J, Hand P, Sandercock P. A systematic review of barriers to delivery of thrombolysis for acute stroke. Age Ageing. 2004; 33: 116–121.
Tanne D, Bates VE, Verro P, Kasner SE, Binder JR, Patel SC, Mansbach HH, Daley S, Schultz LR, Karanjia PN, Scott P, Dayno JM, Vereczkey-Porter K, Benesch C, Book D, Coplin WM, Dulli D, Levine SR. Initial clinical experience with IV tissue plasminogen activator for acute ischemic stroke: a multicenter survey: the t-PA Stroke Survey Group. Neurology. 1999; 53: 424–427.
Wang DZ, Rose JA, Honings DS, Garwacki DJ, Milbrandt JC. Treating acute stroke patients with intravenous tPA: the OSF stroke network experience. Stroke. 2000; 31: 77–81.
Hill MD, Buchan AM. Canadian Alteplase for Stroke Effectiveness Study (CASES) Investigators. Thrombolysis for acute ischemic stroke: results of the Canadian Alteplase for Stroke Effectiveness Study. CMAJ Can Med Assoc J. 2005; 172: 1307–1312.
Kent DM, Ruthazer R, Selker HP. Are some patients likely to benefit from recombinant tissue-type plasminogen activator for acute ischemic stroke even beyond 3 hours from symptom onset? Stroke. 2003; 34: 464–467.