Targeting Recombinant Tissue-Type Plasminogen Activator in Acute Ischemic Stroke Based on Risk of Intracranial Hemorrhage or Poor Functional Outcome
An Analysis of the Third International Stroke Trial
Background and Purpose—Intravenous recombinant tissue-type plasminogen activator (r-tPA), despite a risk of early symptomatic intracranial hemorrhage (sICH), is of net clinical benefit to acute stroke patients. We tested if predictive models could identify patients least likely to be harmed by sICH or those who gained no net benefit.
Methods—We used the Third International Stroke Trial (IST-3) trial data set, an international, multicenter, open treatment randomized trial of 0.9 mg/kg r-tPA versus control in 3035 patients with acute ischemic stroke. We compared the discrimination and calibration of previously developed predictive models for ICH and poststroke poor outcome and developed a new model using variables selected by systematic review. We calculated the absolute and relative risk reduction of death or dependency with r-tPA in patients at a low, medium, or high predicted risk of sICH or poor functional outcome.
Results—Prediction models for sICH or poor outcome (Hemorrhage After Thrombolysis [HAT]; Sugar, Early Infarct Signs, Dense Artery, Age, National Institutes of Health (NIH) Stroke Score (SEDAN); Glucose Race Age Sex Pressure Stroke Severity [GRASPS]; Stroke Thrombolytic Predictive Instrument; Dense Artery, Rankin Score, Age, Glucose, Onset to Treatment Time, NIHSS [DRAGON]; Totaled Health Risks in Vascular Events [THRIVE]; our new model; and a model with National Institutes of Health Stroke Scale and age) had similar area under receiver operator characteristic curves (AUROCC) to predict sICH (P for difference >0.05). The simplest model (with covariates National Institutes of Health Stroke Scale and age) predicted both sICH (AUROCC, 0.63; 95% CI, 0.58–0.68) and poststroke poor functional outcome (AUROCC, 0.80; 95% CI, 0.77–0.82) similarly to complex models. There was no evidence that the effect of r-tPA in patients at high predicted risk of sICH or poor functional outcome after stroke was less than in those at lower risk.
Conclusions—There is a clinically relevant net positive effect of r-tPA in patients with acute stroke at a high predicted risk of sICH or poor functional outcome.
Intravenous thrombolytic therapy <3 hours of acute ischemic stroke is associated with a 3% to 4% absolute increase in the risk of symptomatic intracranial hemorrhage (sICH),1 which is either fatal or increases the risk of dependence.2 Despite this early hazard, the net clinical effect of thrombolysis is substantial; for every 1000 patients treated <3 hours, 90 more will be alive and independent poststroke.1 However, clinicians may be unduly concerned about the early risk of sICH,3 perhaps denying some patients the opportunity of clinical benefit from treatment.
Scores to predict very high sICH risk or negligible clinical benefit from intravenous thrombolysis might help clinicians to select patients for treatment. To test the hypothesis that prediction scores improve selection and lead to greater net clinical benefit, we analyzed data from the Third International Stroke Trial (IST-3). We wished to use the best possible prediction models and so aimed to undertake a systematic review of previous models, develop new models with novel statistical approaches, and then estimate the likely clinical impact with each model. In other words, we sought to assess whether the benefits and harms of thrombolysis vary in groups with different predicted prognosis.
The study was approved by the Multicenter Research Ethics Committee, Scotland (reference MREC/99/0/78) and by local ethical committees. Patients or a valid proxy gave written consent to participate. This trial was registered (ISRCTN25765518).
IST-3 Study Design and Participants
The details of the IST-3 study protocol,4 statistical analysis plan,5 and primary outcomes6 have been published previously. In brief, ischemic stroke patients (with no upper age limit) who could start recombinant tissue-type plasminogen activator (r-tPA) treatment <6 hours of symptom onset, and in whom the randomizing clinician was substantially uncertain about the risks and benefits of r-tPA, were randomized 1:1 to standard care with an infusion of 0.9 mg/kg rt-PA or standard care without r-tPA.
Measurement of Baseline Variables and Clinical Outcomes
For these analyses, we used baseline clinical variables that had been measured and recorded by the treating clinician before randomization, nonblinded information collected postrandomization, and findings from the brain scans that had been read by an expert panel blinded to clinical details and allocated treatment.
The trial event adjudication committee defined symptomatic post–r-tPA ICH (sICH) as a clinically significant deterioration or death within the first 7 days of treatment with evidence of either significant brain parenchymal hemorrhage (local or distant from the infarct) or significant hemorrhagic transformation of an infarct on brain imaging.5 In addition, we extracted the variable any significant radiological post–r-tPAICH by 7 days, measured by a blinded neuroradiology rater either on routine brain imaging 24 to 48 hours postrandomization or any scans performed in case of clinical deterioration (equivalent to parenchymal hemorrhage type 2 measured in previous trials of intravenous r-tPA).7 The primary measure of clinical outcome was the Oxford Handicap Scale (OHS) measured at 6 months after randomization. We defined poor functional outcome as an OHS of 3 to 6 (dead or dependent). We performed a post hoc sensitivity analysis using a definition of poor functional outcome of OHS 5 to 6 (dead or dependent for all cares).
Identification of Previously Developed Prediction Models
We identified published clinical prediction scores by systematically searching the literature for models that aimed to predict post–r-tPA sICH or poor functional outcome after r-tPA. We hypothesized that a simple model containing only the variables National Institutes of Health Stroke Scale (NIHSS) and age8 would predict both sICH and poor functional outcome as well as the other scores.
Development of a New Predictive Model for sICH
We developed a new model to predict sICH in patients randomized to r-tPA from IST-3. We created a binary logistic regression model with variables significantly associated with post–r-tPA ICH in a systematic review.7 We tested model assumptions of linearity and additivity and the effect of missing data. We internally validated the model with 150 bootstrap replicates and shrinkage of estimated regression coefficients to correct for overfitting.
Calibration, Discrimination, and Classification of Predictive Models for sICH and Poor Functional Outcome
We tested model performance in r-tPA–treated patients for the outcomes sICH, any significant radiological post–r-tPA ICH, and poor functional outcome. We measured discrimination with the area under receiver operator characteristic curve (AUROCC), which we compared nonparametrically. An AUROCC=1 indicates perfect discrimination, and AUROCC=0.5 indicates no better discrimination than chance. To test model calibration, we calculated the calibration slope and intercept by fitting a logistic regression model with predicted risk as the only predictor (where a slope=1 and intercept=0 indicates a perfectly calibrated model) and compared the proportions of patients classified as low, medium, and high risk with each model.
The choice of risk thresholds is controversial. In the absence of generally agreed thresholds, we used the mean of risk thresholds from previous studies to define low, medium, and high risk of post–r-tPA ICH and poor functional outcome. The means of the published thresholds for ICH were ≤3%, 3% to 8%, and >8%, and for poor functional outcome were ≤35%, 35% to 56%, and >56%. In a secondary analysis, we examined thresholds for a very high risk of sICH (>20%) and very high risk of poor functional outcome (>70%).
Effect of r-tPA in Patients at High, Intermediate, and Low Risk of ICH
We investigated the interaction between r-tPA treatment and predictions of sICH or poor functional outcome on an absolute risk scale, by calculating the difference in the proportion of patients with poor functional outcome between patients treated with and without r-tPA in groups of patients at low, medium, and high risk. Where possible, we recalibrated the intercept of prediction models to the IST-3 data set. To support this analysis, we looked for interactions on a relative scale between treatment and predicted risk as a continuous variable, using ordinal logistic regression with the whole OHS as the dependent variable, after examining the proportional odds assumption.
We performed sensitivity analyses excluding those few patients randomized to r-tPA who did not receive any, examining only those patients where the time to randomization was <4.5 hours and only those treated after 4.5 hours, and in addition made further adjustment for delay to treatment as a continuous variable. Post hoc we repeated this analysis in patients randomized <3 hours after stroke onset.
We used R version 2.13.1 for statistical analysis. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the article.
In patients treated with r-tPA, 6.8% (104 of 1515) had a sICH <7 days of randomization; another 2% (31) had a radiological hemorrhage by 7 days with no detectable clinical deterioration. The median time from randomization to sICH was 1 day (interquartile range, 1–2). By 6 months after randomization, few patients who had a sICH were independent in activities of daily living (8 of 104; 8%) compared with r-tPA–treated patients who did not have a sICH (546 of 1411; 39%).
Patients who had an sICH (Table 1) were significantly (P<0.05) more likely to have had a history of stroke or transient ischemic attack, to have been taking an antiplatelet agent in the 48 hours before randomization, to have had more neurological impairment or a higher blood glucose at randomization, or to have had a visible infarct (in any location) or hyperdense artery on brain imaging. There was no detectable effect of delay to randomization or delay to treatment on the odds of sICH.
Identification of Previously Developed Prediction Models
We identified 5 scores to predict post–r-tPA ICH and 3 scores to predict post–r-tPA poor functional outcome. We excluded 4 potentially relevant scores: 2 for which we were unable to calculate predicted risks from the published information, and 2 as they required baseline information that was not available in IST-3 (platelet count and a diagnosis of cancer or renal failure; Table I in the online-only Data Supplement).
Development of Model to Predict sICH in IST-3
A logistic regression model for the prediction of sICH, developed in 1515 r-tPA–treated patients with variables significantly associated with ICH from our previous systematic review7 (age, NIHSS, glucose, previous hypertension, atrial fibrillation, antiplatelets, diabetes mellitus, leukoaraiosis, and visible infarction), was able to discriminate modestly between patients with and without sICH (AUROCC corrected for optimism, 0.65) and was well calibrated in this data set (calibration slope corrected for optimism, 1.12; intercept, 0.32; Table II in the online-only Data Supplement).
There were no statistically or clinically significant 2-way interactions between categorical and continuous variables, and there was no evidence of a nonlinear relationship between any continuous variables with the odds of sICH. Multiple imputations for missing data made very little difference to the magnitude or direction of the estimates.
Calibration and Discrimination of Predictive Models for sICH and Poor Functional Outcome
All models to predict sICH or poor functional outcome discriminated modestly between patients who did and did not have a sICH (AUROCC range, 0.56–0.68; Table 2). The AUROCCs of all models were similar (P>0.05), apart from the dichotomized Stroke Prognostication Using Age and NIHSS (SPAN) score, which had significantly worse discrimination (P<0.05). Each previously developed model discriminated less well than in previous validation data sets. Models developed to predict sICH were better calibrated for the sICH outcome than those models developed to predict post–r-tPA poor functional outcome, though all models (apart from the new score) overpredicted the risk of sICH. There were no important qualitative or quantitative differences in discrimination or calibration for any of the models when the outcome was radiological post–r-tPA ICH rather than symptomatic ICH (Table III in the online-only Data Supplement), or when we examined only those patients randomized <3 hours of stroke onset. Each model classified a different proportion of the r-tPA–treated population at high, medium, and low risk of sICH, differences that are potentially clinically relevant (Figure 1).
All models discriminated moderately well between patients who did and did not have a poor functional outcome after stroke (AUROCC range, 0.66–0.80). There were no significant differences in discrimination between models designed to predict poor functional outcome post–r-tPA (Stroke Thrombolytic Predictive Instrument; NIHSS/age; Dense Artery, Rankin Score, Age, Glucose, Onset to Treatment Time, NIHSS [DRAGON]; new model; Totaled Health Risks in Vascular Events [THRIVE]; differences all P>0.05), except the SPAN score that was significantly worse than other models (P<0.001). All the models were well calibrated for death or dependence, whether or not they aimed to predict sICH or poststroke poor functional outcome. A sensitivity analysis examining a different definition of functional outcome (OHS, 5–6) made no difference to these conclusions.
Although the novel IST-3 score we developed had better discrimination than previous models to predict sICH, the absolute difference in the AUROCC between it and other models was small and likely because of model overfitting, and therefore we do not think it will perform better than the previously developed models in external validation.
Effect of r-tPA in Patients at High, Intermediate, and Low Risk of ICH
In the 3035 patients in IST-3, we observed that the absolute risk reduction in poor functional outcome with r-tPA treatment was greater both among patients at higher predicted risk of sICH (Figure 2) and among patients at higher risk of poor functional outcome (Figure 3). With the more statistically efficient ordinal logistic regression to measure treatment effect, there was no evidence of significant interactions between r-tPA with continuous predicted risk of sICH or poor functional outcome on a relative scale. These conclusions were not changed by excluding patients who were randomized but not treated with r-tPA, patients who were randomized >3 or >4.5 hours after stroke, making adjustment for delay to treatment as a continuous variable, or when examining higher thresholds of risk for sICH (>20%) or poor functional outcome (>70%). There was, -therefore, no evidence to support a strategy of avoiding r-tPA treatment in patients at a higher predicted risk of sICH or poor functional outcome in the IST-3 data set.
The clinical effect of r-tPA in patients at a higher predicted risk of sICH or poor functional outcome was at least as good as, and possibly more than, in patients with a lower risk. We found prediction scores discriminated only modestly well between patients who did and did not have a sICH, though discriminated moderately well between patients who did and did not have a poor functional outcome.
Our analyses suggest that clinical prediction scores are unlikely to play a role in selecting individual ischemic stroke patients for r-tPA in routine practice. Patients (or their families) who want to know the probability of poor functional outcome or sICH could choose any 1 of these scores, accepting the uncertainty in absolute predicted risks for an individual. A simple score constructed with the fewest, most easily measured clinical variables (eg, NIHSS and age) would be the easiest to implement.
Our approach had several strengths. We selected comparator models from a systematic review and measured the performance of models to predict important clinical outcomes in a large data set. IST-3 is broadly representative of current clinical practice as it included many elderly patients and patients with severe stroke, and the rate of sICH was similar to that seen in clinical practice and previous clinical trials of r-tPA.9 This wider range of patients with differing prognoses from previous cohorts is a strength of the analysis. We developed a new prediction model for ICH minimizing data-dependent biases and maximizing the use of predictive information. Despite this, we were unable to make very much better prediction compared with previously published models. We, therefore, did not validate this model in a new data set.
There were no differences in our conclusions after sensitivity analyses were restricted to patients treated <4.5 hours after stroke (the time threshold of the current European Union license for r-tPA). IST-3 had few missing baseline or outcome data (though glucose was not collected in the first 282 patients randomized), had a wide range in potentially predictive variables because of its wide inclusion criteria, and randomly allocated r-tPA; so our conclusions about the use of scores to predict response to treatment are robust. Our conclusions are supported by recent work with observational data comparing treated and untreated acute stroke patients with several relative contraindications to r-tPA (high glucose levels, extensive CT findings, etc).10
We can, however, identify limitations. IST-3 was an unblinded trial, though steps were taken to minimize bias. Overall, IST-3 was a neutral trial in that there was no statistically significant difference in the dichotomous primary outcome—the proportion of patients dead or dependent after treatment with r-tPA (OHS, 0–2; adjusted odds ratio, 1.13; 95% CI, 0.95–1.35). However, the key secondary outcome, assessed by the more statistically efficient ordinal regression analysis, showed clear evidence of a favorable shift in disability scores at both 6 and 18 months,6,11 and the effect of r-tPA in IST-3 was similar to previous trials, after accounting for time to randomization.1 We tested the predictions of models constructed with easily measured baseline clinical and simple imaging variables. Future improvements in prediction are only likely if variables that we did not measure, such as advanced imaging methods, genotyping, or blood biomarkers related to the pathophysiology of post–r-tPA ICH, better predict response to treatment.
Clinical prediction models were unable to identify patients least likely to be harmed by sICH or those who gained no net benefit from r-tPA. These data suggest that intravenous r-tPA has an absolute beneficial effect in patients at a high predicted risk of sICH or poor functional outcome.
Sources of Funding
Supported by Stroke Association, The Health Foundation UK, UK Medical Research Council (G0400069, G0902303, G0800803, EME 09-800-15), Research Council of Norway, AFA Insurances, the Swedish Heart Lung Fund, Foundation of Marianne and Marcus Wallenberg, Stockholm County Council and Karolinska Institute, the Government of Poland (2PO5B10928), Australian Heart Foundation (G 04S 1638), Australian NHMRC (457343), Swiss National Research Foundation, the Swiss Heart Foundation, Assessorato alla Sanita, Regione dell’Umbria, Danube University, Chest Heart and Stroke Scotland, DesAcc, University of Edinburgh, Danderyd Hospital R&D Department, Karolinska Institutet, Oslo University Hospital, and the Dalhousie University Internal Medicine Research Fund. Drug and placebo for the 300 patients in the double-blinded component of the start-up phase were supplied by Boehringer Ingelheim. Dr Whiteley is supported by a Clinician Scientist Fellowship from the UK Medical Research Council (G0902303).
R.I. Lindley has received payment in his role as conference scientific committee member and for occasional lectures from Boehringer Ingelheim; has attended national stroke meetings organized and funded by Boehringer Ingelheim; and is not a member of any industry advisory boards. P. Sandercock has received lecture fees (paid to the Division of Clinical Neurosciences, University of Edinburgh) and travel expenses from Boehringer Ingelheim for occasional lectures given at international conferences, and was a member of the Independent Data and Safety Monitoring Board (DSMB) of the Randomized Evaluation of Long-Term Anticoagulation Therapy (RE-LY) trial funded by Boehringer Ingelheim and received attendance fees and travel expenses for attending DSMB meetings (paid to the Division of Clinical Neurosciences, University of Edinburgh). J. Wardlaw received reimbursement for reading CT scans for European Cooperative Acute Stroke Study III (ECASS III) from Boehringer Ingelheim in the form of funding to her department (the Division of Clinical Neurosciences, University of Edinburgh); has attended meetings held by Boehringer Ingelheim as an unpaid independent external adviser during the licensing of rt-PA, but was refunded for travel expenses and the time away from work; has attended and spoken at national and international stroke meetings organized and funded by Boehringer Ingelheim for which she received honoraria and travel expenses. The other authors have no conflicts to report.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.113.004362/-/DC1.
- Received December 2, 2013.
- Revision received January 16, 2014.
- Accepted January 27, 2014.
- © 2014 American Heart Association, Inc.
- Whiteley WN,
- Slot KB,
- Fernandes P,
- Sandercock P,
- Wardlaw J
- König IR,
- Ziegler A,
- Bluhmki E,
- Hacke W,
- Bath PM,
- Sacco RL,
- et al
- Frank B,
- Grotta JC,
- Alexandrov AV,
- Bluhmki E,
- Lyden P,
- Meretoja A,
- et al