Prediction of Recurrent Stroke and Vascular Death in Patients With Transient Ischemic Attack or Nondisabling Stroke
A Prospective Comparison of Validated Prognostic Scores
Background and Purpose— Several predictive scores have been developed and validated for stratifying cerebrovascular patients based on the risk of future (cerebro)vascular events. We aimed to prospectively compare the predictive accuracy of the Essen Stroke Risk Score, Stroke Prognostic Instrument, Hankey score, and the Life Long After Cerebral ischemia score.
Methods— Between August 2005 and December 2006, we included 2381 patients from 10 German stroke centers with an acute nondisabling ischemic stroke or transient ischemic attack and with prospective assessment of clinical variables for calculation of the predictive scores. A total of 1897 patients (79.7%) could be followed up for a median of 1 year. To evaluate the performance of each model, we calculated the area under the curve by receiver operating characteristic. In addition, we used the recommended cutoff values for calculation of sensitivity and specificity for stroke or the combined outcome of stroke or cardiovascular death.
Results— The Kaplan–Meier estimate for the overall annual stroke risk was 4.8% and for recurrent stroke or cardiovascular death 6.6%. We could confirm the predictive value of all 4 previously developed scores with a marginally superior performance of the SPI-II.
Conclusions— In patients with acute nondisabling ischemic stroke or transient ischemic attack, all 4 scores are able to stratify the risk of recurrent stroke or the combined outcome. Simple point scores (Essen Stroke Risk Score, Stroke Prognostic Instrument) may help to raise awareness for medical prevention in clinical routine and increase compliance with risk factor modification.
Identification of those patients with transient ischemic attack (TIA) or ischemic stroke at high risk of recurrent stroke is an important aspect of in-hospital and outpatient management to ensure an optimal risk– and cost–benefit relation. In addition, predictive scores could help to raise awareness of recurrent stroke risk in both patients and physicians, which in turn might improve adherence to medical and nonmedical prevention strategies. Finally, prognostic stratification can improve randomization in clinical trials and may be helpful to compare different study populations, the effect of nonrandomized treatment strategies, and effects of outpatient clinics or structured prevention programs. Several predictive scores for cerebrovascular patients have been proposed and some have also demonstrated external validity. The Essen Stroke Risk Score (ESRS; Table 1) was derived from the data subset of 6431 patients with ischemic stroke in the Clopidogrel versus Aspirin in Patients at Risk of Ischemic Events (CAPRIE) trial1,2⇓ and validated in 2 observational studies as well as in the European Stroke Prevention Study-2.3–6⇓⇓⇓ The Stroke Prognostic Instrument (SPI-II) score was developed from the original SPI-I by incorporating new variables identified in the Women’s Estrogen for Stoke Trial (WEST)7 and subsequent validation in 3 independent cohorts.8 The predictive score by Hankey et al was developed in 469 patients with TIA or minor stroke without prior stroke referred to a university hospital and was subsequently validated in a population-based cohort and in a clinical study population.9,10⇓ Finally, the Life Long After Cerebral ischemia (LiLAC) score was based on the Dutch TIA trial (recruitment in 1986 to 1989; arterial cause of cerebral ischemia) including 2447 patients with a mean follow-up of 10.1 years.11 Although this score so far has not been externally validated, its development is based on some of the most robust data on long-term risk of stroke and death in patients with TIA and minor ischemic stroke. Because recurrent stroke and cardiovascular death are clinically relevant outcomes, we sought to investigate the applicability of the existing prognostic scores by prospectively comparing their predictive accuracy in patients with acute TIA or nondisabling stroke.
This study was part of a prospective cohort study in 16 German neurology departments with an acute stroke unit, which recruited consecutively admitted patients with acute stroke or TIA. Ischemic stroke (TIA) was defined as a sudden focal neurological deficit of presumed arterial origin lasting ≥24 hours (<24 hours) with or without a corresponding ischemic lesion on brain imaging. Standardized case report forms included demographic information, risk factors, comorbidity, stroke severity on the National Institutes of Health Stroke Scale, modified Rankin Scale before stroke and at discharge, results of diagnostic workup, length of hospital stay, and secondary prevention at discharge. To prospectively assess the established prognostic scores, several new variables were included in the case report form starting in August 2005 until December 2006. After central double-key stroke input of all baseline data, data quality was monitored by automated consistency checks and queries were sent to the documenting physician. If consented, a central follow-up interview by telephone (or paper questionnaire on request) was performed biannually by trained interviewers (graduate medical students) who were unaware of the individual risk scores. All recurrent strokes and vascular events were counted from the day after admission. In case of a suspected recurrent cerebrovascular event, confirmation was sought from the treating general physician or hospital. The final confirmation of a recurrent vascular event was made by the study coordinator together with the principal investigator (C.W.) who were likewise not aware of the individual risk scores. Local citizen registries were checked if any patient could not be reached for follow-up and uncertain causes of death were confirmed on death certificates. The study was approved by the ethics committee of the medical faculty, University of Essen, and all patients included in the follow-up provided signed, informed consent. The 16 centers mentioned in the Appendix consecutively documented 5773 patients with a TIA (including amaurosis fugax) or ischemic stroke. To reduce selection bias, we only considered 3805 patients from 10 centers where >65% of all patients had given informed consent for central follow-up. We excluded patients with disabling stroke, defined as a modified Rankin Scale 4 or 5 1 day after admission to allow for a more reliable assessment of recurrent stroke, leaving 2568 patients with a modified Rankin Scale score <4. We did not substitute missing values for calculation of a single score. Therefore, 187 patients (7.3%) with missing variables were also excluded from the analysis. The flow chart of patient inclusion is depicted in Figure 1.
The 4 scores under investigation were calculated as shown in Table 1. The Hankey score was calculated by exponentiating the sum of the β coefficients multiplied by the corresponding item of the Cox proportional hazard model (Hankey score=exp[β1*×1+β2*×2+…β8*×8] with the β coefficients β1 to β8 from Table 1 and the corresponding items ×1 to ×8). The variable “residual neurological signs” in the original model was modeled as (minor) stroke compared with TIA (G. Hankey, personal communication). The LiLAC score was calculated in the same way after transforming the hazard ratios for each corresponding item in Table 1 into β coefficients (β coefficient=ln [hazard ratio]). We assessed the discrimination of each score for fatal and nonfatal stroke as well as for stroke or cardiovascular death.
The characteristics of all patients were reported as percentages or mean (median). Statistical comparisons were carried out by Fisher exact, Person χ2, or Mann–Whitney U test, as appropriate. Survival analysis of outcome events (recurrent stroke, recurrent stroke, or cardiovascular death) over the entire follow-up period was performed by means of the actuarial Kaplan–Meier technique.12 In addition, a receiver operating characteristic was drawn for each model plotting specificity against sensitivity. The ability of the models to discriminate between outcomes across the entire range of predicted probabilities was determined from the area under the curve (AUC) varying from 0.5 for a model that correctly predicts outcome no better than chance to 1.0 for a model that perfectly discriminates between the outcomes. For calculation of sensitivity and specificity, we used the recommended cutoffs of the ESRS and SPI-II for identification of high-risk patients. All statistical analyses were performed in SPSS 17.0.
After exclusion of 187 patients with incomplete baseline data, 2381 patients with an acute nondisabling ischemic stroke (N=1607 [67.5%]) or TIA (N=774 [32.5%]) remained for calculation of the 4 scores. A total 1929 (81.0%) of these patients had given informed consent for follow-up and 1897 (79.7%) could be followed for at least 6 months after the index event. Twenty-two patients (0.9%) withdrew their consent but were still alive after 6 months and 10 (0.4%) were lost to follow-up. Follow-up was terminated after June 2007 due to lack of funding, resulting in a mean follow-up of 10.7 (±5.0 SD) months (median, 12.0; quartiles 6.6 to 13.1; range, 6 to 22 months). Total follow-up time until first recurrent stroke, death, or study closeout was 1699 patient-years. Patients without follow-up or with incomplete baseline data were significantly older, more often had a previous cerebral ischemia or cardiac comorbidities, greater functional impairment on the modified Rankin Scale at discharge, and were at greater risk as indicated on the SPI-II, ESRS, and LiLAC scores (all significant). The baseline characteristics of all patients with and without follow-up or with incomplete baseline data are provided in Table 2.
Overall, 75 patients (4.0%) died during follow-up, 13 (17%) of cerebral stroke, 28 (37%) of other cardiovascular causes, 27 (36%) of noncardiovascular causes, and 7 (9%) of unknown causes. The Kaplan–Meier (KM) estimate for annual mortality was 4.2% (95% CI, 3.1% to 5.4%). A recurrent cerebral stroke occurred in 107 patients (5.6%), 42 (39%) of which were fatal or major strokes with persisting disability. Ninety-six percent of all stroke events could be confirmed by the treating physician or on a hospital report. The KM curves for survival free of stroke and free of recurrent stroke or cardiovascular death are shown in Figure 2.
The KM estimate for the overall annual risk of stroke was 4.8% (95% CI, 3.9 to 6.2). Thirty-one stroke events (29%) occurred during the initial hospital stay in the documenting center. Cumulative numbers of recurrent stroke at 30 and 90 days were 38 (KM estimate 2.0%; 95% CI, 1.4 to 2.6%) and 50 (KM estimate 2.6%; 95% CI, 1.8% to 3.4%), respectively. The KM estimate for overall annual risk of recurrent stroke or cardiovascular death was 6.6% (95% CI, 5.3% to 7.9%). On stratification on the ESRS score, the annual KM estimate for recurrent stroke was 4.2% (95% CI, 3.0% to 5.5%) in the low-risk group (0 to 2 points) and 7.4% (95% CI, 4.9% to 9.8%) in the high-risk group (≥3 points; P=0.001). The corresponding AUC for the ESRS score was 0.62 (95% CI, 0.57% to 0.67%; Table 3). On stratification on the SPI-II score, the annual KM estimate for recurrent stroke was 3.2% (95% CI, 1.8% to 4.6%) in the low-risk group (0 to 3 points), 5.5% (95% CI, 3.8% to 7.3%) in the medium-risk group (4 to 7 points), and 9.1% (95% CI, 5.5% to 12.7%) in the high-risk group (8 to 15 points; P<0.001). The corresponding AUC for the SPI-II score was 0.65 (95% CI, 0.60 to 0.70). The sensitivity, specificity, and positive predictive value for stroke according to the predefined cutoffs for both scores are given in Table 4. The AUC for prediction of stroke on the Hankey score was 0.62 and on the LiLAC score 0.64 (Table 3). The receiver operating characteristic curves for prediction of stroke by each score are shown in Figure 3.
For prediction of stroke or cardiovascular death, the AUC was 0.66 for the SPI-II score, 0.65 for ESRS and LiLAC score, and 0.64 for the Hankey score (Table 3). The sensitivity, specificity, and positive predictive value for stroke or cardiovascular death of the ESRS and SPI-II are given in Table 5. The crosstabulation for classification on the ESRS and SPI-II at prespecified cut points is provided in Table 6.
Although several predictive scores for cerebrovascular patients have been validated, this is the first prospective study that directly compares the accuracy of these scores in a large cohort of patients with acute TIA or nondisabling stroke. Although consecutive patients were documented in the participating centers, only 81% of them gave informed consent for central follow-up. Because patients without follow-up or with incomplete baseline data were slightly older, had more cardiac comorbidities and greater stroke severity, and therefore a significantly elevated recurrent stroke risk on several scores, we cannot exclude an underestimation of the recurrent stroke rate or the performance of the predictive scores. Although all scores were designed for longer follow-up periods, our study could only compare the predictive accuracy within the first year. Nevertheless, almost all end point events could be confirmed by the treating physician or on death reports. Because no confirmation of other (nonfatal) vascular events was available, we were unable to investigate the prediction for additional end points. Although all scores were developed in different research cohorts, the discrimination for stroke and the combined end point in our prospective validation study was very similar across each score with only a marginal superiority of the SPI-II score. In addition, the rates of stroke or cardiovascular death for the ESRS and SPI-II across different risk groups corresponded well to those identified in previous validation studies.3,8⇓ Thus, there seems to be an acceptable transferability to a more general population. Preference should therefore be given to a score that is well described and easy to calculate. Simple point scores such as the SPI-II or ESRS also have defined cutoff values for low-, medium-, and high-risk patients. However, positive predictive values of these scores were still low and physicians may question the usefulness of predictive scores in daily clinical routine. Retrospective analyses have suggested that a combination therapy of dipyridamole/aspirin6 or clopidogrel monotherapy,2 compared with aspirin alone, may be particularly beneficial in patients at high risk as assessed by the ESRS. Thus, a stratification of patients according to risk of recurrent stroke may possibly lead to further optimized treatment regimens in addition to modification of stroke risk factors. Unfortunately, the efficacy of different medication regimens could not be tested in this observational study.
In conclusion, we could confirm the predictive value of all 4 previously developed scores for recurrent stroke and the combined end point of stroke or cardiovascular death in patients with acute TIA or nondisabling ischemic stroke. Given a similar (although not very accurate) prediction of the scores, preference should be given to simple point scores (ESRS, SPI-II), which are more likely to be used in clinical routine where they could help to raise awareness for recurrent stroke and cardiovascular risk.
Participating Departments of Neurology (investigator): Ostalbklinikum Aalen (M. Heyden, MD), Klinikum Bernburg (M. Müller, MD), Krankenhaus Buchholz (K. Luckner, MD), University of Essen (C. Weimar, MD), University of Freiburg (C. Fritzsch, MD), University of Hannover (K. Weissenborn, MD), Klinikum Heidenheim (S. Kaendler, MD), Krankenhaus Köln-Mehrheim (U. Frost, MD), University of Leipzig (D. Michalski, MD), Landesklinik Lübben (C. Röhrig, MD), University of Magdeburg (M. Goertler, MD), Ruppiner Kliniken Neuruppin (G. Zindler, MD), University of Rostock (A. Kloth, MD), Bürgerhospital Stuttgart (T. Mieck, MD), University of Ulm (R. Huber, MD), and Heinrich-Braun-Krankenhaus Zwickau (S. Grieshammer MD).
Source of Funding
This study was supported by the German Research Foundation (Deutsche Forschungsgemeinschaft, DI 327/8-1, DI 327/9-1).
- Received July 9, 2009.
- Revision received October 26, 2009.
- Accepted November 17, 2009.
- ↵Weimar C, Diener HC, Alberts MJ, Steg PG, Bhatt DL, Wilson PW, Mas JL, Rother J. The Essen Stroke Risk Score predicts recurrent cardiovascular events: a validation within the REduction of Atherothrombosis for Continued Health (REACH) registry. Stroke. 2009; 40: 350–354.
- ↵Weimar C, Goertler M, Rother J, Ringelstein EB, Darius H, Nabavi DG, Kim IH, Benemann J, Diener HC. Predictive value of the Essen Stroke Risk Score and ankle brachial index in acute ischemic stroke patients from 85 German stroke units. J Neurol Neurosurg Psychiatry. 2008; 79: 1339–1343.
- ↵Kernan WN, Viscoli CM, Brass LM, Makuch RW, Sarrel PM, Roberts RS, Gent M, Rothwell P, Sacco RL, Liu RC, Boden-Albala B, Horwitz RI. The Stroke Prognosis Instrument II (SPI-II): a clinical prediction instrument for patients with transient ischemia and nondisabling ischemic stroke. Stroke. 2000; 31: 456–462.
- ↵Hankey GJ, Slattery JM, Warlow CP. Can the long term outcome of individual patients with transient ischaemic attacks be predicted accurately? J Neurol Neurosurg Psychiatry. 1993; 56: 752–759.
- ↵Hankey GJ, Slattery JM, Warlow CP. Transient ischaemic attacks: which patients are at high (and low) risk of serious vascular events? J Neurol Neurosurg Psychiatry. 1992; 55: 640–652.