External Validation of the SEDAN Score for Prediction of Intracerebral Hemorrhage in Stroke Thrombolysis
Background and Purpose—The SEDAN score is a prediction rule for assessment of the risk of symptomatic intracerebral hemorrhage (SICH) per the European Cooperative Acute Stroke Study (ECASS) II definition in patients with acute ischemic stroke treated with intravenous thrombolysis. We assessed the performance of the score in predicting SICH per the ECASS II and Safe Implementation of Treatments in Stroke Monitoring Study (SITS-MOST) definitions in the SITS–International Stroke Thrombolysis Register (SITS-ISTR).
Methods—We calculated the SEDAN score in 34 251 patients with complete data, enrolled into the SITS-ISTR. The risk for SICH by both definitions was calculated per score category. Odds ratios for SICH per point increase of the score were obtained using logistic regression. The predictive performance was assessed using area under the curve of the receiver operating characteristic (AUC-ROC).
Results—The predictive capability for SICH per ECASS II was moderate at AUC-ROC=0.66. With rising scores, there was a moderate increase in risk for SICH per ECASS II (odds ratio, 1.65 per point; 95% confidence interval, 1.59–1.72; P<0.001), with SICH rates between 1.6% for 0 points and 16.9% for ≥5 points, average 5.1%. The predictive capability for SICH per SITS–MOST was weaker, AUC-ROC=0.60, with lower increase per score point (odds ratio, 1.36 per point; 95% confidence interval, 1.28–1.46; P<0.001), and SICH rates between 0.8% for 0 points and 5.4% for ≥5 points, average 1.8%.
Conclusions—In this very large data set, the predictive and discriminatory performances of the SEDAN score were only moderate for SICH per ECASS II and low for SICH per SITS–Monitoring Study.
Symptomatic intracerebral hemorrhage (SICH) remains a rare but feared complication of thrombolytic therapy for acute ischemic stroke. Four clinical prediction algorithms have been published with the specific aim to estimate the risk of its occurrence in individual patients: the safe implementation of treatments in stroke (SITS)-SICH score, the SEDAN score, the hemorrhage after thrombolysis (HAT) score, and the Multicentre Stroke Survey score.1–4 The novel SEDAN score comprises 5 risk factors for SICH, obtained using multivariate logistic regression analysis on a single-center stroke thrombolysis population (n=974). Its total sum ranges between 0 and 6 points. The components are baseline blood glucose 8.1 to 12.0 mmol/L (145–216 mg/dL) =1 point and >12.0 mmol/L (>216 mg/dL) =2 points, early infarct signs on baseline CT =1 point, hyperdense cerebral artery sign =1 point, age >75 years =1 point, and baseline National Institutes of Health Stroke Scale (NIHSS) score ≥10 =1 point. The SEDAN score is designed to predict SICH per the definition introduced in the European Cooperative Acute Stroke Study (ECASS) II trial: “clinical deterioration causing an increase in the NIHSS score of ≥4 points and whether the hemorrhage was likely to be the cause of the clinical deterioration. However, in case of doubt regarding whether edema or hemorrhage was the leading pathology, an association of the hemorrhage with the deterioration was assumed.”5 The allowed time window for a hemorrhage to be labeled as SICH was 7 days from treatment. All variables used in the SEDAN score are available in the SITS–International Stroke Thrombolysis Register (ISTR) database. This allowed us to undertake an attempt to validate the risk score externally in the largest stroke thrombolysis register worldwide. Our secondary objective was also to test whether the SEDAN score can predict SICH per the SITS–Monitoring Study definition.6 It requires, in addition to a clinical deterioration of ≥4 NIHSS points, a parenchymal hemorrhage in excess of 30% of the cerebral infarct. This requirement eliminates false-positive labeling of SICH in patients who deteriorate because of infarct edema or extracerebral causes, which is a possibility with the ECASS II definition, as it permits SICH labeling even in patients with purely petechial hemorrhagic transformation of the infarct.
All patients recorded in the SITS-ISTR between December 25, 2002, and December 12, 2011, were included in this study, n=45 074. Patients were included if they presented with stroke symptoms and were treated with intravenous alteplase (Actilyse, Boehringer Ingelheim, Germany) within or without license criteria. Need for ethical approval or patient consent for participation in the SITS-ISTR varied among participating countries. Ethics approval and patient consent were obtained in countries that required this; other countries approved the register for conduct as an anonymized audit. The SITS-MOST data (n=6483)6 are embedded within the SITS-ISTR. The SITS-MOST was approved by the ethics committee of the Karolinska Institutet, Stockholm, Sweden and by the Swedish Medical Products Agency. The SITS International Coordination Office monitored the SITS-ISTR data online and checked individual patient data monthly to identify errors or inconsistencies. For a sample of patients included in SITS-MOST, source data were verified onsite by monitors under the supervision of the national coordinator.
The SITS-ISTR is an ongoing, prospective, Internet-based, academic-driven, multinational, observational monitoring register for clinical centers using thrombolysis for the treatment of acute ischemic stroke. The methodology of the SITS-ISTR, including procedures for data collection and management, patient identification and verification of source data, has been described previously.6,7 We collected baseline and demographic characteristics, stroke severity per the NIHSS, onset-to-treatment time, medication history, and imaging data at admission and follow-up. All assessments of imaging studies and neurological status were done according to clinical routine at centers participating in the SITS-ISTR. A follow-up computed tomography (CT) or MR scan at 22 to 36 hours after intravenous tissue-type plasminogen activator treatment was strongly recommended for all patients. All SICH events were adjudicated centrally by the SITS International Coordination Office based on recorded imaging and clinical data.
SICH per ECASS II was defined in the database as any type of intracerebral hemorrhage on any post-treatment imaging after the start of thrombolysis and increase of ≥4 NIHSS points from baseline, or from the lowest value within 7 days, or leading to death.
SICH per SITS-MOST was defined as a local or remote type 2 parenchymal hemorrhage on imaging 22 to 36 hours after treatment or earlier if the imaging scan was performed because of clinical deterioration combined with a neurological deterioration of ≥4 NIHSS points from baseline, or from the lowest NIHSS score between baseline and 24 hours, or leading to death within 24 hours. A grading of type 2 parenchymal hemorrhage for intracranial hemorrhage indicates a bleed >30% of the infarct, with substantial space occupation.
We performed descriptive statistics for baseline, imaging, and demographic data, comparing patients with and without SICH per ECASS II (Table 1). For continuous variables, median and interquartile range values were obtained. For categorical variables, we calculated percentage proportions by dividing the number of events by the total number of patients, excluding missing or unknown cases, as done in previous SITS publications.6,7 For calculation of significance of difference between medians and proportions, we used the Mann–Whitney U test and the Pearson χ2 method, respectively.
Assessment of the total SEDAN score was performed, including only patients for whom all necessary variables were available. Logistic regression was performed to determine the odds ratios for both SICH definitions per point increase of the score. The absolute rates of SICH per ECASS II and SITS-MOST definitions were calculated for each point sum category (Table 2). The area under the curve of the receiver operating characteristic (AUC-ROC or c statistic) was calculated to assess the overall predictive performance of the score for both SICH definitions. Multivariate logistic regression coefficients were obtained for the variables in the SEDAN score model, to compare their magnitude to the ones reported in the original SEDAN derivation cohort. The SITS investigators have previously reported a multivariate analysis of the SITS-MOST study population, showing independent risk factors for SICH per SITS-MOST and SICH per National Institute of Neurological Disorders and Stroke.8 However, independent risk factors for SICH per ECASS II have never been described in the SITS-ISTR material. To better understand the results from our validation attempt of the SEDAN score, we therefore performed a denovo backward stepwise logistic regression analysis for this outcome. Variables were included in the model based on univariate associations (P≤0.10; Table 1), with stepwise removal at P>0.10. To avoid likely loss of information through stratification of continuous variables, the unstratified independent risk factors for SICH per ECASS II were included in a comprehensive prediction model for this outcome, and its discriminative capacity was calculated using the AUC-ROC method.
A total of 45 079 patients treated with intravenous thrombolysis were recorded in the SITS-ISTR between December 25, 2002, and December 12, 2011, with 95% of the patients coming from European centers. Baseline characteristics of patients with SICH per ECASS II in the SITS-ISTR are shown in Table 1, compared by univariate analysis to patients without SICH.
Median age was 70 years (interquartile range 17), and median baseline blood glucose was 6.6 mmol/L (interquartile range 2.1), both identical to values reported in the SEDAN score derivation cohort.2 Our median NIHSS score was 12 (interquartile range 10), higher than the 10 (interquartile range 9) reported by the Helsinki group. The proportion of patients with a hyperdense cerebral artery sign was similar in our populations (18.6% versus 19.4%). Early infarct signs on baseline CT imaging were less common in the SITS-ISTR (20.1% versus 34.2%). The overall rate of SICH per ECASS II was 5.1%, and 1.7% for SICH per SITS-MOST. Data for all SEDAN score variables and outcomes were complete in 36 027 patients (80%) for ECASS II and 36 127 patients (80%) for the SITS-MOST SICH definitions. The radiological parameters, dense artery sign and early infarct changes, contributed the most to the missing data, being unavailable in 7.1% and 6.7% of cases, respectively, followed by blood glucose, missing in 5.7%.
Table 2 shows the risk for SICH per both definitions based on the SEDAN score categories. Patients with a SEDAN score of 6 points (n=20) comprised only 0.055% of the material and were included in the ≥5 points category. They had a SICH per ECASS II rate of 20%. The rarity of a maximum score is also noted in the original SEDAN publication, which reported no patients with 6 points at all, among 1802 cases.
After logistic regression analysis, the SEDAN score was associated with SICH per ECASS II (odds ratio, 1.65 per point increase; 95% confidence interval, 1.59–1.72; P<0.001) and less so with SICH per SITS-MOST (odds ratio, 1.36 per point increase; 95% confidence interval, 1.28–1.46; P<0.001). The AUC-ROC (c statistic) values were 0.66 for SICH per ECASS II and 0.60 for SICH per SITS-MOST. The regression coefficients of all variables of the SEDAN, including stratified glucose, age and NIHSS, are shown in Table 3, to allow comparison with the coefficients reported in the SEDAN design cohort.
The independent risk factors for SICH per ECASS II in our material, obtained by backward stepwise logistic regression, are shown in Table 4. The discriminating capacity of a prognostic model for SICH per ECASS II, using all 10 of these risk factors, without stratification of continuous variables, was 0.70 using AUC-ROC.
The SEDAN score is a well-designed clinical prediction rule for SICH per ECASS II. Its methodology follows closely the recommendations outlined by Moons et al9,12 and Altman et al11 in a recent authoritative article series in the British Medical Journal, covering the design, validation, and use of prediction scores.10 In our material, the SEDAN score showed a moderate ability to predict SICH per ECASS II. Its discriminatory capacity was lower than in its original derivation cohort, as well as in a Swiss external validation cohort (c statistic 0.66 versus 0.77).2 The absolute rates of SICH were lower in score categories 2 to ≥5 in our material, whereas more patients predicted to have the lowest risk (0–1 points) actually had SICH, compared with the original publication, as shown in Table 2. The predictive capability for SICH per SITS-MOST was weaker, with a c statistic of 0.60 and a smaller increase in risk per score level at odds ratio of 1.36.
Thus, the performance of the SEDAN score was not as strong in the SITS-ISTR as in its original derivation cohort and the external Swiss validation population.2 To elucidate why this was the case, we searched for independent risk factors for SICH per ECASS II in our own population, asking whether any factor of the original score lacked association with the outcome in our data, or whether any additional variables could have improved the model. All 5 parameters of the SEDAN were confirmed to be independent predictors of SICH per ECASS II. However, their regression coefficients, and hence adjusted odds ratios for SICH, were much lower than those reported in the original SEDAN publication (Table 3). Using logistic regression, we found 5 additional independent risk factors for SICH per ECASS II in our population: treatment with aspirin or clopidogrel at baseline, known hypertension, atrial fibrillation, and pre-thrombolysis systolic blood pressure (Table 4). Nevertheless, even when all 10 predictors (5 original SEDAN parameters plus 5 new ones) were put into a multivariate prediction model, its discriminative capability did not exceed a c statistic (AUC-ROC) of 0.70, that is, still lower than the c statistic of 0.77 reported in the original SEDAN publication.2 We consider the 2 main reasons for the weaker performance of the SEDAN score in our material to be as follows: (1) in the SITS-ISTR, individual risk factors for SICH are weaker predictors of this complication than suggested by results from smaller materials; therefore, their combinations also show less predictive capability; and (2) the SEDAN model could potentially have benefitted from the inclusion of additional risk factors (eg, antiplatelet therapy, baseline blood pressure, etc).
The ECASS II definition allows any type of hemorrhage, including petechial, for the diagnosis of SICH if the prerequisite clinical deterioration of ≥4 points on the NIHSS has occurred within 7 days from treatment. In our material, 31% of all patients classified as having SICH per ECASS II had only petechial bleeds, whereas 69% had parenchymal hemorrhages. Contrary to parenchymal hemorrhages of sufficient size, hemorrhagic petechiae are hard to conceptualize as true SICH. We think that the SEDAN score, attributable to the nature of the ECASS II definition, thus, predicts neurological deterioration of ≥4 NIHSS points, however, not exclusively attributable to hemorrhage but in at least 31% of cases rather attributable to cerebral edema and extracerebral complications. This conclusion is supported by recent publications reporting that the ECASS II definition of SICH is less strongly associated with poor outcome and death (modified Rankin scale, 5–6) than the SITS-MOST definition.13–15
Our study has several limitations. Data were collected via a prospective clinical register; hence, the study holds all the drawbacks of observational design. CT and MRI scans were read by local radiologists, according to clinical routine; therefore, they were not blinded to clinical information; however, this was also the case in the original SEDAN derivation and validation cohorts. The proportion of patients with missing data for ≥1 variable or the main outcome in our material was 20%, which may have influenced the outcome. This can be compared with 12.7% patients with missing data in the Swiss external validation cohort in the original SEDAN publication.2 The rate of SICH per ECASS II was lower in our material, at 5.1% versus 7.0% in the SEDAN derivation cohort. The reason for this is not clear because important SICH risk factors, such as median age and blood glucose, were identical between ours and the Helsinki cohort, whereas stroke severity per NIHSS was higher in the SITS-ISTR. The SEDAN derivation cohort excluded 10.8% patients (119/1104) with basilar artery occlusion.2 These patients have a higher risk for SICH per ECASS II at 15.7%, according to previously reported data from the Helsinki group.16 During the study period, the SITS-ISTR lacked the possibility to register specific arterial occlusion sites. We were thus unable to exclude basilar artery occlusion patients, which may have influenced the risk factor profile for SICH per ECASS II compared with the Helsinki material, potentially affecting the predictive capability of the score in our material.
The past few years have seen a proliferation of prognostic scores in the field of stroke.17 They reflect our need for more precise instruments for prediction of complications, as well as prognostication of long-term functional outcome. The hemorrhage after thrombolysis and the multicenter stroke survey were the first SICH scores to be published.3.4 They both have undergone an independent validation attempt in the Stroke Acute Ischemic NXY-059 Trial I and II study cohorts, unable to confirm the performance of the scores, reporting c statistic values of 0.59 for both scores.18 The SEDAN has clinical advantages compared with these scores, in addition to the somewhat higher c statistic value of 0.66 in our external validation. It does not require waiting for a blood platelet count (needed in the multicentre stroke survey score) and does not include visible infarction of >33% of the middle cerebral artery territory on baseline CT as a risk factor (as in the HAT score), which is reasonable, because patients with such large manifest infarcts are frequently excluded from IV tPA treatment. One potential difficulty built into the SEDAN score is the need for including the hyperdense cerebral artery sign on baseline CT. The ability to detect this sign may vary with CT image slice thickness used, as well as the level of training and experience of the physician interpreting the baseline scan, which may lead to difficulty in calculating the SEDAN score.
The SITS-ISTR lacks data on platelet count and manifests infarct size at baseline. For this reason, we have been unable to perform a validation attempt for the HAT and multicentre stroke survey scores using our material. The SITS-SICH score designed by our group remains yet to be externally validated.1 We believe that an advantage of the SITS score over the SEDAN is that it is designed for the SICH per the SITS-MOST definition, which consistently shows a stronger association with mortality and poor functional outcome than the ECASS II and National Institute of Neurological Disorders and Stroke definitions.
We emphasize the need for prospective studies evaluating whether the use of SICH scores alters the risk versus benefit profile of IV tPA treatment, as well as external validation studies of existing and future scores. Until prospective studies are available to confirm their clinical use, none of the available scores should be regarded as ready for use as the basis for a decision to withhold treatment with IV thrombolysis in otherwise eligible patients. At present, although the scores may provide us with valuable information, clinical judgment remains the basis for treatment decisions in acute stroke care.
In this large database material of >36 000 patients, the SEDAN score predicts SICH per ECASS II less convincingly than previously reported. The predictive capability for SICH per SITS-MOST is lower still. The search for optimal predicting factors and algorithms for truly SICH continues. We believe that studies combining multimodal imaging and clinical factors to improve the predictability of SICH are warranted.
Scientific Committee of SITS International
Nils Wahlgren (chair), Antoni Dávalos, Gary A Ford, Martin Grond, Werner Hacke, Michael Hennerici, Markku Kaste, Kennedy R Lees, R Mikulik, Risto Roine, Turgut Tatlisumak, Danilo Toni, and K.S. Wang.
Scientific Committee of Fighting Stroke (Uppdrag Besegra Stroke)
Nils Wahlgren (chair), Niaz Ahmed, Maaret Castrén, Ulf Eriksson, Jonas Frisén, Ulf Hedin, Staffan Holmin, Åke Sjöholm, Mikael Svensson, Mia von Euler.
We thank all Safe Implementation of Treatments in Stroke (SITS)–International Stroke Thrombolysis Register (ISTR) investigators and their centers for their participation. We also pass on our thanks to all patients who participated in the SITS-ISTR. Uppsala Clinical Research center, Uppsala, Sweden, developed, maintained, and upgraded the software for the SITS register in close collaboration with SITS until September 2010. The present SITS registry is developed, maintained, and upgraded by Zitelab, Copenhagen, Denmark, in close collaboration with SITS. Drs Wahlgren and Ahmed coordinated the study. Dr Mazya performed the statistical analysis. Drs Mazya, Ahmed, and Wahlgren wrote the initial draft of the article. Dr Wahlgren was the Chairman of the SITS Scientific Committee. Drs Castillo, Jatuzis, and Kobayashi were national coordinators of leading recruiting countries. Dr Bovi was the coordinator for a leading recruiting center. All authors have read and commented on the first draft, with regard to interpretation of the data and editing of the article, and have seen and approved the final version. Drs Mazya, Wahlgren, and Ahmed have direct access to the original data and vouch for the accuracy and completeness of this report. The views expressed are those of the authors. Drs Mazya, Wahlgren, and Ahmed had full access to all data in this study and had the final responsibility for the preparation of this article and its submission for publication.
Sources of Funding
Safe Implementation of Treatments in Stroke (SITS)–International Stroke Thrombolysis Register (ISTR) is funded by an unrestricted grant from Boehringer Ingelheim, Ferrer, and by a grant from European Union Public Health Executive Authority. Financial support was also provided through the regional agreement on medical training and research (ALF) between Stockholm County Council and the Karolinska Institute. This study is a part of the Fighting Stroke Project (Uppdrag Besegra Stroke), supported by the Swedish Heart and Lung Foundation and Karolinska Institutet. The project is supported by funding from Friends of Karolinska Institutet, USA, and Johanniterorden.
Drs Mazya, Bovi, Castillo, and Kobayashi report no conflict of interest. Dr Jatuzis has received lecture fees from Boehringer Ingelheim. Dr Wahlgren has received expenses from Boehringer Ingelheim for his role as member of the Steering Committee in relation to the European Cooperative Acute Stroke Study (ECASS) III trial with alteplase and served as a consultant to Thrombogenics as chairman of the DSMB. Safe Implementation of Treatments in Stroke (SITS) International (chaired by Dr Wahlgren) received a grant from Boehringer Ingelheim and Ferrer for the SITS- Monitoring Study (MOST)/SITS–International Stroke Thrombolysis Register (ISTR). His institution has also received grant support toward administrative expenses for coordination of the ECASS III trial. Dr Wahlgren has also received lecture fees from Boehringer Ingelheim and Ferrer. Dr Ahmed is a senior researcher in SITS International, which receives a grant from Boehringer Ingelheim and Ferrer for the SITS-MOST/SITS-ISTR.
Steven C. Cramer, MD, MMSc, was guest editor for this article.
- Received January 28, 2013.
- Revision received February 23, 2013.
- Accepted March 13, 2013.
- © 2013 American Heart Association, Inc.
- Mazya M,
- Egido JA,
- Ford GA,
- Lees KR,
- Mikulik R,
- Toni D,
- et al
- Wahlgren N,
- Ahmed A,
- Eriksson N,
- Aichner F,
- Bluhmki E,
- Dávalos A,
- et al.
- Moons KG,
- Royston P,
- Vergouwe Y,
- Grobbee DE,
- Altman DG
- Royston P,
- Moons KG,
- Altman DG,
- Vergouwe Y
- Altman DG,
- Vergouwe Y,
- Royston P,
- Moons KG
- Moons KG,
- Altman DG,
- Vergouwe Y,
- Royston P
- Gumbinger C,
- Gruschka P,
- Böttinger M,
- Heerlein K,
- Barrows R,
- Hacke W,
- et al
- Sairanen T,
- Strbian D,
- Soinne L,
- Silvennoinen H,
- Salonen O,
- Artto V,
- et al