A Risk Score Based on Get With the Guidelines–Stroke Program Data Works in Patients With Acute Ischemic Stroke in China
Background and Purpose—There are few validated models for prediction of in-hospital mortality after acute ischemic stroke. In 2010, Smith et al developed and internally validated models for predicting in-hospital mortality based on Get With the Guidelines–Stroke program data. We demonstrate the applicability of this Get With the Guidelines risk model in Chinese patients.
Methods—The prognostic model was used to predict survival in 7015 patients with acute ischemic stroke from China National Stroke Registry data set. Model discrimination was quantified by calculating C statistic. To clarify the role of National Institutes of Health Stroke Scale (NIHSS), we also calculated the C statistics for NIHSS alone and for the model without NIHSS.
Results—The C statistic was 0.867 (95% CI, 0.839–0.895) through the Get With the Guidelines risk model, suggesting good discrimination in the China National Stroke Registry. The model without NIHSS produced significantly lower C statistic (0.735; 95% CI, 0.701–0.770; P<0.001), indicating the important role of NIHSS in the prediction of survival. Furthermore, a model with NIHSS alone also provided significant discrimination (C statistic, 0.847; 95% CI, 0.816–0.879). A plot of observed versus predicted mortality showed excellent model calibration in the external validation sample from the China National Stroke Registry.
Conclusions—The Get With the Guidelines risk model could correctly predict in-hospital mortality in Chinese patients with ischemic stroke. In addition, the NIHSS provides substantial incremental information on a patient’s short-term mortality risk and is the strongest predictor of mortality.
Predicting in-hospital mortality may help clinicians to identify the patients at high risk for poor outcome and aid clinical care. Most previous in-hospital mortality-predicting models, however, are either setting-specific or not easy to implement. To allow for an immediate and accurate prognosis based on a few simple variables, Smith et al1 recently developed and internally validated models for predicting in-hospital mortality using Get With the Guidelines–Stroke (GWTG-Stroke) program data in 2010. To demonstrate the accuracy and use of the GWTG risk model, the current study aimed to apply it to patients from the China National Stroke Registry (CNSR) to see if (1) the prognostic model adequate in different stroke populations; and (2) whether the National Institutes of Health Stroke Scale (NIHSS) plays a key role in predicting in-hospital mortality.
The design, rationale, and baseline information of CNSR has been described previously.2 In brief, CNSR was a nationwide prospective hospital-based cohort study of consecutive patients with stroke aged ≥18 years admitted to 132 hospitals within 14 days after the onset of symptoms between September 2007 and August 2008 in China. Detailed baseline data were abstracted prospectively using paper-based registry forms. The study was approved by the central Institutional Review Board at Beijing Tiantan Hospital. All patients or their designated relatives were informed about study participation, and informed written consent was obtained. We included the acute ischemic patients only for the present study.
Details of the selection of variables for the risk score based on GWTG-Stroke program data, data sources, eligibility criteria, imputation of variables with missing data, and the creation and conceptualization of the risk score have been published elsewhere.1
The main outcome of interest was in-hospital death.
The normality of all continuous variables was checked using the Shapiro-Wilk test. In the case of normality, Student t test was used; otherwise, the Kruskal-Wallis test was used. The differences between qualitative data were analyzed by χ2 test (with Yates correction or by Fisher exact test when needed). The performance of the GWTG risk model in the external validation sample was determined by generating C statistic and plot of observed versus predicted mortality with 10 deciles of predicted risk. The risk scoring system our analysis was based on has been published elsewhere.1 The fit of the model was assessed using Hosmer-Lemeshow goodness-of-fit χ2 test, and the model was regarded significant at P≥0.05. Pearson correlation coefficient was used to compare the observed and predicted mortality. In addition, the C statistics in the age-specified (median age) and sex-specified subgroups were also calculated. To evaluate the value of NIHSS in prediction of mortality, the C statistics for NIHSS alone and for the GWTG risk model without NIHSS were calculated.
All analyses were conducted using SAS 9.2.
Among 22 216 patients enrolled in CNSR, there were 12 415 patients with ischemic stroke who consented for follow-up. After excluding patients not from the first visit hospitals (2512) and those with missing data on discharge destination (2623), there were 7280 patients left. Furthermore, 265 patients were excluded because of missing data on NIHSS. Therefore, 7015 eligible patients were included in our analysis. There was no difference between patients with NIHSS and those without it (see the online-only Data Supplement I).
The patients in CNSR were quite different from those in GWTG in demographic characteristics and risk factors (online-only Data Supplement II). The Chinese patients were younger, had a higher proportion of men, and presented somewhat milder severity.
Based on the GWTG risk model, the C statistic of the external validation cohort from CNSR data set was 0.867 (95% CI, 0.839–0.895), which was significantly greater than that of the model without adjusting for NIHSS (0.735; 95% CI, 0.701–0.770; P<0.001). The C statistic of NIHSS alone, without adjusting any other predictors, was also very high (0.847; 95% CI, 0.816–0.879) and not significantly different from that of the GWTG risk model (P=0.370).
In addition, the GWTG risk model showed strong discrimination in both men and women (C statistic 0.856 versus 0.870) and in younger patients (<68 years, dichotomized at the median, C statistic 0.846) but weak discrimination in older patients (>68 years, C statistic 0.693).
A plot of observed versus predicted mortality rates showed excellent calibration (Figure; Pearson correlation coefficient 0.978; P<0.0001). The significance level of the Hosmer-Lemeshow test was 0.674.
Our results showed that the GWTG risk model is able to predict the in-hospital mortality accurately in the patients with ischemic stroke in China (C: 0.867); even the Chinese patients were significantly different in clinical baseline from those in the United States (mainly with younger median age and milder stroke severity). A risk model with NIHSS alone had almost the same C statistic (0.847), which indicated the strong relationship between stroke severity and short-term mortality. Additionally, the C statistic for the risk model without NIHSS substantially decreased to 0.735, further suggesting that the NIHSS score provides significant incremental information on a patient’s in-hospital mortality risk and is the strongest predictor of mortality. Our study results are in accordance with previous ones.1,3–5
To our knowledge, this is the first external validation study for the in-hospital mortality-predicting model by Smith et al through a different independent population from the nationwide stroke registry in China. In addition, our data set has more information on NIHSS than the original data set (96.4% versus 39.7%1), suggesting that the original derivation was not biased by the amount of missing NIHSS information.
There were limitations in this study. The study sites were selected from urban areas representing the institutes with better inpatient care than those in rural areas. Therefore, whether the GWTG risk model can apply to any stroke care institution merits further investigation.
Sources of Funding
This study was funded by the Ministry of Science and Technology and the Ministry of Health of the People’s Republic of China: National Science and Technology Major Project of China (2008ZX09312_008) and State Key Development Program of (for) Basic Research of China (2009CB521905).
↵* Drs Zhang and Liu share first authorship.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.112.669085/-/DC1.
- Received August 7, 2012.
- Revision received August 7, 2012.
- Accepted August 20, 2012.
- © 2012 American Heart Association, Inc.
- Smith EE,
- Shobha N,
- Dai D,
- Olson DM,
- Reeves MJ,
- Saver JL,
- et al
- Wang Y,
- Cui L,
- Ji X,
- Dong Q,
- Zeng J,
- Wang Y,
- et al
- Smith WS,
- Lev MH,
- English JD,
- Camargo EC,
- Chou M,
- Johnston SC,
- et al
- Weimar C,
- Konig IR,
- Kraywinkel K,
- Ziegler A,
- Diener HC
- Muir KW,
- Weir CJ,
- Murray GD,
- Povey C,
- Lees KR