National Institutes of Health Stroke Scale Item Profiles as Predictor of Patient Outcome
External Validation on Safe Implementation of Thrombolysis in Stroke–Monitoring Study Data
Background and Purpose—National Institutes of Health Stroke Scale (NIHSS) item profiles that were recently proposed and validated may prove useful for clinical prognostication and research studies. We aimed to validate the NIHSS item profiles in hyper-acute stroke patients who received thrombolysis treatment (tissue-type plasminogen activator).
Methods—We applied the latent class analysis probabilities of the profile membership generated from the derivation study onto NIHSS data from the Safe Implementation of Thrombolysis in Stroke–Monitoring Study (SITS-MOST). We separately considered NIHSS data collected within 3 hours and at ≈24 hours after stroke onset to obtain 2 sets of symptom groupings. The discrimination and calibration of both sets of symptom profiles were assessed from their association with outcomes. The outcome measures included modified Rankin Scale (mRS; using full distribution and dichotomized, mRS 0–1 or back to baseline) at day 90 and mortality by 90 days.
Results—We obtained data for 6843 patients. Ordinal analysis of mRS showed odds of better outcome across the profiles, for each set of symptom profiles, adjusted for age, sex, and prestroke mRS. Dichotomized outcomes mirrored the ordinal findings. There were significant differences in prognostic discrimination ability for the dichotomized outcome measures between the 2 sets of symptom profiles, with the latter set (ie, 24-hour symptom profiles) performing better.
Conclusions—The NIHSS item profiles are individually associated with functional outcome and mortality in acute stroke patients treated with tissue-type plasminogen activator. Considering profiles of NIHSS subscores rather than only the total score is informative for prognostication, particularly for assessments collected 24 hours after stroke onset.
National Institutes of Health Stroke Scale (NIHSS) item profiles that were recently proposed and validated may prove useful for clinical stroke prognostication and research studies.1,2 Rather than considering individual components of the NIHSS that contributed to a score to be equal, these NIHSS item profiles grouped the 15 individual attributes of NIHSS, using latent class analysis,3 into 6 symptom profiles that are associated with distinct functional outcome and mortality poststroke.1,2
The symptom profiles were initially generated on the Greater Cincinnati and Northern Stroke Study population using retrospectively derived NIHSS.1,4–6 The symptom profiles were then validated using prospectively obtained NIHSS acquired from a clinical trial cohort provided from the Virtual International Stroke Trials Archive (VISTA).2,7 Clinical trial patients are highly selected, potentially limiting generalizability of the symptom profiles in a hyper-acute stroke setting. Second, there is a potential for early intervention to modify the relationship between initial symptoms and final outcome, and these samples using assessments of NIHSS from very early in the course of stroke took no account of this.
The aim of thrombolysis treatment (tissue-type plasminogen activator [tPA]) in an acute stroke is to reperfuse brain tissue that is hypoperfused but still viable (ie, the penumbra). An NIHSS score in the hyper-acute setting, before tPA administration, may be a surrogate marker of the reversible and irreversible ischemic damage in the brain. Meanwhile, the 24-hour NIHSS following tPA may reflect a proxy indicator of an early recanalization, early neurological improvement, and the resultant ischemic damage.
We aimed to validate the previously published NIHSS item profiles1,2 as predictors of patient outcome in a hyper-acute stroke cohort who received thrombolysis treatment. We also planned to contrast the relation of NIHSS item profiles collected either before or 24 hours after thrombolysis treatment with functional outcome and mortality.
We retrospectively analyzed data that had been prospectively collected within the Safe Implementation of Thrombolysis in Stroke–Monitoring Study (SITS-MOST).8 SITS-MOST was a prospective, open-label, postmarketing study of tPA administration within 3 hours of ischemic stroke onset, performed in 285 centers across 14 European countries, between December 2002 and April 2006.8 The study was established as a cohort of the SITS International Stroke Thrombolysis Register (SITS-ISTR), an interactive, internet-based, academic-driven, thrombolysis registry.9 The duration of follow-up was 90 days post stroke.
Participants and Variables
We obtained demographics and outcome information: individual components of the baseline and 24-hour NIHSS recordings, age, sex, comorbidities, occurrence of adverse events, as well as serious adverse events, modified Rankin scale (mRS) at day 90, and mortality.
We applied the probabilities of profile membership generated by Sucharew et al1 to the baseline NIHSS data collected from hyper-acute stroke patients before receiving thrombolysis treatment. These were labeled baseline–NIHSS item profiles. Descriptive statistics were generated for the entire cohort and for each profile separately. We described mean (SD) or median (interquartile range) for continuous variables and count (%) for categorical variables.
The discrimination and calibration of the baseline–NIHSS item profiles were assessed from the association of the symptom profiles with outcome. Outcome measures assessed at 90 days included the full distribution of the mRS, good outcome (mRS 0–1 or back to baseline), mortality, and time-to-death. Odd ratios and 95% confidence intervals of the ordinal outcome measures were obtained using ordinal logistic regression, and the associated P values were calculated using the Cochrane–Mantel–Haenszel test. Dichotomized outcome measures were investigated using binary logistic regression. Time-to-death was examined using Kaplan–Meir survival curves and Cox proportional hazards model. Analyses were adjusted for age,10,11 sex, and prestroke mRS.
We then applied the same probabilities of profile membership1 to the NIHSS data collected 24 hours after administration of intravenous tPA to form the 24 hour–NIHSS item profiles. We calculated and compared descriptive statistics and analyzed the association of the 24 hour–NIHSS item profiles with the outcome measures as above. The symptom profiles’ ability to discriminate between dichotomized outcomes was evaluated using the area under the receiver operating curve technique. Calibration was assessed with the Hosmer–Lesmeshow goodness-of-fit test.
A sensitivity analysis was also performed to illustrate the performance of the profiles among patients with the same baseline overall NIHSS.
All analyses were undertaken using SAS version 9.3 (SAS Institute Inc, Cary, NC).
We obtained individual patients’ data from the SITS-MOST study consisting of 6843 patients with hyper-acute ischemic stroke who received intravenous thrombolysis treatment. The cohort had median age of 68 (interquartile range, 59–75) and baseline NIHSS (before tPA administration) of 12 (8–17). Baseline characteristics of the cohort are shown in Table 1.
By applying the probabilities of profile membership to the SITS-MOST cohort, we replicated the 6 profiles identified by Sucharew et al1 (profiles A–F, denoted baseline–NIHSS item profiles). The NIHSS item profiles ranged from most severe profile A to mild profile F. Profiles A and F had median baseline NIHSS of 18 (21–16) and 4 (3–6), respectively. The boxplots of the profiles are shown in Figure I in the online-only Data Supplement. The individual baseline–NIHSS items that contributed to each profile are listed in Table 2. Approximately 90% of patients had good premorbid function (ie, pre–stroke mRS of 0–1). The profiles that incorporated higher baseline NIHSS derived from cohorts of patients who were generally older and with comorbidity, such as atrial fibrillation, chronic heart failure, or hypertension.
There was no notable difference in the onset to treatment delay nor in the dosage of tPA across the profiles (Table 1). Early ischemic changes on baseline neuroimaging were more commonly described among patients with greater stroke severity, that is, in profiles A and B. The profiles that fell between the extremes of stroke severity, that is, profiles C, D, and E, had median baseline NIHSS of 7 (6–9), 11 (8–14), and 7 (5–9), respectively. Profile C captured dominant hemisphere strokes with dysphasia and dysarthria, whereas profile D included the same hemisphere strokes with abnormal motor function of the right side and dysarthria. Profile E represents nondominant hemisphere stroke with abnormal motor function of the left side (Table 2).
Ordinal analysis of mRS at day 90, adjusted for age, sex, and prestroke mRS, confirmed greater odds of better outcome across all profiles, B–F, when compared against profile A (Table 3; Figure II in the online-only Data Supplement). The dichotomized outcomes (ie, mRS 0–1 or back to baseline and mortality) and the overall survival analysis at 90 days mirrored the findings from the ordinal analysis (Table 3; Figure 1).
There were statistically significant differences for good outcome, mortality, and survival rates when comparing profile C with profile D, with profile D consistently associated with worse outcomes than profile C (Table 3; Figure 1). Profiles C and E, which shared a common median baseline NIHSS, did not differ in terms of mortality and survival rates when compared with each other after adjustment (Table 3).
To compare the performance of symptom profiles generated from baseline NIHSS with the 24-hour NIHSS, we applied the probabilities of profile membership identified by Sucharew et al1 onto 24-hour NIHSS data to generate 6 distinct 24 hour–NIHSS item profiles (profiles a–f). The baseline characteristics of the cohort by 24 hour–NIHSS item profiles are shown in Table I and Figure II in the online-only Data Supplement. The median 24-hour NIHSS for the whole cohort was 6 (1–12).
The performance of the 24 hour–NIHSS item profiles mirrored the findings from the baseline–NIHSS item profiles for the ordinal analysis of mRS using the full distribution and the dichotomized outcome measures, but with higher odds ratios, after adjustment for age, sex, and prestroke mRS (Table II and Figure III in the online-only Data Supplement). There were clear distinctions in the survival curves between profiles a and b versus the remaining profiles. Meanwhile, c, e, and f shared common survival outcomes when compared with each other (Figure 2).
There were significant differences in discrimination ability for the dichotomized outcomes between the baseline- and 24 hour–NIHSS item profiles. The latter set of symptom profiles had better area under the receiver operating curve for the dichotomized outcome measures. The performances of both sets of NIHSS item profiles were detailed in the online-only Data Supplement.
To illustrate the performance of the profiles among patients with the same overall NIHSS, a sensitivity analysis was performed including only those with baseline NIHSS score of 10, as an example. After adjusting for age, sex, and prestroke mRS, the profiles had statistically different 90-day mortality and time-to-event survival rates (Figure 3; Table III in the online-only Data Supplement). There were no significant differences across the profiles for ordinal mRS and good outcome at 90 days (Table III in the online-only Data Supplement). Among patients with NIHSS score of 10 at baseline, those in profile D were associated with 89% reduction in risk of 90-day mortality when compared with those in profile A. Similarly for overall survival, compared with profile A, profile D was associated with 86% reduction in risk of death.
Profiles of component NIHSS items show distinct patterns of stroke deficit that cannot be identified by NIHSS alone. By differentiating between subjects with the same total NIHSS but contrasting clinical symptoms, these profiles offer predictions of prognosis that are more clinically informative than total NIHSS alone. This conclusion arose from retrospectively derived NIHSS scores and appeared to be valid within a selective clinical trial population but has now been confirmed in a more clinically useful population, the population of patients who proceed to acute intervention with intravenous tPA.
Traditionally, the use of overall NIHSS score considers the individual components that contributed to the score to be equal. Consider a 70-year-old man who had an acute stroke with NIHSS of 10. The interpretation of the score is the same whether it indicates severe loss of consciousness or motor weakness. Reporting this patient’s condition using the total NIHSS and the symptom profile may be more informative to clinicians, for example, NIHSS 10A may imply a different clinical outcome compared with NIHSS 10D (Figure 3; Table III in the online-only Data Supplement). This is prognostically more helpful if the symptom profile remains unchanged in the following 24 hours.
Although our analysis showed that symptom profiles B to F had better odds of favorable outcome when compared with profile A, the odds of improvement did not increase in a stepwise manner across the profiles, in contrast to the previous reports.1,2 In particular, the previous studies showed 2 symptom profiles that possessed comparable median NIHSS (ie, profiles C and D) had widely disparate outcomes, profile C being associated with worse outcome than profile D.1,2 Whereas here, profile D had worse outcome than profile C. Of note, patients in profiles C and D each represent patients with dominant hemisphere stroke but patients in profile C predominantly have language deficit, whereas patients in profile D have right-sided paresis. We may only speculate about the reason for this change in findings. Perhaps clinical assessment for the NIHSS score is cursory and inaccurate when conducted under time pressure during assessment for potential thrombolysis treatment, possibly by less experienced clinicians. In this analysis, profile C which only contained 466 patients may have underpowered some determinant variables like aphasia. Alternatively, perhaps certain neurological deficits are genuinely more amenable to improvement after thrombolysis. In addition, the cohort in which the symptom profiles were initially derived was different from the patients eligible for tPA within SITS-MOST registry. The original derivation cohort from Greater Cincinnati and Northern Stroke Study population has generally milder stroke but more comorbidities and older age. Although some SITS centers will have been less experienced, they had access to the same training and certification for NIHSS as other trialists, and staff in all centers are frequently replaced. We consider it unlikely that staff in SITS centers were generally less competent at neurological assessment than staff in the derivation cohort centers. Even so, the differences in the prognostic discrimination between derivation and validation cohorts may be partly but not exclusively because of data collection (ie, retrospective versus prospective). We have shown that the performance of the symptom profiles using prospective data acquired from clinical trials was similar to the ones of the derivation study.2 The validation analysis using SITS-MOST data may have captured the effect of early intervention that modifies the relationship between initial symptoms and final outcome.
In addition, the nonlinear distribution of patients that concentrated in profiles B and D may reflect the standard of the practice of clinicians during the period, that is, they are more comfortable to treat a clear-cut stroke in profiles B and D when compared with profiles A, C, E, and F. Recall that patients in profile B represent severe stroke but with some decreased level of consciousness, left hemiparesis, and dysarthria; whereas patients in profile D predominantly have stroke with right hemiparesis and dysarthria. Profile A includes element of markedly reduced conscious level. Profile C has element of aphasia. Profiles E and F may have been regarded as too mild.
As for timing, the NIHSS profiles measured at 24-hour after stroke onset were more strongly associated with outcome than those assessed within the first 3 hours before thrombolysis. The 24-hour symptom profiles will exclude the impact of any reversible ischemic stunning that resolves through reperfusion and for this reason may better predict the outcome at 90 days. A stronger correlation has previously been reported for 90-day outcome with total NIHSS scores recorded at 24 hours than within 4.5 hours from stroke onset.12 Likewise, neuroimaging studies have shown greater mismatch of NIHSS with ischemic core volume at baseline than with ischemic volume in the following 24-hour post-tPA.13,14 Therefore, the 24-hour symptom profiles may be a better pointer to long-term prognosis.
The strengths of this analysis include the large sample size recruited from the international SITS-MOST registry and the prospective nature of the NIHSS data collected, both before and 24 hours after thrombolysis treatment. Having these 2 data collections allowed us to explore the discrimination and calibration power of the NIHSS item profiles generated from 2 time points potentially extending their prognostic value and generalizability.
Our analysis also has limitations. Our cohort remains highly selected: these are only patients who were eligible for thrombolysis treatment according to the European marketing authorization of the day. The patients were aged between 18 and 80 years and had a maximum onset-to-treatment time of 3 hours. These eligibility criteria may have resulted to the proportion of patients observed, for example, restriction to the number of elderly patients with greater prevalence of atrial fibrillation. In addition, stroke tPA treatment is now offered more widely, being recommended without any upper age limit and with a maximum onset-to-treatment time of 4.5 hours.15,16 Limitations of the ordinal outcome analysis that we chose as our principal outcome measure, and of the latent class analysis methods to generate the NIHSS item profiles, have been discussed elsewhere.2
In conclusion, our analysis demonstrates the reliability of the NIHSS item profiles to predict functional and mortality outcomes in acute stroke patients treated with thrombolysis treatment. The 24-hour symptoms profiles have better associations with outcomes than profiles of symptoms at presentation. Considering the NIHSS not just as a total score but in terms of its components enhances its prognostic value and may be useful in clinical and research settings.
Appendix: SITS-MOST Steering Committee
N. Wahlgren, N. Ahmed, A. Dávalos, G.A. Ford, M. Grond, W. Hacke, M.G. Hennerici, M. Kaste, S. Kuelkens, V. Larrue, K.R. Lees, R.O. Roine, L. Soinne, D. Toni, G. Vanhooren.
We thank the Safe Implementation of Thrombolysis in Stroke–Monitoring Study (SITS-MOST) Steering Committee for providing access to the data. Dr Lees supervised the project. Dr Abdul-Rahim conducted the analyses and drafted the initial manuscript. Drs Fulton and Sucharew provided statistical guidance and ran the latent class analysis. Drs Abdul-Rahim and Lees were involved in reviewing and reporting of the work. All authors provided critical revision of the manuscript for important intellectual content and approved the final version. Members of the SITS-MOST Steering Committee approved the study plan in advance and approved the final manuscript.
Dr Khatri received research support from Genentech and Penumbra (Principal Investigator of A Study of the Efficacy and Safety of Activase [Alteplase] in Patients With Mild Stroke [PRISMS] and Assess the Penumbra System in the Treatment of Acute Stroke [THERAPY] trials) and Data and Safety Monitoring Board (DSMB) member for Biogen. Dr Khatri also received honoraria from UpToDate. Dr Kleindorfer received research grant from the National Institute of Neurological Disorders and Stroke (NINDS) for her research activities. None of these relate to the content of this article. The other authors report no conflicts.
*A list of all SITS-MOST Steering Committee members is given in the Appendix.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.115.010380/-/DC1.
- Received July 6, 2015.
- Revision received July 27, 2015.
- Accepted July 29, 2015.
- © 2015 American Heart Association, Inc.
- Sucharew H,
- Khoury J,
- Moomaw CJ,
- Alwell K,
- Kissela BM,
- Belagaje S,
- et al
- Abdul-Rahim AH,
- Fulton RL,
- Sucharew H,
- Kleindorfer D,
- Khatri P,
- Broderick JP,
- et al
- 3.↵SAS procedures for latent class analysis & latent transition analysis. The Methodology Center of The Pennsylvania State University. Website. http://methodology.psu.edu/downloads/proclcalta. Accessed April 27, 2015.
- Broderick J,
- Brett T,
- Kothari R,
- Miller R,
- Khoury J,
- Pancioli A,
- et al
- Kleindorfer D,
- Broderick J,
- Khoury J,
- Flaherty M,
- Woo D,
- Alwell K,
- et al
- Lindsell CJ,
- Alwell K,
- Moomaw CJ,
- Kleindorfer DO,
- Woo D,
- Flaherty ML,
- et al
- Ali M,
- Bath P,
- Brady M,
- Davis S,
- Diener HC,
- Donnan G,
- et al
- Wahlgren N,
- Ahmed N,
- Dávalos A,
- Ford GA,
- Grond M,
- Hacke W,
- et al
- 9.↵Safe Implementation of Treatments in Stroke (SITS). The Safe Implementation of Treatments in Stroke (SITS) Website. https://sitsinternational.org. Accessed April 27, 2015.
- Mishra NK,
- Lyden P,
- Grotta JC,
- Lees KR
- Wardlaw JM,
- Murray V,
- Berge E,
- Del Zippo GJ
- Strbian D,
- Saarinen T,
- Rantanen K,
- Piironen K,
- Atula S,
- Tatlisumak T,
- et al
- Kharitonova T,
- Mikulik R,
- Roine RO,
- Soinne L,
- Ahmed N,
- Wahlgren N
- Jauch EC,
- Saver JL,
- Adams HP Jr,
- Bruno A,
- Connors JJ,
- Demaerschalk BM,
- et al