Validating a Predictive Model of Acute Advanced Imaging Biomarkers in Ischemic Stroke
Background and Purpose—Advanced imaging to identify tissue pathophysiology may provide more accurate prognostication than the clinical measures used currently in stroke. This study aimed to derive and validate a predictive model for functional outcome based on acute clinical and advanced imaging measures.
Methods—A database of prospectively collected sub-4.5 hour patients with ischemic stroke being assessed for thrombolysis from 5 centers who had computed tomographic perfusion and computed tomographic angiography before a treatment decision was assessed. Individual variable cut points were derived from a classification and regression tree analysis. The optimal cut points for each assessment variable were then used in a backward logic regression to predict modified Rankin scale (mRS) score of 0 to 1 and 5 to 6. The variables remaining in the models were then assessed using a receiver operating characteristic curve analysis.
Results—Overall, 1519 patients were included in the study, 635 in the derivation cohort and 884 in the validation cohort. The model was highly accurate at predicting mRS score of 0 to 1 in all patients considered for thrombolysis therapy (area under the curve [AUC] 0.91), those who were treated (AUC 0.88) and those with recanalization (AUC 0.89). Next, the model was highly accurate at predicting mRS score of 5 to 6 in all patients considered for thrombolysis therapy (AUC 0.91), those who were treated (0.89) and those with recanalization (AUC 0.91). The odds ratio of thrombolysed patients who met the model criteria achieving mRS score of 0 to 1 was 17.89 (4.59–36.35, P<0.001) and for mRS score of 5 to 6 was 8.23 (2.57–26.97, P<0.001).
Conclusions—This study has derived and validated a highly accurate model at predicting patient outcome after ischemic stroke.
Reliably predicting a patient’s functional outcome in response to therapy is a critically important requirement of patient management. Ischemic stroke poses many challenges for outcome prediction because of patient and pathophysiological heterogeneity. In addition, effective treatment with reperfusion therapy, be it recombinant tissue-type plasminogen activator or intra-arterial therapy, can also dramatically change the outcome of patients,1,2 particularly where alteplase therapy results in a remarkably low rate of recanalization.1 Advanced acute imaging with computed tomography (CT), including perfusion (CTP) and angiography (CTA), offers insights into the disease pathophysiology and patient suitability for reperfusion therapy. Acute CTP3 has been shown to predict patients who will benefit from reperfusion therapy4 and those who may be harmed because of hemorrhagic transformation.5 In addition, advanced imaging can identify patients with a low probability of benefit from intravenous reperfusion therapy because their ischemic lesion is too severe or there is no tissue to salvage.4 However, there is currently a lack of specific rules to prognosticate in individual patients based on their advanced imaging profile. Mismatch criteria have been used for magnetic resonance imaging–based intravenous thrombolysis selection in some clinical trials, such as DEFUSE2 (Diffusion Weighted Imaging Evaluation for Understanding Stroke Evolution Study-2)6 and EPITHET (Echoplanar Imaging Thrombolytic Evaluation Trial).7 However, these criteria were largely empirically derived from limited cohorts and using earlier generation scanners and software. We hypothesized that in individual patients, advanced imaging profiles will accurately predict 3-month functional outcome. In this study, we aimed to derive and validate a predictive model for functional outcome based on acute clinical and advanced imaging measures in 3 groups of patients: (1) all patients considered for recombinant tissue-type plasminogen activator therapy, eventually treated or not; (2) those receiving recombinant tissue-type plasminogen activator; and (3) those showing 24-hour recanalization.
Consecutive acute ischemic stroke patients presenting to hospital within 4.5 hours of symptom onset at 5 centers ( John Hunter Hospital;  Gosford Hospital, NSW, Australia;  Huashan Hospital, Shanghai;  the Second Affiliated Hospital of Zhejiang University, Hangzhou, China; and  Sunnybrook Health Science Centre, Toronto, Canada) between 2011 and 2014 were prospectively recruited for the INSPIRE (International Stroke Perfusion Imaging Registry). As part of this study, patients underwent baseline multimodal CT imaging with noncontrast CT, CTP, and follow-up imaging with magnetic resonance imaging at 24 hours post stroke. Clinical stroke severity was assessed at the 2 imaging time points using the National Institutes of Health Stroke Scale (NIHSS). Eligible patients were treated with intravenous thrombolysis according to local guidelines and the clinical judgment of the treating physician. The modified Rankin scale (mRS) was assessed 90 days after stroke. Written informed consent was obtained from all participants, and the INSPIRE study was approved by the local ethics committees in accordance with Australian NHMRC guidelines. Endovascular procedures were not captured in the INSPIRE database.
Acute Multimodal CT Protocol
Participants recruited at John Hunter Hospital were scanned using a Toshiba Aquilion 320-slice CT scanner (Toshiba Medical Systems; Tokyo, Japan). A total of 19 acquisitions occurred in 60 s. Forty milliliter of contrast agent (Ultravist 370; Bayer HealthCare; Berlin, Germany) was injected at 6 mL/s, followed by 30 mL of saline. Participants recruited at Gosford Hospital were scanned using a 64 detector GE lightspeed (GE Healthcare, Waukesha, WI). A total of 19 acquisitions occurred in 54 s. Forty-five milliliter of contrast agent (Ultravist 370; Bayer HealthCare; Berlin, Germany) was injected at 6 mL/s. Participants recruited at Huashan Hospital were scanned using a Brilliance iCT 126-slice CT scanner (Philips Medical Systems, Cleveland, Ohio). A total of 23 acquisitions occurred in 60 s. Forty milliliter of the same contrast was injected at 5 mL/s, followed by 20 mL of saline. Participants recruited at the Second Affiliated Hospital of Zhejiang University were scanned using a Definition Flash duel source CT scanner (Siemens, Erlangen, Germany) with 64 detectors. A total of 10 acquisitions occurred in 60 s. Fifteen milliliter of contrast was injected at 4mL/s followed by 20 mL of saline. Participants recruited at Sunnybrook Health Sciences Center underwent an acute CT with a 64-row CT scanner (GE Healthcare). Baseline and 24-hour CT angiogram parameters were as follows: 0.7 mL/kg iodinated contrast agent up to a maximum of 90 mL (Omnipaque 300 mg iodine/mL; GE Healthcare, Piscataway, NJ); 5- to 10-s delay; 120 kVp; 270 mA; rotation, 1 s; 1.25-mm thick sections; table speed, 20.62 mm per rotation. Biphasic CTP protocol from basal ganglia to the lateral ventricles: 80 kVp; 150 mA: collimation, 8×5 mm; rotation, 1 s for 45 s followed by 6 further acquisitions 15 s apart for a total of 135 s. Iodinated contrast agent 0.5 mL/kg (maximum 50 mL) at 4 mL/s was administered 5 s before start of the sequence. All acute perfusion imaging acquired 60 mm or more of coverage.
Twenty-Four-Hour Imaging Protocol
As close as possible to 24 hours after acute imaging, all patients, regardless of treatment, underwent a stroke magnetic resonance imaging protocol on a 1.5-T or 3-T scanner (Siemens Avanto or Verio). The MR protocol included: diffusion-weighted imagine, perfusion-weighted imaging, MR time-of-flight angiography, and fluid-attenuated inversion recovery imaging. For those with a contraindication to magnetic resonance imaging, repeat noncontrast CT and CTP were performed using the above protocols.
CTP Analysis and Classification of Patients
All perfusion imaging was post processed on commercial software MIStar (Apollo Medical Imaging Technology, Melbourne, Australia). Acute perfusion imaging was processed using single value deconvolution with delay and dispersion correction.8 Previously validated thresholds were applied to measure the volume of the acute perfusion lesion (relative delay time [DT] >3 s) and acute infarct core (relative cerebral blood flow <30%).9 Penumbral volume was calculated from the volume of the perfusion lesion (DT threshold >3 s) minus the volume of the infarct core (relative cerebral blood flow threshold <30% within the DT >3 s lesion), the volume of severely hypoperfused tissue (DT >6 s) was also recorded for hemorrhage prediction.3
All acute CTA scans were assessed for occlusion severity blind to CTP. The scoring system used was defined as (1) normal, (2) anterograde flow or partial occlusion, or (3) complete occlusion.4 Collateral vessel flow was assed using the Mitef scale.10 Collateral ratings and vessel occlusion status on baseline CTA were performed by 2 stroke neurologists, with any disagreement resolved by consensus with a third neurologist.
Statistical analyses were programmed using SAS v9.4 (SAS Institute, Cary, NC), Stata v13.0 (StataCorp Ltd, College Station, TX), and R (R Foundation for Statistical Computing, Vienna, Austria). Analysis was undertaken to predict clinical outcomes in patients for both a derivation and validation cohort. The derivation cohort was composed of patients from a single site (John Hunter Hospital, Newcastle, Australia), and the validation cohort was composed of patients from 4 sites (Gosford Hospital, NSW, Australia; Huashan Hospital, Shanghai; the Second Affiliated Hospital of Zhejiang University, Hangzhou, China; and Sunnybrook Health Science Center, Toronto, Canada). Patients were divided into (1) all patients being considered for thrombolysis therapy, including those who did and those who did not receive thrombolysis; (2) only patients who received thrombolysis; and (3) patients who received thrombolysis and showed complete recanalization at 24 hours on advanced imaging.
Descriptive statistics were used to characterize all cohorts of patients. Continuous variables were described using means and SDs, plus medians and the interquartile range. Tests for group differences in means were performed using t tests. For medians, group differences were assessed using the Kruskal–Wallis (nonparametric) test. Categorical variables were described using frequencies, with group differences (deviation from independence) tested using χ2 statistics.
A Classification and Regression Tree (CART) analysis was used with variables having previously been shown to predict clinical outcome: acute ischemic core volume, acute penumbra volume, acute volume of severe hypoperfusion (DT >6 s, a marker of hemorrhage and poor outcome), Mismatch ratio, acute occlusion severity (complete compared with partial/none), collateral circulation grade (good compared with reduced/none), acute NIHSS, and age and time to scan on each variable separately to determine the ideal cut points at predicting the 3 prespecified outcome measures.
Once the optimal cut points to predict the prespecified outcome measures (mRS score of 0–1, 0–2, and 5–6) were determined from the first CART analysis, a multivariable logistic model was fitted using the full set of predictors for each sample outcome arm (all patients, those treated with thrombolysis and those with recanalization on 24-hour imaging) using the 3 outcome groups (mRS score of 0–1, 0–2, and 5–6). To identify the most parsimonious predictive model, variables not associated with the outcome at P<0.1 and whose removal had little impact on model fit were removed. Once the optimized cut points were identified using the CART analysis, a receiver operating characteristic curves analysis was performed to calculate the area under the curve (AUC) with its 95% confidence interval for each model.
A multivariable logistic regression and AUC analysis were performed on the independent validation cohort with the variables and cut-off points defined by the derivation cohort CART analysis. The AUC values were compared statistically between the derivation and the validation cohorts using the Handly method.11
During the study period, the INSPIRE registry collected 1519 patients who underwent acute CTP imaging within 4.5 hours of symptom onset and who were clinically eligible for intravenous thrombolysis. A derivation cohort was identified from John Hospital consisting of 635 patients, 366 (58%) of which were treated with intravenous thrombolysis, based on combined clinical, noncontrast CT and CTP criteria (Table 1). The validation cohort consisted of 884 patients with 582 (66%) being treated with intravenous thrombolysis. Reasons for nontreatment relate to identification of comorbidities during acute assessment which were not immediately apparent, identification of extensive early ischemic change on baseline noncontract CT imaging or patient rapidly resolving.
A CART analysis was performed to determine the optimal cut points at determining excellent, good, or poor patient outcome (mRS score of 0–1, 0–2, and 5–6; Table 2). Multivariable logistic regression using the CART to derive optimal cut points was performed on the 3 outcome targets (mRS score of 0–1, 0–2, and 5–6; Table 3) in the 3 patient groups (all patients being considered for thrombolysis therapy including those who were thrombolised and those who were not, thrombolysed patients only and patients showing recanalization at 24 hours after thrombolysis). For the prediction of mRS score of 0 to 1 in all patients from the derivation cohort, the variables that remained in the multivariable regression model were core ≤25 mL, penumbra ≤20 mL, and baseline occlusion status (complete compared with partial/none). The odds ratio (OR) of patients achieving an mRS score of 0 to 1 who met these model criteria were 17.89 (4.59–36.35, P<0.001). The AUC of this model was 0.947 (95% CI, 0.916–0.973). For patients who received thrombolysis, the final multivariable regression model included core ≤25 mL, collateral flow and baseline occlusion for being able to predict mRS score of 0 to 1. The OR of thrombolysis patients achieving mRS score of 0 to 1 who met the model criteria was 14.23 (5.86–34.55, P<0.001). The AUC of the model with only thrombolysis patients was 0.885 (95% CI, 0.85–0.922) at predicting mRS score of 0 to 1. Finally, for patients who showed recanalization at 24 hours to achieve mRS score of 0 to 1, the final multivariable regression model included core ≤25 mL, DT6≤30 mL, and collateral flow. The OR of recanalized patients achieving mRS score of 0 to 1 who met the model criteria was 75.29 (14.4–393.70, P<0.001). The AUC of the model with only thrombolysis patients achieving mRS score of 0 to 1 was 0.892 (95% CI, 0.837–0.947).
For all patients in the derivation cohort (including patients who were thrombolised and those who were not thrombolised) who achieved mRS score of 0 to 2, the variables that remained in the multivariable regression model were core ≤25 mL, time to onset ≤120 minutes, collateral flow, and baseline occlusion. The OR of patients achieving mRS score of 0 to 2 who met these model criteria were 10.29 (1.48–22.87, P<0.001). The AUC of this model was 0.919 (95% CI, 0.985–0.945). For patients who received thrombolysis and who had an outcome of mRS score of 0 to 2, the final multivariable regression model included core ≤25 mL, penumbra ≤20 mL, and DT6≤30 mL. The OR of thrombolysis patients achieving mRS score of 0 to 2 who met the model criteria was 11.31 (5.54–23.06, P<0.001). The AUC of this model was 0.870 (95% CI, 0.836–0.904). Finally, for predicting mRS score of 0 to 2 in patients who showed recanalization at 24 hours, the final multivariable regression model included core ≤25 mL and collateral flow. The OR of recanalized patients achieving mRS score of 0 to 2 who met the model criteria was 29.05 (8.87–95.09, P<0.001). The AUC of the model was 0.921 (95% CI, 866–0.976).
For all patients in the derivation cohort (including patients who were thrombolised and those who were not thrombolised), the variables in the model to predict mRS score of 5 to 6, were core ≥60 mL, penumbra ≥45 mL, and collateral flow. The OR of patients being mRS score of 5 to 6 who met these model criteria was 8.23 (2.57–26.97, P<0.001). The AUC of this model was 0.915 (95% CI, 872–0.96). For patients who received thrombolysis, the final multivariable regression model included core ≥60 mL, penumbra ≥45 mL, DT6≥50 mL, time to onset >120 min, and collateral flow. The OR of thrombolysis only patients being mRS score of 5 to 6 who met the model criteria was 11.14 (4.55–27.29, P<0.001); the AUC was 0.894 (95% CI, 0.855–0.934). Finally, for patients who showed recanalization at 24 hours to be mRS score of 5 to 6, the final multivariable regression model included core ≥60 mL, penumbra ≥45 mL, and collateral flow. The OR of recanalized patients being mRS score of 5 to 6 who met the model criteria was 8.32 (2.57–26.97, P<0.001); the AUC was 0.915 (95% CI, 0.872–0.96).
The validation cohort had similar results when applying the optimal variables from the CART and backward regression models from the derivation cohort (Table 4). Notably, the values in the thrombolysis and recanalized cohorts remained extremely robust, particularly for the prediction of excellent or good outcome. Finally, there was no statically significant variation in AUC from the derivation cohort for the validation cohort in AUC for any group (P>0.05, Table 4).
We present a large scale analysis of clinical and advanced imaging profiles that predict favorable or poor response to intravenous thrombolysis in patients with ischemic stroke and have identified a variety of cut points using validated perfusion thresholds to show that it is possible to accurately (AUC ≈90) predict patient outcome and response to thrombolysis. There was a clear “sweet spot” of baseline ischemic core volume of <25 mL predicted good clinical outcomes for both the mRS 0 to 1 and 0 to 2 models. Similarly, when the acute ischemic core volume was >60 mL, it was included as a predictor of poor outcome in a high proportion of patients when treated with intravenous thrombolysis. The most powerful predictor for patient prognostication was ischemic core volume, which was highly related to patient outcome in all tested models. This analysis also confirms several previous studies, which have also illustrated the importance of advanced imaging profiles in identifying responders to intravenous thrombolysis.2,5,12 Although the effect of earlier time to thrombolytic treatment, age, and baseline NIHSS on outcome is indisputable from the pooled randomized control trial analyses,13 our data suggest that infarct core volume and other advanced imaging variables are likely to be several magnitudes greater in influence on patient outcome.
This study compared a derivation cohort and a separate validation cohort from different comprehensive stroke care hospitals to observe if there was variation from a pooled or single dataset. The derivation cohort results were successfully replicated in the similar sized validation cohort, albeit with slightly larger variation. This may reflect, in part, the heterogeneity of different patient healthcare systems as this project combined data from Australia, Canada, and China. Alternatively, sites may have attempted to interpret the CTP data acutely and altered their treatment decision-making, despite standardized automated imaging post processing not being available to clinicians during the study. This is the strength of the presented data, as despite the significant variation between international healthcare settings and between clinicians, the use of advanced imaging for acute ischemic stroke treatment decision-making remained a robust predictor of patient outcome.
The imaging-based parameters were the dominant prognostic variables. Time from symptom onset to imaging dropped out of all but one model in the present analysis (for the prediction of poor outcome in thrombolysed patients, in which was onset-to-imaging >120 minutes was retained). This finding strongly suggests that imaging-based selection provides the clinician with a more robust and predictive method in identifying reperfusion therapy eligibility. Although the effect of earlier time to thrombolytic treatment on outcome is indisputable from the pooled analyses of trials using conventional patient selection methods, it is a relatively weak predictor. Our data indicates that advanced CT imaging provides such as ischemic core volume provides several magnitudes greater precision in identifying the likelihood of a beneficial recombinant tissue-type plasminogen activator treatment response. Therefore, it is important to consider the effects of avoiding patient assessment with advanced imaging to save on time to treatment. Of potentially greater importance, is the ability for clinicians to identify a high likelihood of intravenous thrombolysis futility. Our analysis provides a strong suggestion that the use of an ischemic core volume of >60 mL, a total ischemic lesion volume of >105 mL, at DT6 lesion volume of >50 mL (a marker of symptomatic intracranial hemorrhage) and a onset to imaging time of greater than 2 hours indicates a high probability of an unacceptably poor outcome of thrombolysis treatment. Our data highlight the potential clinical value of advanced CT imaging in accurate prognostication and rational therapeutic decision-making.
One important limitation of our study is the observational design as a source of data collection and the underpinning limitations in pragmatic clinical practice. Importantly, this was an analysis using only intravenous therapy with alteplase, which has only a modest rate of reperfusion (40%). The thresholds we have described are unlikely to be predictive of outcome in patients treated with endovascular reperfusion strategies with higher reperfusion rates and earlier or known time of effective reperfusion. However, we have shown an extremely strong predictive model for recanalized patients, based on small core, small severe perfusion lesion, and good collaterals. Existing studies of endovascular reperfusion have also shown the strength of collateral status for outcome prediction. On the basis of our findings in the recanalized patients, we would predict an even stronger effect in patients receiving endovascular therapy. Hence, it should be possible to derive a prediction model with a smaller cohort of such patients receiving advanced imaging. In addition, endovascular therapy was not assessed in this study because it was not uniformly available at all centers during the study period, it is feasible that patient selection for endovascular therapy would have different cut offs for patient benefit and harm. Importantly, access to automated advanced imaging post processing was not available during this study’s data collection phase and as such we could not assess the role of the advanced imaging outputs on clinician decision-making. This study identified therapeutic cut-off points from advanced imaging outputs that may be useful in guiding clinician treatment decision-making; however, a randomized trial of the role of advanced imaging is required to draw any firm conclusions on the usefulness and impact of these imaging techniques on patients.
Application of the thresholds for benefit prediction can be applied in clinical trials of intravenous thrombolysis.14 However, the application of these results into clinical practice should be cautious, as there are different levels of acceptance for long-term disability between patients. Such individual patient preference or attitude needs to be taken into consideration when interpreting advanced imaging data for clinical use. In conclusion, we have demonstrated that it is possible to accurately predict patent outcome from intravenous thrombolysis using advanced CT imaging profiles. Advances in automated perfusion imaging analysis now allow application of these techniques in both clinical trials and routine clinical practice.
Sources of Funding
Dr Spratt was supported by National Health and Medical Research Council APP1013719. National Health and Medical Research Council/National Heart Foundation cofunded Career Development/Future Leader Fellowship APPS1110629/100827. Dr Lin was supported by Greater Charitable Foundation. X. Cheng was supported by The Science and Technology Commission of Shanghai Municipality China 1241119a8100 and 15QA1400900. K. Butcher was supported by The Canada Research Chair Program.
- Received August 18, 2016.
- Revision received November 29, 2016.
- Accepted December 2, 2016.
- © 2017 American Heart Association, Inc.
- Bivard A,
- Levi C,
- Krishnamurthy V,
- McElduff P,
- Miteff F,
- Spratt NJ,
- et al
- Yassi N,
- Parsons MW,
- Christensen S,
- Sharma G,
- Bivard A,
- Donnan GA,
- et al
- Davis SM,
- Donnan GA,
- Parsons MW,
- Levi C,
- Butcher KS,
- Peeters A,
- et al
- Miteff F,
- Levi CR,
- Bateman GA,
- Spratt N,
- McElduff P,
- Parsons MW
- Murphy A,
- Symons SP,
- Hopyan J,
- Aviv RI