Multivariable Analysis of Outcome Predictors and Adjustment of Main Outcome Results to Baseline Data Profile in Randomized Controlled Trials
Safe Implementation of Thrombolysis in Stroke-MOnitoring STudy (SITS-MOST)
Background and Purpose— The Safe Implementation of Thrombolysis in Stroke-MOnitoring STudy (SITS-MOST) unadjusted results demonstrated that intravenous alteplase is well tolerated and that the effects were comparable with those seen in randomized, controlled trials (RCTs) when used in routine clinical practice within 3 hours of ischemic stroke onset. We aimed to identify outcome predictors and adjust the outcomes of the SITS-MOST to the baseline characteristics of RCTs.
Methods— The study population was SITS-MOST (n=6483) and pooled RCTs (n=464) patients treated with intravenous alteplase within 3 hours of stroke onset. Multivariable, backward stepwise regression analyses (until P≤0.10) were performed to identify the outcome predictors for SITS-MOST. Variables appearing either in the final multivariable model or differing (P<0.10) between SITS-MOST and RCTs were included in the prediction model for the adjustment of outcomes. Main outcome measures were symptomatic intracerebral hemorrhage, defined as National Institutes of Health Stroke Scale deterioration ≥1 within 7 days with any hemorrhage (RCT definition), mortality, and independency as defined by modified Rankin Score of 0 to 2 at 3 months.
Results— The adjusted proportion of symptomatic intracerebral hemorrhage for SITS-MOST was 8.5% (95% CI, 7.9 to 9.0) versus 8.6% (6.3 to 11.6) for pooled RCTs; mortality was 15.5% (14.7 to 16.2) versus 17.3% (14.1 to 21.1); and independency was 50.4% (49.6 to 51.2) versus 50.1% (44.5 to 54.7), respectively. In the multivariable analysis, older age, high blood glucose, high National Institutes of Health Stroke Scale score, and current infarction on imaging scans were related to poor outcome in all parameters. Systolic blood pressure, atrial fibrillation, and weight were additional predictors of symptomatic intracerebral hemorrhage. Current smokers had a lower rate of symptomatic intracerebral hemorrhage. Disability before current stroke (modified Rankin Score 2 to 5), diastolic blood pressure, antiplatelet other than aspirin, congestive heart failure, patients treated in new centers, and male sex were related to high mortality at 3 months.
Conclusions— The adjusted outcomes from SITS-MOST were almost identical to those in relevant RCTs and reinforce the conclusion drawn previously in the unadjusted analysis. We identified several important outcome predictors to better identify patients suitable for thrombolysis.
One of the preconditions laid down for the European Union license provided in 2002 for the use of intravenous alteplase in the treatment of acute ischemic stroke within 3 hours of symptom onset was that the Safe Implementation of Thrombolysis in Stroke-MOnitoring STudy (SITS-MOST) be performed. Another condition was that a randomized, placebo-controlled study be undertaken to assess the use of alteplase in patients with stroke within the extended therapeutic window of between 3 and 4.5 hours (European Cooperative Acute Stroke Study [ECASS] III).
SITS-MOST has demonstrated that intravenous alteplase is well tolerated and that when used in routine clinical practice (administered to suitable patients within 3 hours of the onset of acute ischemic stroke), the effects were comparable with those seen in the randomized, controlled trials (RCTs).1 The conclusions were based on a comparison of the results from SITS-MOST with those from the active arms of relevant pooled, RCTs.2 The results suggested a trend toward a more favorable outcome in SITS-MOST compared with RCTs. For mortality, the difference in favor of SITS-MOST was robust. However, the results also showed differences for several baseline variables between the 2 study populations. This prompted the current multivariable analysis to evaluate the prognostic potential of these variables and adjust the SITS-MOST main outcome results for these prognostically important variables.
In this article, multivariable models were used to identify the prognostically important baseline factors for the main outcomes in SITS-MOST as a basis for adjustment of the main outcomes compared with a case-mix from RCTs.
Patients and Procedures
The methodology for SITS-MOST and the SITS International Stroke Thrombolysis Register (SITS-ISTR), including the structure of the SITS network, procedure for data collection and management, patient identification, and verification of source data, have been described previously.1 In brief, SITS-MOST was a prospective, open, multinational, observational monitoring study for clinical centers using thrombolysis for the treatment of acute stroke within countries affiliated to the European Evaluation of Medicines Agency (EMEA). The study period was between December 2002 and April 2006. The study was established as a cohort of SITS-ISTR, an Internet-based, academic-driven, interactive thrombolysis register.3
Baseline and demographic characteristics, stroke severity (as measured by National Institutes of Health Stroke Scale [NIHSS] score), time intervals, risk factors, medication history, and imaging scan data taken on admission were collected. Outcome measurements were NIHSS score at 2 hours, 24 hours, and 7 days; imaging scan data after treatment; and modified Rankin Score (mRS) at 3 months.
A center’s previous experience with stroke thrombolysis was designated as “experienced” if they had participated in one or both of the ECASS studies and/or had performed at least 5 thrombolytic procedures in patients with stroke before entering SITS-MOST. Centers that lacked such experience were designated “new.”
Main Outcome Measurements
The main outcome measurements from SITS-MOST were symptomatic intracerebral hemorrhage (SICH), mortality, and functional independence. These measurements were defined as follows: (1) SICH per the RCT definition: symptomatic (any deterioration in NIHSS score within 7 days) intracerebral hemorrhage of any type in any posttreatment imaging scans after the start of thrombolysis treatment.2 This definition was used for the current study to enable comparisons with RCTs; (2) death within 3 months (mRS 6); (3) rate of independency (mRS 0 to 2) at 3 months; and (4) SICH per the SITS-MOST definition: symptomatic (deterioration in NIHSS score of ≥4 points within 24 hours) intracerebral hemorrhage type 2 in the 22 to 36 hours follow-up imaging scans after the start of thrombolysis treatment. SICH per the SITS-MOST definition results were available for only the first part of the analysis, because the comparison of data from pooled RCT with this definition is not available.
Descriptive statistics for the baseline and demographic data from SITS-MOST were presented according to the main outcomes: SICH per SITS-MOST definition, SICH per RCT definition, mortality, and rate of independence at 3 months (Table 1). Unknown values were excluded from the denominator when calculating proportions as done in the previous publication.1
Multivariable logistic regression analyses were performed in an explorative manner to find predictors for the main outcomes. These analyses were based on the baseline and demographic data from SITS-MOST. Variables not showing a linear relationship to an outcome on the logit scale were categorized according to clinical relevance. To avoid variable selection caused by spurious correlations, only variables showing a relationship to the outcome (defined as univariable P≤0.25) were included as potential predictors. No interaction terms were included in the models. For each main outcome, a backward stepwise procedure was performed using P>0.10 of the likelihood ratio test for exclusion. The variables included for each respective outcome were (1) common in all analyses—age, antiplatelet other than aspirin, aspirin, atrial fibrillation, glucose, hypertension, and signs of current infarction at baseline imaging; (2) SICH per SITS-MOST definition—alteplase dose (milligrams/kilograms body weight), diabetes, NIHSS baseline categories, previous stroke, systolic blood pressure, and weight; (3) SICH per RCT definition—alteplase dose (milligrams/kilograms body weight), antihypertensive treatment, diabetes, hyperlipidemia, NIHSS baseline categories, smoking, systolic blood pressure, and weight; (4) mortality—alteplase dose (milligrams/kilograms body weight), centers’ previous stroke experience, congestive heart failure, diabetes, diastolic blood pressure categories, sex, hyperlipidemia, NIHSS baseline, previous stroke, rate of normal function (mRS 0 to 1) before stroke, and systolic blood pressure; and (5) independence—antihypertensive treatment, congestive heart failure, diabetes, diastolic blood pressure categories, sex, NIHSS at baseline, previous stroke, rate of normal function (mRS 0 to 1) before stroke, systolic blood pressure categories, and weight.
For the second analysis, a prediction model was built with a similar approach as it was performed in the first multivariable analysis of the SITS-MOST data but repeated only with the variables occurring in both SITS-MOST and RCTs. Variables appearing in the final model of the first multivariable analysis and variables not appearing in the final model but differing significantly (defined as P<0.10) between SITS-MOST and pooled RCTs were added into the prediction model. Baseline data from NINDS A (n=143),4 NINDS B (n=168),4 ECASS I (n=49),5 ECASS II (n=81),6 and ATLANTIS A, B (n=23)7,8 for patients treated with alteplase within 3 hours of stroke symptom onset were extracted and pooled for the RCT case-mix. The SITS-MOST outcomes were then predicted using the baseline data from the relevant pooled RCTs.
Variables included in each prediction model were as follows: (1) common in all analyses—age, aspirin, congestive heart failure, diabetes, glucose, previous stroke, time from stroke onset to treatment, and weight; (2) SICH per SITS-MOST definition—diastolic blood pressure, NIHSS baseline categories, and systolic blood pressure; (3) SICH per RCT definition—atrial fibrillation, diastolic blood pressure, NIHSS baseline categories, and systolic blood pressure; (4) mortality—diastolic blood pressure categories, sex, NIHSS at baseline, and systolic blood pressure; and (5) independence—diastolic blood pressure categories, hypertension, NIHSS at baseline, and systolic blood pressure categories.
In the second analysis, the performance of each model on SITS-MOST data was assessed by the c statistic (area under the receiver operating characteristic curve).9 The c statistic of 0.5 means that the model is inadequate and might as well have occurred by chance and 1.0 equals perfect prediction. Because prediction models tend to be overoptimistic when evaluated on the same data (ie, SITS-MOST data), crossvalidation procedures were performed on SITS-MOST to achieve unbiased estimates of the c statistic. The uncertainty of the prediction of the RCT data and the c statistic was therefore evaluated by randomly selecting 70% of the observations as training data and the rest as validation data. This was repeated 10 000 times; for each run, the prediction model based on the training data was used to calculate the c statistic on the validation data and to predict the result using RCT data. The mean and 2.5 of 97.5 percentiles of the resulting distributions of the c statistic and prediction results were calculated.
To calculate the 95% CIs of the observed proportions for SICH, mortality, and rate of independence in the SITS-MOST and RCT mix population, a score method with continuity correction10 was used.
For statistical analyses, R version 2.5.1 (R foundation for statistical computing, Vienna, Austria) and SAS version 9.1.3 (SAS Institute Inc, Cary, NC) were used.
In SITS-MOST, 6483 patients were included at 285 centers across 14 European countries between December 2002 and April 2006. Table 1 shows the baseline and demographic characteristics of patients according to SICH as per the SITS-MOST and RCT definitions, mortality, and rate of independence at 3 months.
The Figure shows the OR and 95% CIs for the variables appearing in the final model of the backward stepwise multivariable regression analyses (P≤0.10). Older age, high blood glucose, and high NIHSS score at baseline were significantly related to poor outcome in all parameters. A sign of current infarction on baseline imaging scans was also related to poor outcome in all parameters except for SICH per the SITS-MOST definition. An uncertain current infarction on the baseline imaging scans was also related to poor outcome compared with no sign of current infarction for mortality and independence at 3 months.
Systolic blood pressure and weight were additional predictors of both types of SICH. Aspirin therapy at stroke onset was related to a high rate of SICH per SITS-MOST definition and atrial fibrillation was related to a high rate of SICH per RCT definition. Current smokers had a significantly lower rate of SICH per RCT definition.
Disability before current stroke (mRS 2 to 5), diastolic blood pressure >90 mm Hg, antiplatelet other than aspirin, congestive heart failure, patients treated in new centers, male sex, and lower alteplase dose in milligrams/kilograms body weight were related to high mortality at 3 months. Disability before current stroke (mRS 2 to 5), diastolic blood pressure >90 mm Hg, and previous stroke were related to lower functional independence at 3 months.
Table 2 shows the unadjusted and adjusted outcomes of SITS-MOST based on the prediction models. Outcome data for pooled RCTs were extracted from the 0- to 3-hour pooled alteplase studies and their unadjusted outcome results and 95% CI were compared.
The adjusted proportion of SICH per RCT definition for SITS-MOST was 8.5% (95% CI, 7.9 to 9.0) compared with 8.6% (6.3 to 11.6) in the pooled RCTs. The adjusted proportion of mortality was 15.5% (14.7 to 16.2) versus 17.3% (14.1 to 21.1), and the rate of independence was 50.4% (49.6 to 51.2) versus 50.1% (44.5 to 54.7), respectively.
SICH according to the SITS-MOST protocol definition could not be used for comparison with RCTs because these data were not available. The adjusted SICH proportion for SITS-MOST according to this definition was 2.0% (95% CI, 1.7 to 2.3). The c statistic was 0.736 (0.701 to 0.773).
For SICH as per the RCT definition, the mean (95% CI) for c statistic was 0.674 (0.635 to 0.712) after crossvalidation. For mortality at 3 months, the c statistic was 0.752 (0.722 to 0.780) after crossvalidation. For rate of independence at 3 months, the c statistic was 0.782 (0.765 to 0.799) after crossvalidation.
The results of this multivariable analysis reinforce the previous SITS-MOST report1 that intravenous alteplase is safe and that the effects were comparable with those seen in RCTs in a routine clinical setting when administered to suitable patients within 3 hours of the onset of acute ischemic stroke. The prediction models used for calculating the adjusted main outcomes were built on SITS-MOST data and applied to the RCT patient case-mix. The results showed that the trend toward favorable outcomes in SITS-MOST compared with RCTs2 in the unadjusted analysis1 was mainly due to differences in baseline characteristics between SITS-MOST and RCTs.
We identified several important predictors for SICH, mortality, and rate of independence in the first multivariable analyses. Our findings support some of the previously published results of prognostic factors for intravenous thrombolysis in acute ischemic stroke.11–13 Older age, high blood glucose, and stroke severity as measured by NIHSS score appeared to be significantly related to poor outcome for all outcome parameters. The association between NIHSS and SICH per the SITS-MOST protocol definition was not linear and, therefore, we categorized the NIHSS score. High systolic blood pressure but not diastolic blood pressure was related to both definitions of SICH. Alteplase dose in milligrams/kilograms body weight was not related to any definition of SICH but high body weight (in kilograms) was associated with both SICH definitions. Interestingly, higher dose of alteplase in milligrams/kilograms body weight was associated with lower rate of mortality.
Aspirin treatment at stroke onset was related to SICH as per the SITS-MOST definition but not as per the RCT definition. These results suggest that concomitant aspirin treatment at stroke onset in patients receiving thrombolysis increases the risk for large intracerebral hemorrhage. However, it should be noted that the overall rate of SICH per SITS-MOST definition was very low (1.7% unadjusted and 1.9% after adjustment). Patients treated with other antiplatelet agents (combined variable for dipyridamole, clopidogrel, and other unspecified) was associated with higher mortality at 3 months, but importantly, this was not due to high rates of SICH. Antiplatelet treatment for patients with stroke on thrombolysis is largely unexplored and further research in this area is needed to establish optimal antiplatelet treatment strategies.
Current smoking appeared to be related to lower incidence of SICH as per the RCT definition, a result that has previously been observed in the National Institute of Neurological Diseases and Stroke (NINDS) study.14 A secondary analysis of NINDS data also indicated that recent smokers treated with intravenous alteplase had a significant drop in NIHSS score at 24 hours and lower mortality over 1 year. In the unadjusted analysis, there were fewer smokers in the better outcome group, but this did not remain significant in the multivariable analysis.15
Stroke severity at baseline measured by NIHSS score and functional disability before current stroke measured by a mRS 2 to 5 appeared to be the strongest predictors for mortality and rate of independence at 3 months. These factors are already recognized to be strong outcome predictors.16 Signs of current infarction on the pretreatment imaging scans were related to high mortality and a low rate of independence at 3 months. A sign of current infarction was also related to SICH per RCT definition but not to SICH per SITS-MOST definition. Male sex was associated with increased mortality, which has also been observed in other studies.11,17,18 Previous hypertension and high diastolic blood pressure at baseline were predictors of poor outcome of independence.
As observed in the unadjusted analysis, patients treated in new centers participating in SITS-MOST had a higher rate of mortality than those treated in experienced centers1 also in the multivariable model. Importantly, as in the unadjusted analysis, treatment in new centers was not associated with any definition of SICH in the multivariable analysis. This result supports the explanation, as stated in the previous report, that the higher mortality in new centers compared with experienced centers was not due to alteplase treatment. It is also supported by a recent analysis of pooled RCT data suggesting that alteplase treatment does not have any detrimental effect on mortality rate after stroke.11 The available baseline and demographic data could not explain the reasons for the difference in mortality between new and experienced centers.
There are a few limitations to this study. First, the comparisons made are between data collected in an observational study and from historical controls, so any conclusion, even after careful multivariable analysis, should be interpreted with caution. Second, we were unable to compare the SITS-MOST definition of SICH with RCTs, because RCT SICH outcome data with this definition were not available. The only available solution was to compare SICH data as per the RCT definition.2 The RCT definition may include small hemorrhagic transformations within large infarct edemas, in which the edema rather than the hemorrhage is the likely major cause of any clinical deterioration. Adjusted SICH rate according to the SITS-MOST definition was slightly higher than in the previous report based on an unadjusted analysis.1
Despite these limitations, SITS-MOST is the largest database in stroke thrombolysis developed so far. The SITS-MOST protocol, developed in close collaboration with Boehringer Ingelheim and the EMEA, stated that the primary analysis should include the unadjusted comparison between the study data and RCT data, which has been reported previously.1 The prediction model used in the multivariable analysis to adjust the SITS-MOST outcome has been validated internally and well described in the literature.11,12,19 The prediction model was built on SITS-MOST baseline variables shown to be prognostically important in the first analysis and baseline variables not appearing as prognostically important but differing significantly between SITS-MOST and RCT. It can be discussed whether the later variables should be included in the prediction model; however, the results were practically identical if they were removed. We believe that for future similar observational studies, the approach used in this article may provide additional value for comparisons between monitoring study results and RCTs.
SITS-MOST outcomes adjusted to the RCT case-mix were almost identical to those in relevant pooled alteplase RCTs. The trend toward more favorable outcomes in SITS-MOST compared with RCTs in the unadjusted analysis seemed to be explained by baseline differences. The results support the previous conclusion based on an unadjusted analysis,1 that intravenous alteplase is safe and that the effects were comparable with those in RCTs in routine clinical use when administered within 3 hours to suitable patients with acute ischemic stroke. We identified several important predictors for SICH, mortality, and rate of independence in the multivariable analyses. These prognostically important variables and the prediction model, based on SITS-MOST, could be used to assist patient selection for thrombolysis after acute ischemic stroke.
Sources of Funding
SITS-MOST is funded by an unrestricted grant from Boehringer Ingelheim in agreement with a regulatory decision by the European Union Commission. Uppsala Clinical Research (UCR), Sweden, develops, maintains, and upgrades the software for the SITS register in close collaboration with SITS. Z. Preston received funding from Boehringer Ingelheim to provide editorial assistance in the coordination of the submission.
N.W. and G.V. have both received compensation from Boehringer Ingelheim for serving on scientific advisory committees and have undertaken speaking engagements for the company. N.A. is an employee of SITS-International. R.R. and M.G. have been involved in organizing educational campaigns supported by Boehringer Ingelheim. R.R. has attended a medical congress as a representative of the Turku University Central Hospital sponsored by Boehringer Ingelheim. M.G.H. received compensation for consultancies, committee advice, speaking and educational engagements by Boehringer Ingelheim. K.L. is the chairman of the independent data monitoring committee for the ECASS III trial of rtPA in acute stroke sponsored by Boehringer Ingelheim and the chairman of the DIAS and DEDAS trials of desmoteplase in acute stroke sponsored by PAION, Forest. K.L. was also an expert clinical advisor to NICE for a recent thrombolysis review. His department has received grants from Boehringer Ingelheim to support secretarial expenses in the coordination of SITS-MOST. V.L., R.R., M.G., D.T., and M. Kaste have all been reimbursed by Boehringer Ingelheim for attending symposia. G.F., A.D., R.R., M.G., D.T., and M. Kaste have received fees from the company for speaking and consulting. Boehringer Ingelheim also funds a staff member for M.G. and D.T.M. Kaste has also served on the Steering Committees of ECASS I, II, and III and has received honoraria and travel expenses for participating in Steering Committee meetings. G.F. has received research funding for stroke-related activities from Boehringer Ingelheim. E.B. and T.M. are employees of Boehringer Ingelheim. W.H. has received fees from Boehringer Ingelheim for speaking and has received consultancy fees from PAION and Forest. Authors are members of the Scientific Committee of SITS-MOST; M. Kaste, M. Köhrmann, and T.E. were leading recruiters of patients into the study. Analyses for reports to the EMEA were developed in close collaboration with Boehringer Ingelheim (Germany).
- Received November 23, 2007.
- Revision received April 27, 2008.
- Accepted May 20, 2008.
Wahlgren N, Ahmed N, Davalos A, Ford GA, Grond M, Hacke W, Hennerici MG, Kaste M, Kuelkens S, Larrue V, Lees KR, Roine RO, Soinne L, Toni D, Vanhooren G. Thrombolysis with alteplase for acute ischaemic stroke in the Safe Implementation of Thrombolysis in Stroke-Monitoring Study (SITS-MOST): an observational study. Lancet. 2007; 369: 275–282.
SITS-ISTR (Safe Implementation of Thrombolysis in Stroke - International Stroke Thrombolysis Register). Available at: www.acutestroke.org. Accessed December 22, 2007.
Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, Boysen G, Bluhmki E, Hoxter G, Mahagne MH, et al. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Stroke Study (ECASS). JAMA. 1995; 274: 1017–1025.
Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D, Larrue V, Bluhmki E, Davis S, Donnan G, Schneider D, Diez-Tejedor E, Trouillas P. Randomised double-blind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Second European-Australasian Acute Stroke Study Investigators. Lancet. 1998; 352: 1245–1251.
Clark WM, Wissman S, Albers GW, Jhamandas JH, Madden KP, Hamilton S. Recombinant tissue-type plasminogen activator (alteplase) for ischemic stroke 3 to 5 hours after symptom onset. The ATLANTIS Study: a randomized controlled trial. Alteplase Thrombolysis for Acute Noninterventional Therapy in Ischemic Stroke. JAMA. 1999; 282: 2019–2026.
Clark WM, Albers GW, Madden KP, Hamilton S. The rtPA (alteplase) 0- to 6-hour acute stroke trial, part A (A0276g): results of a double-blind, placebo-controlled, multicenter study. Thrombolytic therapy in acute ischemic stroke study investigators. Stroke. 2000; 31: 811–816.
Kent DM, Selker HP, Ruthazer R, Bluhmki E, Hacke W. The stroke-thrombolytic predictive instrument: a predictive instrument for intravenous thrombolysis in acute ischemic stroke. Stroke. 2006; 37: 2957–2962.
Kent DM, Selker HP, Ruthazer R, Bluhmki E, Hacke W. Can multivariable risk–benefit profiling be used to select treatment-favorable patients for thrombolysis in stroke in the 3- to 6-hour time window? Stroke. 2006; 37: 2963–2969.
Intracerebral hemorrhage after intravenous t-PA therapy for ischemic stroke. The NINDS t-PA Stroke Study Group. Stroke. 1997; 28: 2109–2118.
Ovbiagele B, Saver JL. The smoking–thrombolysis paradox and acute ischemic stroke. Neurology. 2005; 65: 293–295.
Muir KW, Weir CJ, Murray GD, Povey C, Lees KR. Comparison of neurological scales and scoring systems for acute stroke prognosis. Stroke. 1996; 27: 1817–1820.
Saposnik G, Di Legge S, Webster F, Hachinski V. Predictors of major neurologic improvement after thrombolysis in acute stroke. Neurology. 2005; 65: 1169–1174.
Hill MD, Kent DM, Hinchey J, Rowley H, Buchan AM, Wechsler LR, Higashida RT, Fischbein NJ, Dillon WP, Gent M, Firszt CM, Schulz GA, Furlan AJ. Sex-based differences in the effect of intra-arterial treatment of stroke: analysis of the PROACT-2 study. Stroke. 2006; 37: 2322–2325.
Counsell C, Dennis M, McDowall M, Warlow C. Predicting outcome after acute and subacute stroke: development and validation of new prognostic models. Stroke. 2002; 33: 1041–1047.