Targeting Neuroprotection Clinical Trials to Ischemic Stroke Patients With Potential to Benefit From Therapy
Background and Purpose— Clinical trials of neuroprotective drugs have had limited success. We investigated whether selecting patients according to prognostic features would improve the statistical power of a trial to identify an efficacious treatment.
Methods— Using placebo data from the Glycine Antagonist in Neuroprotection (GAIN) International and National Institute of Neurological Disorders and Stroke (NINDS) recombinant tissue plasminogen activator (rtPA) clinical trials, we developed and validated simple prognostic models for stroke trial end points: Barthel Index ≥95, modified Rankin Scale ≤1, National Institutes of Health Stroke Scale ≤1, and Glasgow Outcome Scale=1. Using these models, we simulated 1000 clinical trials and estimated, under several hypothetical treatment effect patterns of neuroprotection, the effect on statistical power of including only patients with moderate prognosis. We calculated the number of patients that would have to be enrolled to maintain the statistical power achieved in selecting the whole trial population. Reanalysis of actual data from the NINDS rtPA trials confirmed the results independently.
Results— Selecting patients with moderate prognosis (predicted probability of favorable outcome 0.2 to 0.8) enabled a sample size reduction, without loss of statistical power, of between 54.6% (51.3% to 57.6%) and 68.6% (66.0% to 71.1%), depending on the treatment effect pattern and outcome measure. These benefits were largely due to the exclusion of patients with poor prognosis.
Conclusions— Targeting patients with potential to benefit enables a substantial sample size reduction without compromising statistical power or duration of recruitment. As part of a broader trial design strategy, informed use of prognostic data available acutely would help in identifying effective neuroprotective treatments.
Researchers have yet to identify therapeutic benefit from any neuroprotective drug in acute ischemic stroke.1 There are several key areas in stroke clinical trial design2: choosing the appropriate stroke onset to treatment time window, identifying the optimal dose, targeting the appropriate patient population (given the drug mechanism), and selecting appropriate and timely outcome measures. Failures in these areas, as well as investigations of drugs that were genuinely ineffective, account for the lack of success in neuroprotection.
Selecting patients for ischemic stroke clinical trials on the basis of pathophysiology may reduce the heterogeneity of patients enrolled and hence may improve the statistical power.3 Increasing the prognostic homogeneity among trial subjects may complement such targeting of salvageable tissue. When the prognosis of a patient is good after stroke, spontaneous recovery occurs frequently. Conversely, patients with very severe stroke may be destined for a poor outcome, regardless of any intervention.4 Hence, it may be difficult to demonstrate therapeutic benefit of a neuroprotectant agent in subjects at the extremes of the prognostic spectrum; patients with intermediate prognosis may offer the most information on efficacy.
Prognostic markers have been implicit in the entry criteria for trials of neuroprotection and thrombolysis through the exclusion of individuals with a mild neurological deficit5–10 and comatose patients.5,6,8,10–12 To our knowledge, no trial in acute ischemic stroke has used a prognostic model explicitly to delineate inclusion criteria.
In severe head injury, systematic selection of patients with moderate prognosis allows the sample size of a trial to be reduced by >30% without loss of statistical power.13 Equivalently, trials of a given size would be able to detect smaller, yet still clinically important, treatment effects.
Completed clinical trials in acute ischemic stroke contain a wealth of information to guide the design of future trials. In this report we use placebo patient data from previous trials to evaluate patient selection strategies using simulations. We investigate targeting patients on the basis of prognostic features, comparing selection strategies according to their effects on statistical power and, through the proportion of patients remaining eligible, on trial duration.
We used patient data from the Glycine Antagonist in Neuroprotection (GAIN) International14 and National Institute of Neurological Disorders and Stroke (NINDS) recombinant tissue plasminogen activator (rtPA) trials15 in simulations to evaluate patient selection strategies.
Models for predicting outcome were essential for the selection of patients according to prognosis and in simulating outcomes according to specific treatment effect patterns under neuroprotection. A review16 of prognostic models found no models predicting the outcomes we sought to investigate with the use of data readily available acutely.
We therefore developed and validated simple and generalizable prognostic models. The aim was not to produce definitive models for use in clinical practice but rather to illustrate the patient selection strategy under investigation. We considered outcome measures commonly used to assess favorable outcome at 90 days in stroke trials: National Institutes of Health Stroke Scale (NIHSS) ≤1, Barthel Index (BI) ≥95, modified Rankin Scale (mRS) ≤1, and Glasgow Outcome Scale (GOS)=1. We also combined these in a “global” outcome.17
A basic prognostic model for each outcome measure was developed from ischemic stroke patients in the placebo group of GAIN International. We externally validated the models using placebo data from the NINDS rtPA trials. The GOS model was developed and validated internally using NINDS rtPA placebo data. All models incorporated age and baseline stroke severity on the NIHSS.18
Simulations and Selection Strategies
Baseline clinical data were generated for placebo and active treatment groups by randomly sampling patients with replacement19 from the placebo group of the source trial. For each combination of outcome measure, treatment effect pattern, and patient selection strategy, the process was repeated to simulate data for 1000 trials.
Placebo group outcomes for a given outcome measure were simulated by applying the relevant prognostic model to the clinical data of each patient. Active treatment group outcomes were generated similarly, but a specific treatment effect (Table 1), plausible under neuroprotection, was added to the prognostic model. For example, the reduced effect in lacunar stroke would be relevant in investigating a neuroprotectant that showed little preclinical evidence of protection of white matter or subcortical neurons.
Statistical power under the global outcome depends on the correlations among its component scales; the correlations must therefore be preserved in simulating outcomes. The probability of favorable outcome for each patient was calculated separately for each outcome measure with the use of the prognostic models. Intercorrelations among outcome measures were estimated from NINDS rtPA data. Global outcome data were then simulated.20
We simulated groups of 600 and 1400 ischemic stroke patients (the approximate sizes of the NINDS rtPA trials and GAIN International, respectively). We applied 2 selection strategies to these groups: all patients (equivalent to using the entry criteria of the source trial) and patients with moderate prognosis (probability of favorable outcome between 0.2 and 0.8). We then explored whether selecting patients with moderate prognosis provided more information about treatment efficacy.
For each batch of simulated trials, the power was calculated as the proportion of trials that identified a statistically significant treatment benefit. We estimated the number of patients with moderate prognosis that would have to be enrolled to maintain the statistical power achieved by retaining the whole trial group. The selection strategy was also judged on the proportion of patients remaining eligible because this would affect the duration of trial recruitment.
We evaluated each selection strategy under a range of trial characteristics. This reflects the uncertainty regarding the likely treatment effect pattern. We sought to identify which strategy gave the greatest statistical power and which would be least sensitive to changes in treatment effect pattern. Finally, we applied the prognostic selection strategy to data from part II of the NINDS rtPA trials15 and analyzed the data as in the original trial protocol to assess its impact on trial size and conclusions.
Simulated trials were analyzed by unadjusted χ2 test (individual end points) and generalized estimating equations (global end point). Standard theory21 determined the sample size required to achieve a given statistical power for a binary outcome. Simulations were programmed with the use of SAS (version 8.2; SAS Institute Inc).
We used data from 738 ischemic stroke placebo group patients (GAIN International) and 312 placebo group patients (NINDS rtPA trials). The prognostic models for the BI, mRS, and NIHSS outcomes, developed with the use of GAIN International data, fitted adequately (Hosmer-Lemeshow probability values all >0.05). When applied to NINDS trial patients, the proportions of outcomes correctly predicted were 70.5%, 76.0%, and 79.5% for the BI, mRS, and NIHSS outcomes, respectively. The prognostic model for the GOS, developed with the use of NINDS data, also fitted adequately (P=0.81), with a cross-validated predictive accuracy of 73.7%.
Table 2 shows the power achieved for trials based on a pool of 1400 patients simulated from GAIN International data and compares the strategy of selecting patients according to prognosis with the inclusion of all patients eligible for GAIN International. Choosing patients with moderate prognosis invariably increased trial efficiency. Selecting patients on the basis of a predicted probability of favorable outcome between 0.2 and 0.8 generally maintained or increased the statistical power obtained by including all subjects, despite studying substantially fewer patients. For example, for a uniform treatment effect and the NIHSS outcome, power increased from 56.8% to 67.8% despite a reduction in the sample size from 1400 to 806.
Selecting subjects according to prognosis enables substantial sample size reduction (Table 3), regardless of the treatment effect pattern or outcome measure. The smallest reduction was 54.6% (95% CI, 51.3% to 57.6%) when the treatment effect favored nonlacunar strokes and outcome was measured on the mRS. The greatest reduction was 68.6% (95% CI, 66.0% to 71.1%) for the BI outcome when the treatment effect was dependent on the prognosis. Together with the proportion of patients who would be selected, the required sample size enables us to assess whether a selection strategy would require a longer duration of recruitment. When patients with moderate prognosis are selected, the recruitment duration would be shorter than when all patients are included, since the required sample size is lower than the number of patients selected from the original pool.
Evaluation on Independent Data
We assessed the validity of the selection strategy in a different patient population using NINDS trial data. In trials of 600 patients, the larger constant treatment effect (odds ratio 1.54) was simulated, while nonconstant effects had larger differences between subgroups than those shown in Table 1. Comparing results of trials simulated from NINDS data with Table 2 showed the same pattern of efficiency changes. For the moderate prognosis selection strategy, the required sample size results for the NINDS data confirmed a consistent benefit. The NINDS data also allowed investigation of the GOS and global outcome measures (Table 4), for which the moderate prognosis selection strategy enabled consistent sample size reductions across treatment effect patterns.
When we applied the prognostic selection strategy to NINDS (part II) data, between 161 and 196 of 333 patients remained eligible, depending on the outcome measure. The trial findings remained statistically significant (for example, global outcome P=0.027, odds ratio for favorable outcome=1.8 [1.1 to 3.0]). It became clear that only 1% to 2% of patients were being excluded because of very favorable prognosis. The benefit of selecting patients with moderate prognosis, compared with current trial recruitment strategies, is therefore largely due to the exclusion of those with very poor prognosis. Reanalysis of the NINDS data confirmed this, showing that the trial would remain conclusive if we excluded only patients with poor prognosis (Figure 1).
Our results demonstrate that basing entry criteria on a simple prognostic model would enhance the statistical power of trials of neuroprotectant agents, allowing reduced sample sizes or the detection of more modest treatment effects for a given trial size. The benefit is due to more patients having the potential, under efficacious treatment, to cross the threshold defining favorable outcome.
Why might patient selection based on prognosis be advantageous? By considering the effects of clinical features in combination, rather than setting a separate threshold for each factor, the pool of trial patients contains a broad range of clinical presentations while retaining prognostic homogeneity. For example, consider a conventional trial for patients of all ages that required a baseline NIHSS score of ≥4 as an entry criterion. A woman aged 80 years with an NIHSS score of 3 would thus be excluded. The simple prognostic model places her in the dark gray region in Figure 2: there is sufficient uncertainty about her outcome that she would be informative with regard to treatment efficacy. Conversely, a 70-year-old man scoring 18 on the NIHSS would be included under conventional criteria, despite his having little chance of a favorable outcome.
The prognosis-based strategy was consistently effective across all outcome measures and treatment effect patterns. Such a characteristic is important since we have little prior knowledge of the size or pattern of treatment effect that might be observed. Applying the selection strategy to independent data from part II of the NINDS rtPA trials illustrated its external validity and highlighted that the increased statistical power was largely due to the exclusion of patients with poor prognosis.
A selection strategy that enabled a reduced sample size but required a greatly increased enrollment period for a trial would be unattractive. Increasing the homogeneity of patients while sacrificing the generalizability of trial results is also undesirable. The prognosis-based method excluded a moderate proportion of patients (≈40%) compared with a conventional trial. It may also reduce recruitment duration while maintaining the power that would be achieved by a conventional trial.
For treatments such as thrombolysis with associated risk, it is important to assess the risk-benefit balance across a broad spectrum of patients to identify all who might potentially benefit. Although this conflicts with choosing patients more selectively, the issue would be less important in evaluating neuroprotective treatments that have little associated risk.
GAIN International14 included noncomatose adults who were previously independent, had symptoms that included limb weakness, and could be treated within 6 hours of stroke onset. The NINDS rtPA trials15 recruited ischemic stroke patients who presented with measurable deficit on the NIHSS, had a baseline CT scan free of intracranial hemorrhage, and could be treated within 180 minutes of onset. Both trials excluded severely hypertensive patients. Apart from the short time windows, inclusion criteria were general, and there were no explicit prognostic entry criteria. These data were therefore suitable for the present study and allowed evaluation of selection strategies on neuroprotective and thrombolytic trial populations. Nonetheless, patients who would have been informative about treatment efficacy may have been excluded from the original clinical trials and were unavailable for our study. By extending the pool of available patients, it may be possible to increase the benefits and generalizability of prognosis-based selection. Although we studied outcome measures frequently used in acute stroke trials, our findings relate only to those end points.
Prognosis-based patient selection for trials could result in stepwise regulatory approval that is initially given for a subset of patients and subsequently extended to a wider population. Similarly, memantine has been approved for mild to moderate Alzheimer disease and was subsequently investigated in severe disease.22 Stepwise approval seems unlikely to be necessary, however, since the proposed approach will not restrict entry on the basis of any single factor; the trial population will still include patients with a wide range of age and baseline severity. Regulatory approval should be based on this wide range.
Implementing restrictive trial entry criteria may diminish the motivation of investigators and may result in the inadvertent omission of eligible patients. If prognosis-based patient selection proved unacceptable to investigators, one could pursue an alternative strategy. After broad entry criteria are used, data analysis could incorporate a prespecified primary analysis on the moderate prognosis subgroup (defined with the use of pretreatment data). Such a design would optimize the investigation of efficacy while maximizing the potential generalizability of the results. The risk of this approach is that regulatory authorities may consider only the intent-to-treat analysis, in which the diluted treatment effect may be neutral.
Targeting patients with appropriate prognosis is only one of several key aspects of clinical trial design in neuroprotection. Time from stroke onset to treatment is a critical patient selection criterion, because the earlier the time of therapeutic intervention, the greater the chance of observing a treatment benefit.2 Patients should be selected who have a tissue substrate on which the neuroprotective compound may act.3 Accurate estimation of the dose-response curve from early-phase trials is also essential.23 Our methodology could also determine the impact on statistical power and sample size of selecting patients according to the presence of potentially salvageable hypoperfused tissue, as defined by diffusion- and perfusion-weighted MRI.24
The complexity of designing clinical trials of neuroprotective drugs in acute ischemic stroke presents investigators with a considerable challenge. The wide-ranging recommendations in place2 reflect this, and no single technique will provide a solution. Nevertheless, as part of a multifaceted strategy, informed use of prognostic data available acutely will help to ensure that identification of an effective neuroprotective treatment remains an achievable goal.
This study was supported by a UK Medical Research Council Career Development Fellowship to Dr Weir. The authors thank the NINDS rtPA Stroke Study Group for provision of the rtPA trial data. GAIN International Steering Committee members are as follows: K. Asplund, A. Carolei, S.M. Davis, H.-C. Diener, M. Kaste, J.-M. Orgogozo, J. Whitehead, K.R. Lees. The GAIN International Trial was sponsored by GlaxoSmithKline (previously GlaxoWellcome).
Members of the GAIN International Steering Committee are listed in the Acknowledgments.
Review of this article was directed by the Editor-in-Chief, Dr Vladimir Hachinski.
- Received May 14, 2004.
- Revision received June 2, 2004.
- Accepted June 3, 2004.
Kidwell CS, Liebeskind DS, Starkman S, Saver JL. Trends in acute ischemic stroke trials through the 20th century. Stroke. 2001; 32: 1349–1359.
Stroke Therapy Academic Industry Roundtable II (STAIR-II). Recommendations for clinical trial evaluation of acute stroke therapies. Stroke. 2001; 32: 1598–1606.
Muir KW. Heterogeneity of stroke pathophysiology and neuroprotective clinical trial design. Stroke. 2002; 33: 1545–1550.
Dorman PJ, Sandercock PA. Considerations in the design of clinical trials of neuroprotective therapy in acute stroke. Stroke. 1996; 27: 1507–1515.
Wahlgren NG, Ranasinha KW, Rosolacci T, Franke CL, van Erven PM, Ashwood T, Claesson L. Clomethiazole Acute Stroke Study (CLASS): results of a randomized, controlled trial of clomethiazole versus placebo in 1360 acute stroke patients. Stroke. 1999; 30: 21–28.
Clark WM, Wechsler LR, Sabounjian LA, Schwiderski UE, for the Citicoline Stroke Study Group. A phase III randomized efficacy trial of 2000 mg citicoline in acute ischemic stroke patients. Neurology. 2001; 57: 1595–1602.
Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, Boysen G, Bluhmki E, Hoxter G, Mahagne MH, et al. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke. The European Cooperative Acute Study (ECASS) JAMA. 1995; 274: 1017–1025.
Enlimomab Acute Stroke Trial Investigators. Use of anti-ICAM-1 therapy in ischemic stroke: results of the Enlimomab Acute Stroke Trial. Neurology. 2001; 57: 1428–1433.
Lyden P, Shuaib A, Ng K, Levin K, Atkinson RP, Rajput A, Wechsler L, Ashwood T, Claesson L, Odergren T, Salazar-Grueso E. Clomethiazole Acute Stroke Study in Ischemic Stroke (CLASS-I): final results. Stroke. 2002; 33: 122–128.
Davis SM, Lees KR, Albers GW, Diener HC, Markabi S, Karlsson G, Norris J. Selfotel in acute ischemic stroke: possible neurotoxic effects of an NMDA antagonist. Stroke. 2000; 31: 347–354.
Tilley BC, Marler JR, Geller NL, Lu M, Legler JM, Brott TG, Lyden PD, Grotta J. Use of a global test for multiple outcomes in stroke trials with application to the National Institute of Neurological Disorders and Stroke t-PA Stroke Trial. Stroke. 1996; 27: 2136–2142.
Brott TG, Adams HP, Olinger CP, Marler JR, Barsan WG, Biller J, Spilker J, Holleran R, Eberle R, Hertzberg V, Rorick M, Moomaw CJ, Walker M. Measurements of acute cerebral infarction: a clinical examination scale. Stroke. 1989; 20: 864–870.
Efron B, Tibshirani RJ. An Introduction to the Bootstrap. New York, NY: Chapman and Hall; 1993.
Machin D, Campbell MJ. Statistical Tables for the Design of Clinical Trials. Oxford, UK: Blackwell Scientific Publications; 1987.
Schlaug G, Benfield A, Baird AE, Siewert B, Lovblad KO, Parker RA, Edelman RR, Warach S. The ischemic penumbra: operationally defined by diffusion and perfusion MRI. Neurology. 1999; 53: 1528–1537.
Wardlaw JM, del Zoppo G, Yamaguchi T. Thrombolysis for acute ischemic stroke (Cochrane review). In: The Cochrane Library, Issue 1, 2002. Oxford, UK: Update Software.