(Stroke. 2001;32:1349.)
© 2001 American Heart Association, Inc.
Original Contributions |
From the University of California at Los Angeles (UCLA) Stroke Center (C.S.K., S.S., J.L.S.), Department of Neurology (C.S.K., S.S., J.L.S.), and Department of Emergency Medicine (S.S.), UCLA Medical Center; and Comprehensive Stroke Center and Department of Neurology, University of Pennsylvania, Philadelphia (D.S.L.).
Correspondence to Chelsea S. Kidwell, MD, Department of Neurology, 710 Westwood Plaza, UCLA Medical Center, Los Angeles, CA 90095. E-mail ckidwell{at}ucla.edu
| Abstract |
|---|
|
|
|---|
MethodsAll controlled trials targeting acute ischemic stroke with a final report in English were identified through MEDLINE and international trial registries. Data regarding trial design, implementation, and results were extracted. A formal 100-point scale was used to rate trial quality.
ResultsA total of 178 controlled acute stroke trials were identified, encompassing 73 949 patients. Eighty-eight trials involved neuroprotective agents, 59 rheological/antithrombotic agents, 26 agents with both neuroprotective and rheological/antithrombotic effects, and 5 a nonpharmacological intervention. Only 3 trials met conventional criteria for a positive outcome. Between the 1950s and 1990s, the number of trials per decade increased from 3 to 99, and mean trial sample size increased from 38 (median, 26) to 661 (median, 113). During 19801999, median time window allowed for enrollment decreased per half decade from 48 to 12 hours. Reported pharmaceutical sponsorship increased substantially over time, from 38% before 1970 to 68% in the 1990s. Trial quality improved substantially from a median score of 12 in the 1950s to 72 in the 1990s.
ConclusionsAccelerating trends in acute stroke controlled trials include growth in number, sample size, and quality, and reduction in entry time window. These changes reflect an increased understanding of the pathophysiology of acute stroke, the imperative for treatment initiation within a critical time window, and more sophisticated trial design.
Key Words: cerebral ischemia clinical trials stroke, acute stroke, ischemic stroke management
| Introduction |
|---|
|
|
|---|
The advent of the controlled clinical trial in medicine is well illustrated by its application in the field of acute ischemic stroke. As the third leading cause of death and the leading cause of adult disability in the United States, stroke is a major public health concern. At the fundamental science level, remarkable advances occurred in the field of cerebrovascular disease in the latter part of the 20th century, including increased understanding of the basic pathophysiology of intricate regulatory systems for vascular hemostasis and cellular energetic homeostasis that are disrupted in ischemia. These advances generated a multitude of promising therapeutic agents for clinical investigators to evaluate. One set of clinical trials led to the widespread acceptance of intravenous tissue plasminogen activator (tPA) as the first proven, effective treatment for acute ischemic stroke.4 In contrast, there has been persistent failure of human trials targeted at neuroprotective agents for acute ischemic stroke.
To characterize general and domain-specific trends in the conduct of controlled clinical trials, we undertook a systematic and comprehensive analysis of treatment trials for acute ischemic stroke. We hypothesized that, over time, significant changes have occurred in clinical trial design characteristics, time windows from symptom onset allowed for enrollment, choice of outcome measures, and overall trial quality. Moreover, we expected that these trends would afford a contextual framework for understanding both the successes and failures of prior trials and for identifying areas of ongoing deficiency.
| Methods |
|---|
|
|
|---|
Reports that met the following criteria were included for detailed analysis: (1) study was a controlled clinical trial, (2) study treatment was initiated within 15 days of symptom onset, (3) target disease was acute ischemic stroke, and (4) final full-length, brief, or abstract-length report appeared in the English language. Trials were excluded if the authors did not provide enough information to determine whether the study met the aforementioned criteria or whether the study was targeted at long-term secondary stroke prevention or solely at deep venous thrombosis prevention. If multiple reports of the same trial were retrieved, the most comprehensive report was used for final data extraction.
Trial Characteristics
Reports were categorized as abstracts only, brief
reports (including letters), or full-length articles. From each report,
information was abstracted regarding the intervention(s) studied,
number of patients enrolled, time window for enrollment, and mean time
to actual enrollment (median time to enrollment was used if mean time
was not reported). All interventions were categorized as having one of
the following mechanisms of action: rheological/antithrombotic,
neuroprotective, both rheological/antithrombotic and neuroprotective,
or nonpharmacological. The rheological/antithrombotic category was
further subdivided into thrombolytic agents,
antiplatelet agents, anticoagulants, and agents designed to enhance
blood flow or viscosity. Neuroprotective agents were further subdivided
according to the Dorman-Counsel-Sandercock classification system into
the following categories: modulators of excitatory amino acids,
modulators of calcium influx, metabolic
activators, antiedema agents, inhibitors of
leukocyte adhesion, free radical scavengers, promotors of membrane
repair, and agents of other or unknown
mechanism.8
Additional data abstracted regarding aspects of trial design included trial setting (single-center or multicenter), target stroke class (ischemic or both ischemic and hemorrhagic), single target stroke territory (carotid, middle cerebral artery, hemispheric, vertebrobasilar, other), single target stroke mechanism (cardioembolic, lacunar, large-vessel atherosclerosis, or other), randomization (yes, no), blinding (none, single, double), control group (placebo, other active agent, or standard care), number of agents tested, minimum and maximum time windows allowed for enrollment, and study phase (2, 2/3, 3, not specified). A trial was categorized as phase 2 if a surrogate outcome measure was used as the primary end point or if any of the following terms were used: pilot, phase 2, exploratory, dose escalation, dose comparison. A trial was categorized as phase 2/3 if the authors stated that the trial was primarily testing both safety and efficacy and was categorized as phase 3 if the authors stated that the trial was primarily testing efficacy or directly stated that the study was a phase 3 trial. Phase 2 trials were further divided into those that were primarily pilot trials with clinical outcome measures, those that primarily involved surrogate outcome measures, and those that were primarily dose-ranging studies.
Outcome data abstracted from each report included the type of outcome measures used (clinical or surrogate), the time point at which the final outcome measures were assessed (measured in days), and whether the study prespecified a single primary outcome measure. Clinical outcome measures were further categorized into nonvalidated versus validated rating scales (eg, National Institutes of Health Stroke Scale [NIHSS], Barthel Index).9 10 Rating scales were considered validated if there was published evidence for their validity and reliability.
Classification of Trial Results
For all studies, the following classification scheme
was used to record the authors final judgment of trial findings
regarding drug efficacy: beneficial, trend beneficial, neutral with a
subgroup beneficial, neutral, neutral with a subgroup harmful, trend
harmful, harmful, or dose escalation/safety/surrogate trial. Author
judgment of beneficial was
assigned if anywhere in the article, abstract, or text the authors used
the words positive,
efficacious,
beneficial,
better than placebo,
effective,
useful, or similar terms to
describe trial results. Author judgment of
trend beneficial was assigned
if the authors used the words trend
positive,
encouraging,
suggestive,
beneficial trend, or like
terms. Author judgment of subgroup
positive was assigned if the authors explicitly discussed a
subgroup finding as positive, showing benefit or efficacy, or
significantly better than placebo.
Neutral was assigned if the
authors concluded that the trial had no net positive or negative
effects overall or in subgroup analyses.
Harmful,
trend harmful, and
subgroup harmful were assigned
in a manner analogous to the beneficial categories.
All trials were reviewed to identify those that had a positive result by conventional criteria, defined as a trial identifying one primary prespecified efficacy outcome measure and demonstrating a beneficial effect of study treatment on that measure with a reported P value <0.05. Also accepted were trials with >1 primary prespecified outcome measure if (1) statistical criteria for success were prospectively tightened to adjust for testing of >1 hypothesis and (2) results showed a statistically significant beneficial effect of study treatment on a primary end point.
Trial Design and Quality
Trial design and report quality were assessed for all
studies excluding abstracts with a 100-point quality scale (higher
score represents better quality) adapted specifically for
stroke trials from a previously reported treatment trial quality
checklist.11 12
The scale addresses the following 5 aspects of trial design and
reporting: randomization, outcome, inclusion/exclusion criteria,
description of therapeutic regimen(s), and statistical
analyses. This scale was chosen because, in a previous study,
it demonstrated good interrater reliability and good concordant and
superior discriminant validity compared with 2 alternative
instruments.12 Additional
data abstracted from each report included the dates the study was
performed (if provided), year of publication, and how the trial was
funded (nonprofit/governmental, pharmaceutical, both
nonprofit/governmental and pharmaceutical, not
reported).
Data Extraction Reliability and Statistical
Methods
Data from each report were abstracted by 1
neurologist-reviewer (either C.S.K. or D.S.L.). Trial quality scale
ratings were performed by 3 independent neurologist-reviewers (C.S.K.,
D.S.L., J.L.S.) on a representative sample of 10
trials. Once analysis of this sample showed good interrater
reliability (r ranged from 0.91
to 0.97 between raters), the remaining trials were rated by 1
neurologist-reviewer (C.S.K. or D.S.L.). When a reviewer encountered an
ambiguity or difficulty in abstracting data from a report, the study
was reviewed by the second reviewer, and joint consensus was
achieved.
Because trends over time are best communicated visually, multiple graphs are used to illustrate study findings. For many analyses, inferential statistics are not used because the study encompasses the entire population of relevant trials rather than a representative sample. Quantitative differences over time are instead presented directly. Spearman correlation coefficients were calculated to characterize the degree of change over time for select variables.
| Results |
|---|
|
|
|---|
The total number of trials
(Figure 1
) and total number of patients enrolled
progressively increased over time, with a total of 113 patients
enrolled in 3 trials during 19551959 and 56 888 patients enrolled in
59 trials during 19951999. While growth in the number of trials has
been similar for agents with rheological/antithrombotic and
neuroprotective mechanisms of action, the total number of patients
enrolled in rheological/antithrombotic trials is twice that in
neuroprotective trials, primarily because of the 2 recent megatrials,
IST and CAST.
|
Figure 2
delineates shifting trends in neuroprotective
trials by subclass, demonstrating a proliferation of antiedema agent
trials in the 1970s, modulators of calcium influx in the 1980s and
1990s, and modulators of excitatory amino acids in the 1990s. Overall,
modulators of excitatory amino acids, antiedema agents, and modulators
of calcium influx were the most common neuroprotective target
mechanisms of action.
Figure 3
demonstrates the temporal trends of
rheological/antithrombotic trials by subclass, indicating the increase
in anticoagulant and fibrinolytic trials in the 1980s and 1990s.
Table 1
shows the total number of patients enrolled in
each class and subclass of trials.
|
|
|
Trial Design
Details of overall trial design are provided in
Table 2
. The proportion of trials that were
randomized, double-blind, placebo-controlled, and multicenter steadily
increased over time
(Figure 4
). The proportion of trials designated as
phase 2 has increased over the last 20 years. Of the phase 2 trials, 27
were dose escalation/optimization studies, and 16 primarily involved
surrogate outcome measures (2 were both dose escalation and surrogate
outcome trials combined). Only 3 trials specified a target stroke
mechanism (2 embolic, 1 small-vessel lacunar), while 62 trials
specified a target stroke territory (including 19
hemispheric/supratentorial, 18 carotid, and 20
middle cerebral artery).
|
|
Trial Sample Size
For the 178 studies as a whole, the mean sample size
per trial was 415 patients (median, 73). If the 2 megatrials, IST and
CAST, are excluded, mean sample size was 190 (median, 72). The mean and
median sample sizes for the various types of trials (phase 2, 2/3, 3,
and not specified) are provided in
Table 2
. For neuroprotective trials, mean sample size was
186 (median, 69), and for rheologic/antithrombotic trials, mean sample
size was 656 (183 if IST and CAST are excluded) (median, 71). Trial
sample size progressively increased over time from a mean of 38
(median, 26) in the 1950s to a mean of 661 (median, 113) in the 1990s,
including IST and CAST. This trend was evident for both
rheological/antithrombotic and neuroprotective agents. If only
self-designated efficacy trials (phase 2/3 and phase 3) are considered,
sample size increased from a mean of 34 (median, 35) in 19751979 to a
mean of 1820 (median, 561) in 19951999.
To place in perspective the statistical power afforded by
the sample sizes of reported trials, the following general calculation
was performed: the sample size required to demonstrate a clinically
relevant 5% absolute reduction in the number of patients dead or
disabled at 6 months (reduction from 60% to 55%) with 80% power and
=0.05 is 3148, and the sample size required to demonstrate a
clinically relevant 10% absolute reduction in the number of patients
dead or disabled at 6 months (reduction from 60% to 50%) with 80%
power and
=0.05 is 816. Compared with these cut points, only 2% of
the 118 reported nonphase 2 trials were powered to demonstrate a
clinically relevant 5% benefit, and only 7% were powered to
demonstrate a more substantial 10% benefit.
Time Window and Time to Treatment
Of the 178 trials, 8 studies did not report a maximum
time window permitted for patient enrollment but did provide sufficient
data to indicate that the enrollment period was considered to be in the
acute phase and <15 days from symptom onset. For the remaining 170
trials, the median time window for enrollment was 24 hours (range, 3 to
360 hours). Three trials limited enrollment to patients under 3 hours,
29 trials under 6 hours, 50 trials under 12 hours, and 86 trials under
24 hours. For all trials, the duration of time window allowed for
enrollment has shown a progressive decrease over the last 20 years
(Figure 5
). Seventy-three trials reported average
(generally mean) actual time from symptom to enrollment. Trends to
decrease in time window and actual time to enrollment were similar for
both neuroprotective and rheological/antithrombotic trials.
Nevertheless, even in the most recent 19951999 epoch, the median time
window for neuroprotective trials was 12 hours (range, 4 to 288 hours)
(median time to actual treatment, 14.3 hours), and for
rheological/antithrombotic trials the median time window was 12.0 hours
(range, 3 to 288 hours) (median time to actual treatment, 5.2
hours).
|
In the 20 trials that identified a minimum time that deficits had to be present before a patient could be enrolled, this period varied from 0.5 to 96 hours (mean, 26.8 hours; median, 24 hours).
Outcome Measures
Figure 6
shows the authors judgment of trial outcomes, excluding dose escalation and surrogate outcome trials.
Thirty-two trials were considered beneficial in the authors opinion.
However, according to the generally accepted definition of positive
(eg, a single prespecified primary outcome measure with
P value <0.05), only 3 trials
were positive.15 16 17
The percentage of trials with beneficial, neutral, and harmful outcomes
has not significantly changed over time. None of the 19 trials
appearing only in abstract format met our strict criteria for a
positive outcome.
|
For all 178 trials, the latest outcome time point ranged from 2 to 400 days after symptom onset (mean, 85 days; median, 90 days) and increased from 11 days in the 1950s to 90 days from symptom onset in the 1990s. Forty-seven percent of trials reported using at least 1 validated outcome measure, and this number increased from 0% of trials in the 1950s to 95% in the 1990s.
Quality of Trial Design and Reporting
Trial quality assessed with a formal 100-point scale
was generally modest, with a median score of 63 for the 159 full-length
reports. However, trial quality steadily increased over time, from a
median score of 12 in the 1950s to 72 in the 1990s
(Figure 7
). Since the Consolidated Standards of
Reporting Trials (CONSORT) guidelines for clinical trials were
published in 1996,18 34
full-length reports of acute stroke clinical trials have been
published, 9 of these appearing in journals adhering to CONSORT
guidelines and 25 appearing in journals not adhering to CONSORT
guidelines. Mean quality score for the 9 articles appearing in CONSORT
journals was 88.3, substantially higher than the mean of 71.2 for the
25 articles appearing in non-CONSORT journals.
|
Sponsorship
Of the 178 trials, some type of pharmaceutical
sponsorship was reported in 102, nonprofit/governmental sponsorship was
reported in 42 (22 of these had pharmaceutical sponsorship as well),
and 56 trials did not report funding sources. The percentage of trials
reporting pharmaceutical sponsorship steadily increased over time, from
38% before the 1970s to 68% in the 1990s, while the percentage of
trials reporting nonprofit/governmental sponsorship decreased from 36%
to 16% for the same time periods. Sixty-six percent (58/88) of
neuroprotective trials reported some type of pharmaceutical or
commercial sponsorship, compared with 58% (34/59) of
rheological/antithrombotic trials, 38% (10/26) of trials of agents
with both rheological/antithrombotic and neuroprotective effects, and
0% (0/5) of nonpharmacological
trials.
| Discussion |
|---|
|
|
|---|
Many trends in our data reflect growing awareness and sophistication of clinical trial methodology. Our data demonstrate a dramatic rise in the number of reported clinical trials and in the total number of patients participating in clinical trials over the course of the last 50 years. When we compare the first and last 5-year intervals with reported trials in our survey (19551959 versus 19951999), the number of reported acute ischemic stroke clinical trials increased 20-fold, and the total number of patients enrolled increased 500-fold. This growth reflects advances in basic science yielding ever-increasing numbers of promising agents for clinical testing, growing recognition by the clinical investigative community and regulatory agencies of clinical trials as the gold standard for demonstrating treatment efficacy, and the general remarkable growth in the scale of medical research in the late 20th century.
The substantial growth in trial sample size, particularly for phase 2/3 and phase 3 trials, is noteworthy. Clinical investigators in stroke and other conditions were slow to appreciate the full scope of the problem of type II statistical error (failing to detect a difference between treatments because studies are underpowered). Particularly large sample sizes are required to demonstrate efficacy of agents that provide only a modest benefit. In fact, we calculated that only 2% of the 118 nonphase 2 trials in our analysis were powered to demonstrate a clinically relevant 5% absolute reduction in the number of patients dead or disabled at 6 months, and only 7% were powered to detect a more substantial 10% absolute reduction. The first appearance of "megatrials" in the final years of our data set highlights this observation. The combined results of 2 aspirin megatrials (IST and CAST), enrolling >40 000 patients, were required to demonstrate a very modest benefit from aspirin for acute ischemic stroke. Other agents studied in acute ischemic stroke may also have similar small, but clinically worthwhile, treatment effects that could only be determined by trials of this magnitude.23 Yet, acute stroke clinical trials have generally had sample sizes 100-fold less than those in these 2 megatrials.
We found, in accord with Bath and colleagues,19 steadily progressive improvement in trial quality over time as measured by a formal quality rating scale assessing trial design and reporting. However, our data demonstrate that the quality of acute stroke clinical trial design and reporting remains suboptimal, a finding similar to that in many other fields of medicine.24 25 These persisting deficiencies led to the CONSORT initiative to provide guidelines to improve the quality of reporting of randomized clinical trials.18 We found that acute stroke trials appearing in journals with publication policies requiring conformance to the CONSORT guidelines achieved superior quality ratings. This finding suggests a beneficial impact of these guidelines on quality of trial reporting and lends support to calls for more widespread adoption of the CONSORT statement by all journals.
We found that more than one half of all trials reported some type of pharmaceutical sponsorship, with the proportion steadily increasing over time. These findings are in accord with those of Dorman and colleagues26 ; however, we additionally found that the percentage of trials reporting nonprofit/government sponsorship steadily decreased over time. While advantages of pharmaceutical sponsorship include increased funding for both basic science and clinical trial research, disadvantages include potential for bias in agent selection, data analysis and interpretation, and reporting and publication of results.27 28 29 The growing predominance of pharmaceutical sponsorship, as well as the concomitant decline in the proportion of countervailing nonprofit and governmental support, suggests a greater vulnerability in stroke clinical research to these biases.
We found an increasing trend toward prespecification of a primary end point by which to judge trial success or failure. Yet, only 17% of all trials and 36% of 19951999 trials used such a measure. This failure reflects suboptimal understanding of clinical trial methodology and renders interpretation of trial results subject to bias by increasing the risk of a type I statistical error.
Stroke-Specific Trends
Perhaps most important of the stroke domainspecific
trends in clinical trials that we identified is the decrease in both
the time windows permitted for enrollment and the actual time to
enrollment. This development reflects fundamental changes in the
understanding of the pathophysiology of acute cerebral ischemia
during the 20th century, including the concept that the duration of the
ischemic penumbra is brief. While disagreement about the
optimal time window for enrollment
persists,30 a general
consensus has emerged that thrombolytic and
neuroprotective interventions will only be effective if delivered early
after stroke onset, optimally within the first 3 to 6
hours.31 32 33
Enrollment of patients beyond the critical time window for rescuing
salvageable tissue will lead to a dilution of true treatment effects.
This more sophisticated understanding of the
pathophysiological target of acute stroke therapy
is reflected in the decreasing enrollment windows used by clinical
trials. It is notable, however, that use of extended enrollment time
windows persists. Even in the most recent 19951999 epoch, the median
permitted entry window in neuroprotective trials was 12 hours, and the
median reported time to entry was 14.3 hours.
Our analysis documents the emergence of validated outcome clinical trial rating scales to assess functional end points. Over time, validated scales assessing the degree of neurological deficit, such as the NIHSS, functional activities of daily living, and global disability, have increasingly displaced the poorly specified global judgments or ad hoc scales used by early investigators. Consistent and reproducible scales provide readers with a context for interpretation of clinical results and allow comparison of outcomes across different trials.
Our data also demonstrate an evolution in the times at which final outcome end points have been assessed in acute stroke clinical trials. While initial stroke trials tended to measure treatment efficacy at very early time points (2 to 14 days), later trials tend to have settled on 3 months as the most common outcome time point. This choice optimizes potential for ongoing recovery and neural plasticity within the first months after a stroke, minimizes chances of new or recurrent events that would confound interpretation of effects at very late time points (6 to 12 months), and allows assessment of the performance of the individual in real life, in terms of activities of daily living and independence. Our findings are in accord with and extend those of Duncan and colleagues,34 who analyzed outcome measures in a substantially smaller set of acute stroke trials and did not examine trends over time.
It has been suggested that many acute stroke therapies may only be effective for a particular subtype or mechanism because of variations in underlying pathophysiological processes.35 Rheological/antithrombotic therapies, in particular, may have differential effects on ischemia due to large-vessel atherosclerosis versus cardioembolism versus small-vessel lacunar disease. For example, in the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) trial, no benefit was demonstrated from the low-molecular-weight heparinoid danaparoid, yet a secondary analysis suggested a benefit in patients with large-vessel atherosclerosis.36 Our analysis, however, suggests that the approach of targeting specific stroke mechanisms has not yet been translated into changes in clinical trial design. We found that only 3 of the 178 trials targeted a specific stroke mechanism.
Our analysis of authors opinions of trial outcomes provides one explanation for the large variations in what is considered routine clinical practice in the treatment of acute ischemic stroke.37 Only tPA and aspirin are accepted by expert panels as proven acute stroke therapies, and there remains some dispute even about these.38 39 However, 32 trials from our review were reported by their authors as beneficial, 23 as showing a beneficial trend, and 14 as showing a beneficial outcome in a subgroup of patients. While these trials are not widely accepted as definitive by the stroke research community because of flaws in clinical trial design and small sample size, many have influenced clinical practice to some degree.
The aforementioned observations, taken together, may provide important insights into the success and failure of various acute stroke clinical trials performed to date. For example, our analysis suggests that many trials may have been handicapped in their ability to show agent efficacy because of inadequate sample size, inappropriate time windows permitted for patient enrollment, inappropriate choice of outcome time points, or failure to target a specific stroke mechanism. The success of the National Institute of Neurological Disorders and Stroke (NINDS) trial leading to Food and Drug Administration approval of tPA as the first effective treatment for acute stroke can be attributed, in part, to progress in understanding the pathophysiology of acute cerebral ischemia, combined with a growing appreciation for rigorous clinical trial methodology. Conversely, the persistent failure of trials testing neuroprotective strategies may be attributed, in part, to inappropriate trial design, including inadequate sample size and lengthy time windows allowed for patient enrollment. These observations indicate the crucial and evolving role of appropriate clinical trial design in identifying effective treatments for acute stroke.
In some respects, this systematic review of controlled clinical trials for acute ischemic stroke documents a remarkable record of futility. Of 75 different therapeutic strategies that have been tested in acute stroke clinical trials, only 2 (3%) have been widely accepted as of proven benefit (tPA on the basis of the NINDS trials and aspirin on the basis of the combined results of IST and CAST). While disappointing, nonefficacious trials are of some value. Nonefficacious trials help to identify agents that should not be introduced into or should be used less often in routine clinical practice, protecting patients from expensive and potentially harmful interventions that may have otherwise become established elements of care. Observations in the control arms of nonefficacious trials better delineate the natural history of acute ischemic stroke, allowing iterative improvements in clinical trial design. Analysis of the reasons for failure in nonefficacious trials may provide insights that can be used to generate new avenues for basic research and development of more promising agents.40 41 Finally, meta-analysis of several individually nonefficacious trials may sometimes permit identification of subtle beneficial effects of agents.
The generally dismal record of clinical trials in the domain of acute stroke treatment is a departure from the norm in clinical medicine. In a review of 1041 randomized controlled clinical trials for a variety of medical conditions, Chalmers et al42 found that 55% of reported trials showed a positive result, with a documented P value <0.05. This contrasts with 23% of nonphase 2 trials with a positive result in the reporting authors opinion and 3% of nonphase 2 trials with a positive result on a prespecified primary end point with a documented P value <0.05 among the acute ischemic stroke trials we reviewed. Moreover, the statistical significance of the 3 positive trials must be interpreted with caution given the possibility that 1 in 20 trials will be positive by chance. This analysis suggests that acute ischemic stroke, biologically, may be more refractory to therapy than many other medical conditions. The field of acute stroke provides several formidable challenges, including the narrow time window of viability of the ischemic brain, complex pathophysiological processes that depend on a variety of underlying mechanisms (eg, thromboembolism versus hemodynamic ischemia), and dilemmas in diagnosis (a multitude of clinical presentations and the need to differentiate ischemia from hemorrhage).
Our analysis is limited by several factors. In rating trial quality, good interrater reliability was established at the beginning of the study but not formally rechecked at intervals during data abstraction, raising the possibility of coder drift. The joint working of the reviewers when ambiguous or difficult abstractions arose helped to diminish this possibility. We incorporated only trials with a final report appearing in the English language. Our database therefore underrepresents clinical trial methodology and therapies used in countries less likely to publish their results in English-language journals.43 44 In addition, an unknown number of unpublished trials were not included in our analysis. There is a recognized need to gather data from unreported trials because these studies may also provide important lessons to other researchers.28 45
In conclusion, our analysis demonstrated several accelerating trends in acute stroke clinical trial design in the last century, including increased sample size, decreased time windows for enrollment, and improvement in clinical trial methodology/reporting over time. These trends have reflected 2 important themes: (1) emerging understanding of the pathophysiology of cerebral ischemia and the imperative for treatment initiation within a critical time window and (2) increasing awareness of the importance of sound trial design. These findings afford important lessons to clinical trialists in the 21st century, including the need to calculate and recruit adequate sample sizes, the use of appropriate time windows, the importance of rigorous clinical trial design and reporting, and the need to develop comprehensive registries of all ongoing trials.
|
| Acknowledgments |
|---|
Received August 18, 2001; revision received January 17, 2001; accepted March 16, 2001.
| References |
|---|
|
|
|---|
3. Bulpitt CJ. Randomised Controlled Clinical Trials. 2nd ed. Boston, Mass: Kluwer Academic Publishers; 1996.
4.
NINDS rt-PA Stroke
Group. Tissue plasminogen activator for acute
ischemic stroke. N Engl
J Med. 1995;333:15811587.
5. Cochrane Stroke Group CRG. The Cochrane Library, issue 4, 1999. Oxford, UK: Update Software.
6. The Ottawa Stroke Trials Registry Collaborative Group. The Ottawa Stroke Trials Registry Collaborative Group and the Development of the Ottawa Stroke Trials Registry (OSTR). Control Clin Trials. 1994;15:503511.[Medline] [Order article via Infotrieve]
7. Cochrane Controlled Trials Register. The Cochrane Library, issue 4, 1999. Oxford, UK: Update Software.
8. Dorman PJ, Counsell C, Sandercock PA. Recently developed neuroprotective therapies for acute stroke: a qualitative systematic review of clinical trials. CNS Drugs. 1996;5:457474.
9.
Brott T, Adams HP
Jr, Olinger CP, Marler JR, Barsan WG, Biller J, Spilker J, Holleran R,
Eberle R, Hertzberg V. Measurements of acute cerebral infarction: a
clinical examination scale.
Stroke. 1989;20:864870.
10. Mahoney F, Barthel D. Functional evaluation: the Barthel Index. Md State Med J. 1965;14:6165.[Medline] [Order article via Infotrieve]
11. Detsky AS, Naylor DD, ORourke K, McGeer JJ, LAbbe KA. Incorporating variations in the quality of individual randomized trials into meta-analysis. J Clin Epidemiol. 1992;45:225265.
12. Liebeskind DS, Kidwell CS, Saver JL. Neuroprotective therapy for acute ischemic stroke: quality assessment of controlled trials. The Lancet Conference 1998; October 1516, 1998; Montreal, Canada.
13. International Stroke Trial Collaborative Group. The International Stroke Trial (IST): a randomised trial of aspirin, subcutaneous heparin, both, or neither among 19435 patients with acute ischaemic stroke. Lancet. 1997;349:15691581.[Medline] [Order article via Infotrieve]
14. CAST (Chinese Acute Stroke Trial) Collaborative Group. CAST: randomised placebo-controlled trial of early aspirin use in 20,000 patients with acute ischaemic stroke. Lancet. 1997;349:16411649.[Medline] [Order article via Infotrieve]
15. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med. 1995;333:15811587.
16.
Furlan A,
Higashida R, Wechsler L, Gent M, Rowley H, Kase C, Pessin M, Ahuja A,
Callahan F, Clark WM, Silver F, Rivera F. Intra-arterial
prourokinase for acute ischemic stroke: the PROACT II study: a
randomized controlled trial: Prolyse in Acute Cerebral Thromboembolism.
JAMA. 1999;282:20032011.
17.
Kay R, Wong KS,
Yu YL, Chan YW, Tsoi TH, Ahuja AT, Chan FL, Fong KY, Law CB, Wong A.
Low-molecular-weight heparin for the treatment of acute
ischemic stroke. N Engl
J Med. 1995;333:15881593.
18.
Begg C, Cho M,
Eastwood S, Horton R, Moher D, Olkin I, Pitkin R, Rennie D, Schulz KF,
Simel D, Stroup DF. Improving the quality of reporting of randomized
controlled trials: the CONSORT statement.
JAMA. 1996;276:637639.
19.
Bath FJ, Owen VE,
Bath PM. Quality of full and final publications reporting acute stroke
trials: a systematic review.
Stroke. 1998;29:22032210.
20.
Nicolucci A,
Grilli R, Alexanian AA, Apolone G, Torri V, Liberati A. Quality,
evolution, and clinical implications of randomized, controlled trials
on the treatment of lung cancer: a lost opportunity for
meta-analysis. JAMA. 1989;262:21012107.
21.
Scherer RW,
Dickersin K, Langenberg P. Full publication of results initially
presented in abstracts: a meta-analysis.
JAMA. 1994;272:158162.
22. Machin D, Stenning SP, Parmar MK, Fayers PM, Girling DJ, Stephens RJ, Stewart LA, Whaley JB. Thirty years of Medical Research Council randomized trials in solid tumours. Clin Oncol. 1997;9:100114.
23. Sandercock P. What questions can large, simple trials answer? Cerebrovasc Dis. 1998;8(suppl 2):3036.
24.
Moher D, Dulberg
CS, Wells GA. Statistical power, sample size, and their reporting in
randomized controlled trials.
JAMA. 1994;272:122124.
25. Emerson JD, Burdick E, Hoaglin DC, Mosteller F, Chalmers TC. An empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials. Control Clin Trials. 1990;11:339352.[Medline] [Order article via Infotrieve]
26.
Dorman PJ,
Counsell C, Sandercock P. Reports of randomized trials in acute stroke,
1955 to 1995: what proportions were commercially sponsored?
Stroke. 1999;30:19951998.
27. Liebeskind DS, Kidwell CS, Saver JL. Empiric evidence of publication bias affecting acute stroke clinical trials. Stroke. 1999;30:268. Abstract.
28. Easterbrook PJ, Berlin JA, Gopalan R, Mathews DR. Publication bias in clinical research. Lancet. 1991;337:867872.[Medline] [Order article via Infotrieve]
29.
Dickersin K, Min
YL, Meinert CL. Factors influencing publication of research results:
follow-up of applications submitted to two institutional review boards.
JAMA. 1992;267:374378.
30.
Dorman PJ,
Sandercock PA. Considerations in the design of clinical trials of
neuroprotective therapy in acute stroke.
Stroke. 1996;27:15071515.
31. Marchal G, Serrati C, Rioux P, Petit-Taboué MC, Viader F, de la Sayette V, Le Doze F, Lochon P, Derlon JM, Orgogozo JM, et al. PET imaging of cerebral perfusion and oxygen consumption in acute ischaemic stroke: relation to outcome. Lancet. 1993;341:925927.[Medline] [Order article via Infotrieve]
32. Zivin JA. Factors determining the therapeutic window for stroke. Neurology. 1998;50:559603.
33.
Dyker AG, Lees
KR. Duration of neuroprotective treatment for ischemic stroke.
Stroke. 1998;29:535542.
34.
Duncan PW,
Jorgensen HS, Wade DT. Outcome measures in acute stroke trials: a
systematic review and some recommendations to improve practice.
Stroke. 2000;31:14291438.
35.
Caplan LR, Mohr
JP, Kistler JP, Koroshetz W. Should thrombolytic
therapy be the first-line treatment for acute ischemic stroke?
Thrombolysisnot a panacea for ischemic stroke.
N Engl J Med. 1997;337:13091310.
36.
The Publications
Committee for the Trial of ORG 10172 in Acute Stroke Treatment (TOAST)
Investigators. Low molecular weight heparinoid, ORG 10172
(danaparoid), and outcome after acute ischemic stroke: a
randomized controlled trial.
JAMA. 1998;279:12651272.
37.
Counsell C,
Warlow C, Sandercock P, Fraser H, van Gijn J. The Cochrane
Collaboration Stroke Review Group: meeting the need for systematic
reviews in stroke care. Stroke. 1995;26:498502.
38. Barer D. Interpretation of IST and CAST stroke trials: International Stroke Trial/Chinese Acute Stroke Trial. Lancet. 1997;350:440.
39. Wardlaw JM, Warlow CP, Counsell C. Systematic review of evidence on thrombolytic therapy for acute ischaemic stroke. Lancet. 1997;350:607614.[Medline] [Order article via Infotrieve]
40. Lees KR. Cerestat and other NMDA antagonists in ischemic stroke. Neurology. 1997;49(suppl 4):S66S69.
41.
Dyker AG, Lees
KR. Safety and tolerability of GV150526 (a glycine site
antagonist at the
N-methyl-D-aspartate
receptor) in patients with acute stroke.
Stroke. 1999;30:986992.
42. Chalmers TC, Levin H, Sacks HS, Reitman D, Berrier J, Nagalingam R. Meta-analysis of clinical trials as a scientific discipline, I: control of bias and comparison with large co-operative trials. Stat Med. 1987;6:315328.[Medline] [Order article via Infotrieve]
43.
Egger M,
Smith GD. Meta-analysis: bias in location and selection of
studies. BMJ. 1998;316:6166.
44. Moher D, Fortin P, Jadad AR, Juni P, Klassen T, Le Lorier J, Liberati A, Linde K, Penna A. Completeness of reporting of trials published in languages other than English: implications for conduct and reporting of systematic reviews. Lancet. 1996;347:363366.[Medline] [Order article via Infotrieve]
45.
Smith R, Roberts I. An amnesty for unpublished trials. BMJ. 1997;315:622.
This article has been cited by other articles:
![]() |
H. Karatas, Y. Aktas, Y. Gursoy-Ozdemir, E. Bodur, M. Yemisci, S. Caban, A. Vural, O. Pinarbasli, Y. Capan, E. Fernandez-Megia, et al. A Nanomedicine Transports a Peptide Caspase-3 Inhibitor across the Blood-Brain Barrier and Provides Neuroprotection J. Neurosci., November 4, 2009; 29(44): 13761 - 13769. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. Mandava and T. A. Kent A Method to Determine Stroke Trial Success Using Multidimensional Pooled Control Functions Stroke, May 1, 2009; 40(5): 1803 - 1810. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. S. Kidwell, K. R. Lees, K. W. Muir, C. Chen, S. M. Davis, D. A. De Silva, C. J. Weir, S. Starkman, J. R. Alger, J. L. Saver, et al. Results of the MRI Substudy of the Intravenous Magnesium Efficacy in Stroke Trial Stroke, May 1, 2009; 40(5): 1704 - 1709. [Abstract] [Full Text] [PDF] |
||||
![]() |
X. Xu, L. Ye, and Q. Ruan Environmental Enrichment Induces Synaptic Structural Modification After Transient Focal Cerebral Ischemia in Rats Experimental Biology and Medicine, March 1, 2009; 234(3): 296 - 305. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. Rink, G. Christoforidis, A. Abduljalil, M. Kontzialis, V. Bergdall, S. Roy, S. Khanna, A. Slivka, M. Knopp, and C. K. Sen Minimally invasive neuroradiologic model of preclinical transient middle cerebral artery occlusion in canines PNAS, September 16, 2008; 105(37): 14100 - 14105. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. L. Saver Novel End Point Analytic Techniques and Interpreting Shifts Across the Entire Range of Outcome Scales in Acute Stroke Trials Stroke, November 1, 2007; 38(11): 3055 - 3062. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. Sakellaridis and D. Panagopoulos Significance of Experimental Infarct Size as an Indicator of Therapeutic Efficacy in Humans Stroke, September 1, 2007; 38(9): e89 - e90. [Full Text] [PDF] |
||||
![]() |
D. C. Hess NXY-059: A Hopeful Sign in the Treatment of Stroke Stroke, October 1, 2006; 37(10): 2649 - 2650. [Full Text] [PDF] |
||||
![]() |
D. S. Liebeskind, C. S. Kidwell, J. W. Sayre, and J. L. Saver Evidence of publication bias in reporting acute stroke clinical trials. Neurology, September 26, 2006; 67(6): 973 - 979. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. L. Saver Does the Merci Retriever Work?: For Stroke, May 1, 2006; 37(5): 1340 - 1341. [Full Text] [PDF] |
||||
![]() |
J. S. Elkins, T. Khatabi, L. Fung, J. Rootenberg, and S. C. Johnston Recruiting Subjects for Acute Stroke Trials: A Meta-Analysis Stroke, January 1, 2006; 37(1): 123 - 128. [Abstract] [Full Text] [PDF] |
||||
![]() |
Y. Y. Palesch, B. C. Tilley, D. L. Sackett, K. C. Johnston, and R. Woolson Applying a Phase II Futility Study Design to Therapeutic Stroke Trials Stroke, November 1, 2005; 36(11): 2410 - 2414. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Fisher and T. Tatlisumak Use of Animal Models Has Not Contributed to Development of Acute Stroke Therapies: Con Stroke, October 1, 2005; 36(10): 2324 - 2325. [Full Text] [PDF] |
||||
![]() |
M. Fisher and for the Stroke Therapy Academic Industry Roundtabl Enhancing the Development and Approval of Acute Stroke Therapies: Stroke Therapy Academic Industry Roundtable Stroke, August 1, 2005; 36(8): 1808 - 1813. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Krams, K. R. Lees, and D. A. Berry The Past Is the Future: Innovative Designs in Acute Stroke Therapy Trials Stroke, June 1, 2005; 36(6): 1341 - 1347. [Full Text] [PDF] |
||||
![]() |
Y. Murakami, H. Takamatsu, A. Noda, K. Osoda, R. Ichise, M. Tatsumi, K. Tabata, T. Sawamoto, and S. Nishimura Pharmacokinetic Animal PET Study of FK506 as a Potent Neuroprotective Agent J. Nucl. Med., November 1, 2004; 45(11): 1946 - 1949. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. J. Weir, M. Kaste, K. R. Lees, and for the Glycine Antagonist in Neuroprotection Int Targeting Neuroprotection Clinical Trials to Ischemic Stroke Patients With Potential to Benefit From Therapy Stroke, September 1, 2004; 35(9): 2111 - 2116. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. C. Fagan, D. C. Hess, E. J. Hohnadel, D. M. Pollock, and A. Ergul Targets for Vascular Protection After Acute Ischemic Stroke Stroke, September 1, 2004; 35(9): 2220 - 2225. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. L. Saver, C. Kidwell, M. Eckstein, S. Starkman, and for the FAST-MAG Pilot Trial Investigators Prehospital Neuroprotective Therapy for Acute Stroke: Results of the Field Administration of Stroke Therapy-Magnesium (FAST-MAG) Pilot Trial Stroke, May 1, 2004; 35(5): e106 - e108. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. H. Danton and W. D. Dietrich The Search for Neuroprotective Strategies in Stroke AJNR Am. J. Neuroradiol., February 1, 2004; 25(2): 181 - 194. [Full Text] [PDF] |
||||
![]() |
S C Lewis and C P Warlow How to spot bias and other potential problems in randomised controlled trials J. Neurol. Neurosurg. Psychiatry, February 1, 2004; 75(2): 181 - 187. [Full Text] [PDF] |
||||
![]() |
T. Crocco, T. Gullett, S. M. Davis, N. Flores, L. Sauerbeck, E. Jauch, B. Threlkeld, B. Pio, M. Ottaway, A. Pancioli, et al. Feasibility of Neuroprotective Agent Administration by Prehospital Personnel in an Urban Setting Stroke, August 1, 2003; 34(8): 1918 - 1922. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Fisher Recommendations for Advancing Development of Acute Stroke Therapies: Stroke Therapy Academic Industry Roundtable 3 Stroke, June 1, 2003; 34(6): 1539 - 1546. [Abstract] [Full Text] [PDF] |
||||
![]() |
D. J. Gladstone, S. E. Black, and A. M. Hakim Toward Wisdom From Failure: Lessons From Neuroprotective Stroke Trials and New Therapeutic Directions Stroke, August 1, 2002; 33(8): 2123 - 2136. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. G. Sorensen Apparently, Diffusion Coefficient Value and Stroke Treatment Remains Mysterious AJNR Am. J. Neuroradiol., February 1, 2002; 23(2): 177 - 178. [Full Text] [PDF] |
||||
![]() |
J. Fang, M. H. Alderman, and J. V. Tu Trend of Stroke Hospitalization, United States, 1988-1997 Editorial Comment Stroke, October 1, 2001; 32(10): 2221 - 2226. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2001 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |