| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2007;38:388.)
© 2007 American Heart Association, Inc.
Original Contributions |
From Clinical Neurosciences, University of Edinburgh, Scotland.
Correspondence to Dr Malcolm Macleod, Clinical Neurosciences, University of Edinburgh, Western General Hospital, Crewe Road, Edinburgh, EH4 2XU, Scotland. E-mail malcolm{at}apoptosis.freeserve.co.uk
| Abstract |
|---|
|
|
|---|
Methods Systematic review and meta-analysis of studies describing the efficacy of tirilazad in animal models of focal ischemia, in which outcome was measured as infarct volume and/or neurological score. Weighted mean difference random effects meta-analysis was used to measure overall efficacy in prespecified subgroups.
Results Eighteen studies describing outcome in 544 animals were identified. Study quality (median score, 5/10; interquartile range, 4 to 6) was similar to that seen in systematic reviews of other candidate neuroprotective drugs. Tirilazad reduced infarct volume by 29.2% (95% confidence interval 21.1% to 37.2%) and improved neurobehavioral score by 48.1% (95% confidence interval 29.3% to 66.9%).
Conclusion Tirilazad may have substantial efficacy in animal models of stroke, but this conclusion must be qualified because of the presence of potential sources of bias.
Key Words: meta-analysis neuroprotection stroke systematic review tirilazad
| Introduction |
|---|
|
|
|---|
Explaining the discrepancy between efficacy in animal studies and lack of efficacy in clinical trials might lead to important insights and guide the future design of studies of both animal and human strokes.5 There are a number of potential reasons for such differences, including systematic errors in the ways in which data from promising animal studies are used to inform the design of clinical trials. Specifically, the animal data may be falsely positive, reporting a protective effect in which no biological efficacy exists; or the clinical trials may be falsely neutral, reporting no effect when clinical efficacy does in fact exist.
Systematic review of all the available animal evidence reduces selection bias and random error in the assessment of drug efficacy. Useful information can still be extracted even when identified animal studies are too heterogeneous (because of differences in dose, time, species) to provide a reliable estimate of "average" efficacy. Stratified meta-analysis (in which studies are analyzed by dose or time or species) can describe dose-response relationships, time dependence of efficacy, and the impact of study quality and study design characteristics.
These techniques have proved useful in the analysis of data from clinical trials and have been advocated for the analysis of data from animal experiments.5,6 Animal models of stroke lend themselves to this approach, as evidenced by the publication of a growing number of such analyses in recent years.713
During ischemia and reperfusion, free radicals play an important role in inducing cerebral injury through effects on DNA, on mitochondria, and through the effects of lipid peroxidation.14 Tirilazad is a synthetic lipid-soluble 21-aminosteroid with antioxidant effects that is proposed to interact with the lipid peroxidation cascade at various stages including: (1) scavenging of hydroxyl and lipid peroxyl radicals; (2) maintenance of endogenous antioxidant levels; and (3) prevention of propagation of lipid peroxidation by membrane stabilization.15
Experimental animal data had suggested tirilazad as a treatment for ischemic stroke, but it was subsequently demonstrated to increase death and dependancy in a meta-analysis of clinical studies.16 Here we present a systematic review and meta-analysis of the efficacy of tirilazad in experimental stroke. We have set out to establish the quality of the animal studies; the limits (dose response, time dependence) to efficacy; the impact of study characteristics on efficacy; and to establish the fidelity with which clinical trial design reflected the conditions under which efficacy was seen in animals.
| Methods |
|---|
|
|
|---|
Criteria for Inclusion
Two investigators (E.S., P.W.) independently extracted those publications identified here that described controlled studies of tirilazad given in models (whole live animals excluding humans) of focal cerebral ischemia induced by occlusion of the middle or anterior cerebral artery or their branches, where tirilazad was administered by any route and outcome compared with animals receiving placebo or no tirilazad. Disagreements were resolved in discussion with a third investigator (M.M.).
End Points Considered
The primary outcome measure was infarct area or volume (determined histologically or by cross-sectional imaging), with secondary outcome measures of death and of neurobehavioral score.
Methods of the Review
Quality Assessment
There was no quality threshold for inclusion. Study quality was assessed against our published ten item checklist9 comprising: (1) publication in peer reviewed journal; (2) statement of control of temperature; (3) randomization to treatment or control; (4) blinded induction of ischemia; (5) blinded assessment of outcome; (6) avoidance of anesthetics with marked intrinsic neuroprotective properties; (7) use of animals with hypertension or diabetes; (8) sample size calculation; (9) statement of compliance with regulatory requirements; and (10) statement regarding possible conflicts of interest.
Data Extraction
From each source we identified individual comparisons in which outcome was measured in a group of animals receiving a specified dose(s) of drug at a specified time(s) and compared with outcome in a control group. When the treatment group received more than one intervention, this was recorded. For each comparison and for each of treatment and control group, we extracted data for number per group, mean outcome, and its standard deviation. When an outcome was measured serially, only the last measure was used. When data were given graphically, we contacted authors seeking data; when this was not available we estimated values by measurement from publications. Data were extracted onto a data extraction form by 2 reviewers independently, and differences were resolved by discussion.
We also collected other relevant data including anesthetic used, time of outcome measurement, and method of induction of ischemia, as well as the individual component items of the quality checklist.
Analysis
For continuous variables (infarct volume, neurobehavioral score), we calculated an overall weighted mean difference with a random effects model. When a single control group served multiple treatment groups, the size of the control group entered to the meta-analysis was adjusted by division by the number of treatment groups served. Meta-analyses were conducted stratified according to: drug dose; time of administration; study quality; components of study quality checklist; presence of cotreatments; duration of ischemia; method of induction of ischemia; anesthetic used; use of mechanical ventilation; species and gender of animal used; outcome measure used; and interval to quantification of outcome. The significance of differences between n groups was assessed by partitioning heterogeneity and by using the
2 distribution with n1 degrees of freedom (df). To allow for multiple comparisons we set our significance level at P<0.001. Publication bias was assessed with a funnel plot and with the Egger method.7
| Results |
|---|
|
|
|---|
|
Study Quality and Publication Bias
No study described a sample size calculation or contained a statement of potential conflict of interest, although 2 of the 18 publications were directly funded by and a further 2 studies describe some form of assistance from the manufacturers of tirilazad. The median quality score was 5 (interquartile range, 4 to 6), and whereas this compares favorably with similar reviews for other candidate neuroprotective drugs, this still represents an important potential source of bias (Table 2).
|
Efficacy
Tirilazad reduced infarct volume by 29.2% (95% confidence interval, 21.1% to 37.2%; 34 comparisons; Figure 1a), and improved neurobehavioral score by 48.1% (95% confidence interval, 29.3% to 66.9%; 16 comparisons; Figure 1b). There was substantial heterogeneity for both outcome measures (infarct volume,
2=120, df=33, P<1012; neurobehavioral score,
2=35.8, df=15, P=0.002). Only one study reported mortality data and was not analyzed further.
|
Data for infarct volume were used for prespecified stratified meta-analyses. There was a narrow window of therapeutic effect, with doses between 3 and 9.9 mg/kg giving maximum efficacy and doses only slightly lower (1 to 2.9 mg/kg) or higher (10 to 29 mg/kg) being only half as effective (P<0.001; Figure 2a). Although there was no significant relationship between delay to treatment and efficacy it is interesting to note that maximum efficacy was seen when treatment was given before the onset of ischemia, with a trend for efficacy to fall thereafter with time (Figure 2b). Although the longest interval between stroke onset and initiation of treatment was 6 hours, the median was only 10 minutes. There was no relationship between study quality and treatment effect. Efficacy was higher in temporary occlusion than in either permanent or thrombotic occlusion models (
2=16.5, df=2, P<0.001; Figure 3a), and was lower in the presence of comorbidity (4 comparisons undertaken among spontaneously hypertensive rats; no other comorbidities were tested) than in healthy animals (
2=26.6, df=1, P<0.001; Figure 3b). Tirilazad nonsignificantly increased infarct volume in cats, whereas it improved outcome in both rats and rabbits (
2=14.9, df=2, P for difference between species <0.001; Figure 3c). There was no significant effect of the anesthetic used; the use of cotreatments; single or multiple dosages; the route of drug administration; the sex of animal used; the method of quantification of infarct volume; or the use of mechanical ventilation (not shown).
|
|
| Discussion |
|---|
|
|
|---|
Lack of Concordance With Human Studies
This review was conducted in the context of a project examining concordance between animal and human studies for six interventions. Clinical trials in ischemic stroke showed tirilazad increased death and disability.16 While one explanation for this lack of concordance might indeed be that the animal experiments were falsely positively as described, comparison with tPA, a drug with efficacy in both animal and human strokes, suggests 2 further plausible explanations. First, the animal data suggest that tirilazad is effective over a narrow dose range (3 to 10 mg/kg). Other aspects of study design may confound this finding, and a single experiment of adequate power would be required to confirm it; however, these data are consistent with the narrow effective dose range reported for another free radical scavenger, PEG-SOD.36 A narrow dose range for tirilazad would contrast with tPA, for which efficacy in animals was seen across a broad range of doses (Sena E and Macleod M, unpublished observations, 2006). Clinical trials of tirilazad used a broad range of doses (0.15 to 15 mg/kg), with no difference in outcome when patients were dichotomized to those receiving more than or less than 6 mg/kg/d; if the narrow effective dose range suggested by our analysis also exists in humans, this might explain why the clinical studies were negative.
Second, the interval between stroke onset and the initiation of treatment was substantially longer in clinical studies (median,
5 hours) than in the animal studies (median, 10 minutes). This contrasts with tPA, for which the median delay to treatment in the animal studies, 90 minutes, is broadly similar to the time window in which clinical efficacy has been shown (within 180 minutes).
Because of concerns that findings in small laboratory animals may not generalize to much larger human brains, the STAIR criteria includes the demonstration of efficacy in larger animals. The only gyrencephalic species in which tirilazad was tested was in cats, in which no efficacy was seen. Whereas this may reflect lack of efficacy in larger animals, this was also the only experiment using female animals; and partitioning heterogeneity by sex was of borderline significance (P=0.006). Establishing the source of this lack of efficacy would require direct head-to-head comparison.
Robustness of These Conclusions
Although we prespecified our choice of stratification variables and set a very stringent significance level, some of our results may have been attributable to the play of chance; therefore, our observations should be interpreted with caution. This meta-analysis has other weaknesses. First, meta-analysis can only include available data, and publication bias may result in our analyses overestimating the efficacy of tirilazad. Furthermore, although we consider that our search strategy is likely to have ascertained most of the relevant publications, it has yet to be validated.
We elected to use weighted mean difference meta-analysis. We consider that standardized mean difference meta-analysis, although appropriate for clinical studies that individually have large numbers of participants, is less suited to animal studies in which the number of subjects is substantially lower. This is because the observed (sample) standard deviation used in standardization will be a poorer estimate of the population standard deviation when sample size is smaller. However, direct comparisons of the strengths and weaknesses of each statistical approach in this context have not yet been performed.
Third, our analysis is observational and should only be considered as hypothesis-generating. There were insufficient data to allow multiple linear regression analysis that could help disentangle the impact of different factors. However, by pooling such data from reviews of different neuroprotectants, it should be possible to identify those aspects of study quality for instance most closely associated with overstatement of efficacy. This work is underway.
This post-hoc approach allows potential sources of bias to be taken into account in the interpretation of published evidence for efficacy for candidate neuroprotective drugs. However, we believe that a better solution would be for the design, conduct, and reporting of animal studies to be refined to minimize the impact of these sources of bias.
| Acknowledgments |
|---|
Sources of Funding
We acknowledge the financial support of the National Coordinating Centre for Research Methodology of the UK Department of Health.
Disclosures
None.
Received May 1, 2006; revision received June 23, 2006; accepted July 17, 2006.
| References |
|---|
|
|
|---|
Related Article:
This article has been cited by other articles:
![]() |
J. Minnerup, J. Heidrich, J. Wellmann, A. Rogalewski, A. Schneider, and W.-R. Schabitz Meta-Analysis of the Efficacy of Granulocyte-Colony Stimulating Factor in Animal Models of Focal Cerebral Ischemia Stroke, June 1, 2008; 39(6): 1855 - 1861. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. A. Crossley, E. Sena, J. Goehler, J. Horn, B. van der Worp, P. M.W. Bath, M. Macleod, and U. Dirnagl Empirical Evidence of Bias in the Design of Experimental Stroke Studies: A Metaepidemiologic Approach Stroke, March 1, 2008; 39(3): 929 - 934. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. L. Gibson, L. J. Gray, P. M. W. Bath, and S. P. Murphy Progesterone for the treatment of experimental brain injury; a systematic review Brain, February 1, 2008; 131(2): 318 - 328. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2007 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |