Systematic Review and Meta-Analysis of Interventions Tested in Animal Models of Lacunar Stroke
Background and Purpose—A total of 25% of strokes are lacunar, and these are pathophysiologically different from large artery strokes. Despite emerging evidence of a substantial impact on physical disability and dementia, little attention has been paid to the development of specific treatments. The optimal use of the animal models of lacunar stroke used to test candidate interventions is not known.
Methods—We conducted a systematic review and meta-analysis of studies testing candidate interventions in animal models of lacunar stroke. We used random-effects meta-analysis to assess the impact of study characteristics and trim and fill to seek evidence of publication bias.
Results—The efficacy of 43 distinct interventions was described in 57 publications. The median number of quality checklist items scored was 3 of 8 (interquartile range, 2–4). Many models reflected mechanisms of limited relevance to lacunar stroke. Meta-analysis of results from 27 studies showed that on average, infarct size and neurobehavioral outcome were improved by 34.2% (24.1–44.2) and 0.82 standardized mean difference (0.51–1.14), respectively. Four interventions improved both infarct size and neurobehavioral outcome but there were insufficient data for this finding to be considered robust. For infarct size, efficacy was lower in studies reporting blinding and higher in studies reporting randomization. For neurobehavior, efficacy was lower in randomized studies. For infarct size there was evidence of publication bias.
Conclusions—No intervention has yet been tested in sufficient range and depth to support translation to clinical trial. There is limited reporting of measures to reduce the risk of bias and evidence for a substantial publications bias.
Lacunar strokes are small subcortical strokes caused by occlusion of single penetrating arteries. Although thromboembolism is a common cause of large artery stroke, the lacunar hypothesis,1 much debated,2,3 proposes that key mechanisms in lacunar pathology are microatheroma and lipohyalinosis, and that thromboembolic lacunar stroke is uncommon.4 Microatheroma are lipid-containing plaques, which are thought to accumulate in the parent main intracranial artery, such as the middle cerebral artery, and affect the origin of penetrating arteries, or develop in the proximal-penetrating arterioles themselves. Lipohyalinosis is a small-vessel pathology characterized by abnormal endothelial architecture and fibrosis, leading to thickening of the vessel wall and irregular luminal diameter.5 Despite associations with inflammatory endothelial dysfunction and blood–brain barrier disruption,3,6,7 the causes of these small-vessel changes are poorly understood.
Around 25% of all ischemic strokes are lacunar,8 and although they have an apparently good functional prognosis compared with cortical ischemic stroke, similar proportions of patients have poststroke cognitive impairment and long-term studies suggest that they identify cerebral small vessel disease (SVD), which puts patients at high risk of recurrent ischemic strokes9 and of cognitive decline.10,11
However, there is incomplete understanding of the precise mechanisms that lead to the pathology described above, and current therapeutic strategies are limited. This occurs against a background of translational failure, where many interventions reported to improve outcome in animal models of ischemic stroke more generally are not effective in human clinical trial.12
It may be that animal models do not model human disease with sufficient validity to guide drug development; or that they do have this external validity but their conduct and reporting make them a poor guide, in practice, to support clinical drug development and trial design. For several animal models of neurological disease, systematic review and meta-analysis have given useful insights to the impact of study design and quality13–16 within the limitations inherent in combining data from different studies. Here, we assess the evidence supporting the efficacy of different treatments tested in animal models of lacunar stroke on behavioral and structural outcomes, with particular focus on the reporting and impact of measures to reduce bias and on the likelihood of publication bias.
We searched Medline (from 1950), ISI Web of Science (from 1969), and EMBASE (from 1980) on March 6, 2012, with a strategy to identify animal experiments modeling lacunar stroke (including but not limited to intervention studies) modified from a previously reported search strategy18 and described in detail in the online-only Data Supplement. In addition, we searched (March 23, 2012) for publications citing original studies identified in the previous systematic review of the animal modeling of lacunar stroke.18 There were no language restrictions.
Inclusion and Exclusion Criteria
We included experiments where animals had been exposed to a lesion modeling lacunar stroke (defined as those caused by occlusion or stenosis or other disease of perforating blood vessels and leading to presumed ischemic lesions in focal subcortical areas)18,19 and where outcome in a cohort of animals subject to an intervention was compared with that in a control group. We included studies that quantified structural (lesion size) or functional (behavioral) outcomes. We excluded studies where single lesions were >1 of 140th of total brain volume, where lesions were not induced by single-vessel mechanisms (eg, traumatic brain injury), hemorrhages, and those involving partial or complete occlusion of the middle cerebral or common carotid artery. We excluded transgenic studies and those modeling neonatal hypoxia/ischemia. We excluded studies where the intervention was given with the expressed intention of worsening rather than improving outcome.
We recorded the author, year of publication, intervention used and dose, type of animal (including species, strain, and sex), type of intervention, time of administration and of outcome assessment, outcome measure used, mean outcome, SD or SE, number of animals per group, presence of salt loading, method of stroke induction, anesthetic used, study quality assessment parameters, and reporting of measures to avoid bias (see below).
We defined treatment comparisons as those which compared the outcome in control and treated animals. For studies using spontaneously hypertensive stroke–prone rats (SHRSPs), we did not record time of administration or outcome assessment, as there was no clear time point at which stroke was induced. Where >1 intervention was given, we considered this combination to be a separate, unique intervention. Where treatment was administered in multiple doses, we considered treatment to occur at the time of the first dose and the dose to be the sum of all doses administered in the first 24 hours. Where data from multiple brain slices were reported, we included only the infarct size from the slice with the largest corresponding infarct in control animals. Where neurobehavioral outcomes were reported for >1 time point, we included only the latest time of assessment as the most clinically relevant end point. Where data were presented graphically, we contacted authors seeking further information, and if necessary we measured values from graphs (Universal Desktop Ruler, version 2.9).
Risk of bias was assessed using 8 of 10 Collaborative Approach to Meta-Analysis and Review of Animal Data from Experimental Studies (CAMARADES) checklist items.13 We did not include statement of control of temperature or avoidance of anesthetics with marked intrinsic properties as we considered these less relevant in models of lacunar stroke. We recorded whether the study was (1) published in a peer-reviewed journal, and whether it reported (2) randomized allocation to experimental group; (3) the blinding of the group allocation during the conduct of the experiment; (4) the blinded assessment of outcome; (5) the use of animals with relevant comorbidities; (6) a statement of sample size calculation; (7) a statement of compliance with regulatory requirements; and (8) a statement on possible conflicts of interest.
We anticipated substantial heterogeneity between studies so used DerSimmonian and Laird random-effects meta-analysis. For infarct size we assumed that unlesioned animals would have no infarct, and therefore we used a normalized mean difference approach and report percentage improvement in infarct volume. For behavioral outcomes it was not always possible to infer, for the test reported, how an unlesioned animal would perform, so we used a standardized mean difference (SMD) approach and report SMDs in units of SD. Where >1 functional outcome was reported for the same cohort of animals at the same time point, we combined these in a fixed-effects meta-analysis to give a summary estimate of efficacy at that time point, and the last time point was used for meta-analysis. We used stratified analysis with partitioning of heterogeneity to test the extent to which 8 study characteristics explained differences in reported efficacy, with a critical threshold of P<0.057 determined using Bonferroni correction to account for multiple comparisons. This approach tests whether studies are drawn from the same population, but not whether the point estimates in strata are significantly different. Therefore, for randomization and blinding, we calculated the impact of not reporting these measures as a relative change in efficacy compared with studies which did, along with a 95% confidence interval (CI).
SHRSPs were excluded from analyses comparing delay from stroke to drug administration and to outcome assessment as there was no clear time point at which stroke occurred in these studies. Continuous data are reported as mean±95% CI and discrete data are reported as median with interquartile range.
We assessed publication bias using funnel plot and Egger regression20,21 and used trim and fill (STATA, version 10) to estimate the number of missing publications and to calculate adjusted global efficacy.22 Because the process of pooling data from different behavioral outcomes measured at the same time point would confound the analysis of publication bias, we used all data rather than pooled data, resulting in a different global estimate of efficacy in the publication bias analysis.
Our electronic search identified 4379 publications of which 4322 were excluded, leaving 57 for inclusion in the systematic review (see online-only Data Supplement).
Lacunar stroke was introduced by microthrombi injection into the internal carotid artery in 16 of 57 studies (28%), microsphere injection into the internal carotid artery in 15 (25%), and endothelin injection into deep gray matter in 12 (21%). Salt loading to accelerate the occurrence of spontaneous stroke in SHRSPs, and spontaneous strokes without salt loading in SHRSPs, was each used in 5 studies (9%). A total of 41 (72%) used rats, with others using rabbits or mice (see online-only Data Supplement). Only 47 studies reported using an anesthetic during stroke induction.
The median number of study quality checklist items scored was 3 of 8 (interquartile range, 2–4). All studies had been published in peer-reviewed journals. Twenty-six of 57 (46%) studies reported randomized allocation to treatment group, 10 (18%) reported blinding to group allocation during the experiment, 26 (46%) reported blinded assessment of outcome, 12 (21%) used animals with relevant comorbidities (hypertension), 7 (12%) stated possible conflicts of interest, 5 (9%), all from the same laboratory,23–27 reported a sample size calculation, and 44 (77%) stated compliance with animal welfare regulatory requirements (see online-only Data Supplement).
Data from 16 publications could not be included in the meta-analysis because key information such as variance or, in the case of lesion size, data for unlesioned animals were not reported or could not be inferred. Thirteen publications reported the quantity of microclots producing neurological dysfunction in 50% of animals. Although an entirely valid model of lacunar stroke, these do not provide data suitable for meta-analysis and so were excluded. Twenty-seven remaining publications described 67 experiments involving 1099 animals reporting the efficacy of 22 drugs; of these, 37 experiments using 736 animals reported changes in neurobehavior, and 30 experiments using 422 animals reported infarct size (see online-only Data Supplement).
Overall, neurobehavioral score was improved by 0.82 SMD (95% CI, 0.51–1.14; 37 comparisons; 736 animals) in experiments testing 18 interventions, with substantial heterogeneity between studies (χ2=170.3; I2=79%; df=36; P<0.0057). Stratification by intervention showed significant improvement in outcome for 8 of 18 interventions (χ2=128.4; df=17; P<0.0057; Figure 1A).
Infarct size was improved by 34.3% (95% CI, 24.2%–44.4%; 30 comparisons; 422 animals) in experiments testing 18 interventions, and again there was substantial heterogeneity (χ2=91.0; I2=68%; df=29; P<0.0057). Stratified analysis by intervention showed significant improvement in outcome for 10 of 18 interventions (χ2=83.1; df=17; P<0.0057; Figure 1B).
Four interventions improved both neurobehavioral outcome and infarct size: preclamol (1 publication, 68 animals, 4 quality checklist items scored), fasudil (1 publication, 44 animals, 2 quality checklist items scored), nicotiflorin (1 publication, 48 animals, 4 quality checklist items scored), and hepatocyte growth factor (1 publications, 21 animals, 2 quality checklist items scored). Other compounds improved either neurobehavior (DY9760e, atorvastatin, hydroxyfasudil, ozagrel) or infarct size (dihydralazine, neural progenitor cells, γ-hydroxybutyrate, cilostazol, modafinil, and minocycline) but not both, although for 4 of these drugs only neurobehavior (atorvastatin, ozagrel) or infarct size (dihydralazine, cilostazol) was reported.
For neurobehavioral score, stratifying studies by randomization status, but not by blinding status, explained a significant proportion of the observed heterogeneity (Figure 2A). The impact of nonrandomization was a relative increase in reported efficacy of +282% (95% CI, +17% to +546%; Figure 2C). There was no apparent effect of blinding (relative reduction in efficacy of –8% [95% CI, –110% to +94%]) or of the use of animals with relevant comorbidities (Figure 2C).
For infarct size, stratifying studies by either randomization or blinding status explained a significant proportion of the observed heterogeneity (Figure 2B). The impact of nonrandomization was a relative reduction in efficacy of –25.5% (95% CI, –84.3% to +33.1), and of nonblinding a relative increase in reported efficacy of +28.4% (95% CI, –60% to +116%; Figure 2C). There was no apparent difference in animals with comorbidities. Overall, the number of study quality checklist items scored explained a significant proportion of the observed heterogeneity for both neurobehavioral score (χ2=21.0; df=4; P<0.0057; Figure 3A) and infarct size (χ2=44.3; df=5; P<0.0057; Figure 3B), with highest quality studies giving the lowest estimates of efficacy.
Funnel plotting showed obvious asymmetry for neurobehavioral outcome (Figure 4A) but not infarct size (Figure 4B), whereas Egger regression suggested publication bias for both (Figure 4C and 4D). Using trim and fill analysis, we estimate 20 unpublished neurobehavioral outcomes (Figure 4A) giving an adjusted overall effect of 0.13 SMD (95% CI, –0.30 to 0.55; compared with 1.04 SMD [95% CI, 0.67–1.40]). For infarct size, we estimate 2 unpublished studies (Figure 4B) with a small reduction in the global efficacy from 33.9% (95% CI, 24.4–43.3) to 33.0% (95% CI, 24.4–42.4).
For neurobehavioral score, interventions administered intraperitoneally were the most effective (1.62 SMD; 95% CI, 0.90–2.34) and those administered intrastriatally the least (–0.15 SMD; 95% CI, –0.66 to 0.37; χ2=31.3; df=6; P<0.0057; Figure 5A). Highest efficacy was reported in studies using spontaneous stroke in SHRSPs (1.56 SMD; 95% CI, 0.60–2.52) and lowest efficacy in those using microspheres (0.50 SMD; 95% CI, –0.02 to 1.02; χ2=23.3; df=4; P<0.0057; Figure 5B). Studies assessing outcome within 1 week after stroke reported highest efficacy (1.12 SMD; 95% CI, 0.58–1.65). This was significantly lower for later times of assessment—those assessing outcome >1 month after stroke did not report significant improvement (–0.06 SMD; 95% CI, –0.60 to 0.48; χ2=18.7; df=4; P<0.0057; Figure 5C).We found no significant impact of the timing of treatment.
For infarct size, interventions administered >3 hours after stroke onset were substantially less effective (20.8%; 95% CI, 6.8–34.7) than those given within 3 hours (37.4%; 95% CI, 20.0–54.9) or before stroke onset (43.6%; 95% CI, 17.1–70.1; χ2=22.1; df=3; P<0.0057; Figure 6A). Where outcome was measured 1 to 3 weeks after stroke, efficacy was significantly lower than at other times (14.1%; 95% CI, –4.7 to 32.8), whereas the highest efficacy was found in studies assessing outcome in the first week (43.3%; 95% CI, 20.9–65.8; χ2=42.4; df=2; P<0.0057; Figure 6B). We found no significant impact of the route of administration or method of stroke induction.
We report the meta-analysis of 22 interventions tested in 7 distinct animal models of lacunar stroke. Fourteen interventions improved either infarct size or neurobehavioral outcomes, of which 10 already have Food and Drug Administration approval for other indications.28 However, the low prevalence of measures to reduce bias compromises the internal validity of the data and the likelihood of publication bias compromises their external validity; even for apparently promising interventions these concerns suggest the need for further high quality in vivo data in relevant models before embarking on clinical trials, particularly of novel agents.
This study is observational, analyzing previously collected data, and our findings are only hypothesis-generating; it may be that observed differences are because of some other factor that cosegregates with the variable of interest, and the data set is too small to allow multivariate analysis. We were unable to extract required data from 11 publications. Because low-quality studies overstate efficacy, and we included all studies, we will have overestimated treatment effects. Many of the experimental models reflect mechanisms that are not relevant to most lacunar stroke in humans. Some data from SHRSPs, the most relevant current model, could not be included through lack of time about stroke onset.
Four interventions improved both neurobehavioral outcome and infarct size. Of the four, only fasudil has been tested clinically, with promising initial results.29 Several other available agents improved the one outcome on which they were tested. In focal ischemia, stable estimates of efficacy emerge with data from ≈1000 animals,30 and we think that, even for these promising treatments, more and better evidence of efficacy in animals is required to help plan clinical trials.
Risk of Bias
The prevalence of reporting of measures to avoid bias, while modest, compares well with systematic reviews in stroke15,31,32 and other neurological conditions.14,16 Studies reporting fewest measures to avoid bias gave highest measures of treatment effect. The overstatement of efficacy in nonrandomized studies reporting neurobehavioral outcome is consistent with previous findings.14–16,31,32 Although stratification of infarct volume by randomization or blinding status explained a significant proportion of the observed heterogeneity, the strata were different in other respects; for instance, the median time to treatment was substantially shorter in randomized studies. With the exception of randomization status and neurobehavioral score, the 95% CIs for the difference in efficacy in studies at risk of bias include zero. The difference in impact of nonrandomization between studies reporting neurobehavioral outcome and those reporting infarct size may be because of the confounding effect of other variables, or to a true difference perhaps because of baseline differences in neurobehavioral performance biasing group allocation in nonrandomized studies. We think the evidence for small study effects, particularly for neurobehavioral outcomes, reflects publication bias; this provides further support for the development of systems to address this issue.
Diverse lesions are used to model lacunar stroke, and it is not clear whether there is 1 best model, or whether different models are suited to different research questions.18 Thromboembolism is an unusual cause of human lacunar stroke4 but more than half of the included studies used an embolic model. This substantially limits the relevance of the data reported here and highlights the need for use of more relevant existing models and the development of better models of lacunar stroke. Spontaneous strokes in the SHRSPs may be more similar to human lacunar stroke,33 so focus on this model may be relevant. Future trials’ design should account for assessing when stroke occurs in spontaneous lacunar models. Brain damage in patients with lacunar stroke is generally not confined to the tissue affected by the stroke but is much more diverse because of the diffuse nature of SVD. Therefore, a diffuse model, such as the SHRSPs, seems more relevant for drug testing as it models both recovery from the index stroke and these other cerebral effects of SVD.
Even a small improvement in outcome, if in an appropriate model, at an appropriate time and in an experiment at low risk of bias, might provide substantial evidence on which to embark on clinical trial. However, to have adequate power such studies would have to be substantially larger those identified here, and it may be that this can only be achieved in multicenter animal studies.34,35
Evidence-Based Clinical Trial Design
Lacunar stroke is part of the spectrum of SVD. Patients who present with an acute lacunar stroke syndrome might start treatment to prevent recurrent lacunar stroke immediately. However, other SVD features typically accumulate silently, and patients might not present for treatment until they develop cognitive or physical decline, at which stage efficacy may have declined. Treatments preventing further lacunar strokes would be useful, yet only 4 studies tested treatments administered before stroke induction. An ideal treatment would have long-term efficacy preventing both the recurrence of clinically apparent acute lacunar stroke syndromes and the build up of silent features of SVD. It is therefore reassuring that over time the effect on neurobehavioral outcome did not seem to decline, and if anything the impact on structural outcome increased.
Our findings provide some guidance for future laboratory and clinical research. First, it is unclear which is the most appropriate animal model of lacunar stroke in which to test interventions; the most relevant current models raise trial design questions concerning how to detect and time new strokes. Second, improvements in internal (study quality) and external (publication bias) validity might provide a firmer foundation for the translation to clinical trials, and more robust exploration of the limits to efficacy in such studies could inform the inclusion and exclusion criteria for such trials.
We thank Kieren Egan and Dr Emily Sena for their support and advice during this study.
Sources of Funding
Dr Macleod is supported by the UK Medical Research Council Trials Methodology Hub, Dr Vesterinen holds a University of Edinburgh Centre for Clinical Brain Sciences PhD studentship; Dr Vesterinen acknowledges the support of the National Health Service Lothian Research and Development office. Dr Wardlaw is supported by the Scottish Funding Council through the Scottish Imaging Network—A Platform for Scientific Excellence initiative.
Guest Editor for this article was Christoph Kleinschnitz, MD.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.113.003128/-/DC1.
- Received August 7, 2013.
- Revision received October 16, 2013.
- Accepted November 4, 2013.
- © 2014 American Heart Association, Inc.
- Bamford JM,
- Warlow CP
- Millikan C,
- Futrell N
- Del Bene A,
- Makin SDJ,
- Doubal FN,
- Inzitari D,
- Wardlaw JM
- Vesterinen HM,
- Sena ES,
- ffrench-Constant C,
- Williams A,
- Chandran S,
- Macleod MR
- Bailey EL,
- McCulloch J,
- Sudlow C,
- Wardlaw JM
- Light RJ,
- Pillemer DB
- Egger M,
- Davey Smith G,
- Schneider M,
- Minder C
- 28.↵FDA Approved Drug Products. U.S. Department of Health and Human Sciences Web site. http://www.accessdata.fda.gov/scripts/cder/drugsatfda/. Accessed May 01, 2012.
- Macleod MR,
- van der Worp HB,
- Sena ES,
- Howells DW,
- Dirnagl U,
- Donnan GA