‘Yes’ or ‘No’ to Routine Statins After Subarachnoid Hemorrhage to Prevent Delayed Cerebral Ischaemia, Vasospasm, and Death?
A Cautionary Tale of 2 Meta-Analyses
See related article, pages e47–e52.
“Although there are a few striking examples of treatments for serious disease which really do work extremely well, most claims for big improvements turn out to be evanescent.”
Rory Collins and Stephen MacMahon1
Do statins reduce the risk of vasospasm, delayed cerebral ischemia, and death after aneurysmal subarachnoid hemorrhage (SAH)? Two recent meta-analyses2,3 reach opposite conclusions. The 2008 meta-analysis by Sillberg et al of the 3 trials available in 2008 concluded: “Initiation of statin therapy after aneurysmal SAH significantly reduces the incidence of vasospasm, delayed ischemic deficits, and mortality. This is consistent with animal research and retrospective studies and supports the routine use of statins in the care of patients with aneurysmal SAH.”2 By contrast, the 2009 meta-analysis by Vergouwen et al concluded: “The results of the present systematic review do not lend statistically significant support to the finding of a beneficial effect of statins in patients with aneurysmal SAH.”3 Three questions arise: (1) why have 2 systematic reviews reached different conclusions? (2) Is one of them “right” and the other “wrong”? (3) Should statins be used routinely after aneurysmal SAH?
To consider the first question, one must remember that the history of the evaluation of individual therapies is littered with examples in which initial enthusiasm for it (with early adoption by enthusiasts—“this stuff is safe and it’s great”) is followed by a period of disillusion and abandonment (“it doesn’t work as well as we first thought, and it has side effects”). If reliable evidence then emerges, giving a more realistic appreciation of its risks and benefits, the treatment may then take its rightful place in routine medical practice. In ischemic stroke treatment and prevention, the adoption of aspirin, carotid endarterectomy, thrombolytic therapy, and admission to comprehensive care stroke units all followed this pattern. Ioannidis has provided quantitative empirical data on the phenomenon by examining whether the findings of highly cited original clinical trials were subsequently confirmed or refuted by later studies.4 Of 49 highly cited studies, 45 claimed that the intervention was effective. Of these, 7 (16%) were contradicted by subsequent studies, 7 others (16%) had found effects that were stronger than those of subsequent studies, 20 (44%) were replicated, and 11 (24%) remained largely unchallenged. Among randomized trials, studies with contradicted or stronger effects were smaller (P=0.009) than replicated or unchallenged studies. He concluded “. . . even the most highly cited randomized trials may be challenged and refuted over time, especially small ones.”4 We should therefore perhaps not be surprised, given the small number of people with aneurysmal SAH included in either meta-analysis, that they reach different conclusions.1
So, is one of them “right” and the other “wrong”? The important question is not the test of the null hypothesis at the 5% level (is P<0.05, yes or no?), but what is the size and direction of the treatment effect, and what is the precision of the estimate (how wide are the confidence intervals?)? In both papers, the estimates of effect are similar in size and direction for all the outcomes assessed, and the confidence intervals are similarly wide. Both are trying to estimate the true effect of statins. The reasons for the authors reaching different conclusions based on approximately similar results lie in the approach to the interpretation of the results. The situation is very similar to the European Cooperative Acute Stroke Study (ECASS)-1 and ECASS-2 trials of thrombolytic therapy for acute ischemic stroke, which both yielded similar estimates of effects with quite wide confidence intervals; in one analysis of outcome at 3 months, one trial was significant at P<0.05 and the other was not, but both gave point estimates that were close to the best estimate of the “truth” (the overall estimate from the meta-analysis of all recombinant tissue plasminogen activator trials).5 So neither one was “correct” and neither one was “wrong”; both were just reasonably near the truth in their estimates. The confidence intervals in SAH are even wider than for thrombolysis, so we do not know how near the truth the estimates of effect for statins really are.
However, when it comes to interpretation, Vergouwen et al3 are probably nearer “the truth” than Sillberg,2 (1) because they have a larger (albeit still tiny) sample from which to estimate the true effects of statins in this setting; (2) because they have undertaken some sensitivity analyses to assess the robustness of the findings (especially the implausibly large effect on mortality in Sillberg’s review, which disappears by the reallocation of one death2); and (3) because, applying Cochrane methodology, they have been suitably cautious in interpreting the results of their meta-analysis. In particular, in the wording of the conclusions section of the abstract of their review, Vergouwen et al have followed the advice of the Cochrane Handbook for Systematic Reviews of Interventions, which states: “The primary purpose of the review should be to present information, rather than to offer advice or recommendations.”6
Should statins be used routinely after SAH? I think the jury is still out. The effects seen in the small trials to date are certainly promising, but may well be “too good to be true.” However, an inexpensive, safe, and effective therapy to further reduce the disabling effects of delayed cerebral ischemia after aneurysmal SAH would be most welcome, even if the effects were less striking. Further trials are certainly justified. However, it is premature to recommend routine use of statins until the results of the ongoing Simvastatin in Aneurysmal Subarachnoid Hemorrhage (STASH) trial are known. STASH is a Phase III randomized trial evaluating the effects of 40 mg simvastatin in patients with acute aneurysmal SAH.7 The primary outcome is disability at 6 months (modified Rankin Scale). Secondary outcomes include: need for, and intensity of, delayed ischemic deficit rescue therapy; incidence and duration of delayed ischemic deficits; incidence and severity of sepsis; length of intensive care and total acute hospital stay; and discharge destination. The planned sample size is 1600 patients and, as of July 2009, >400 patients had been recruited, already more than double the sample of 190 patients in the Vergouwen review. A Cochrane Systematic Review is planned that will synthesize all the available evidence when the STASH trial is reported.8
The contrasting results of these 2 reviews provide useful lessons in critical appraisal of research evidence, the hazards of overemphasis of the results of small trials, and the need for stringent methods when performing and interpreting systematic reviews and meta-analyses. At the end of it all, let us hope that the benefits of statins in aneurysmal SAH turn out to be real, are confirmed by a larger study, and not—as is so often the case—evanescent.
The author is employed full-time by the University of Edinburgh. He is Coordinating Editor of the Cochrane Stroke Group, which is funded by the Chief Scientist’s Office of the Scottish Government. The Cochrane Collaboration is an international not-for-profit organization that aims to help people make well-informed decisions about healthcare by “preparing, maintaining, and promoting the accessibility of systematic reviews of the effects of healthcare interventions.”
The opinions in this editorial are not necessarily those of the editors or of the American Heart Association.
Sillberg VAH, Wells GA, Perry JJ. Do statins improve outcomes and reduce the incidence of vasospasm after aneurysmal subarachnoid hemorrhage: a meta-analysis. Stroke. 2008; 39: 2622–2626.
Vergouwen MDI, de Haan RJ, Vermeulen M, Roos Y. Effect of statin treatment on vasospasm, delayed cerebral ischemia, and functional outcome in patients with aneurysmal subarachnoid hemorrhage—a systematic review and meta-analysis update. Stroke. e47–e52.
Wardlaw JM, Zoppo G, Yamaguchi T, Berge E, Murray V. Thrombolysis for acute ischaemic stroke. Cochrane Database Syst Rev. 2009; Issue 4.
Higgins JPT, Green S, eds. Cochrane Handbook for Systematic Reviews of Interventions, version 5.0.2 [updated September 2009]. The Cochrane Collaboration 2009. Available at: www.cochrane-handbook.org. Last accessed November 21, 2009.
STASH Trial Group. Simvastatin in Aneurysmal Subarachnoid Haemorrhage (STASH) trial. Available at: www.strokecenter.org/trials/TrialDetail.aspx?tid=930. Accessed September 18, 2009.
Liu Z, Zhao B, Liu L. Cholesterol-reducing agents for aneurysmal subarachnoid haemorrhage. Cochrane Database Syst Rev. In press.