Response to Letter by Ernst and Lee
To the Editor:
Drs Ernst and Lee1 provide a superficial interpretation of our systematic review, meta-analysis, and meta-regression. This is perhaps not surprising because our review does not provide results that agree with their unpublished umbrella review. Given the much smaller number of studies assessed in other reviews of acupuncture in poststroke treatment, it is not surprising that a different assessment has been made by Ernst and Lee. This does not negate the fact that our systematic review is more recent and incorporates a much larger set of relevant primary data for analysis. We address each point raised by the commentators.
Ernst and Lee state that we missed previous relevant systematic reviews. In fact, we did not. The unreferenced reviews evaluated acupuncture for very specific functions, such as dysphagia or insomnia caused by stroke. In comparison, our review and those systematic reviews and meta-analyses we did cite explored improvements that we felt were of greatest clinical relevance and were most generalizable. Contrary to the comments expressed by Ernst and Lee, these reviews did find positive findings, thus speaking to a consistency of effect that seems to be omitted in their unpublished assessment.
Ernst and Lee also state that we relied on the Chinese literature heavily, thus contributing to purported bias in our results. We are aware of the difficulties of including trials from China in this body of literature.2 A thorough review of our meta-regression underscores that we included publication status (Chinese vs Western trials) as part of the model. In this instance, we found no indication of significant bias.
The authors state that odds ratios inflate the effect size over relative risks. They are incorrect; odds ratios do not inflate, they simply report in a different manner than relative risks. Odds ratios are a statistically preferable effect estimate when dealing with regressions and also when dealing with populations in which a patient begins with the disease and improvement is the outcome.3
Just what risks to bias are incurred with an A vs A+B trial design compared to A vs B designs remains to be seen. Ernst and Lee provide as evidence a nonstatistical review of acupuncture for pain clinical trials to suggest that only positive results can come from A vs A+B pragmatic-style trials. On review of this article, another interpretation could be equally made: when acupuncture is used for pain, there actually could be a clinical benefit realized. If Ernst and Lee believe that this trial design results in only positive trials, then they will need to demonstrate this in medical fields where this is the common approach, such as with HIV/AIDS and cancer.4
Finally, Ernst and Lee state that sham acupuncture controlled trials resulted in nonsignificant effects and that these should be analyzed separately. We, and others, disagree with such an approach. Although this is an issue of preference, we believe that one should take a broad approach to pooling trials and then examine predetermined subgroups and explanations of heterogeneity. This methodology is well-accepted and used by groups such as the Antiplatelet Trialists Collaboration and the STOP-IT teams.5,6 We believe the work of the commentators would be improved if they aimed to use statistical approaches rather than stating whether something is significant and not examining more rigorously why this may be.
In conclusion, Ernst and Lee are correct to heed caution in the interpretation of our analysis. Had they read our article thoroughly, they would note that we addressed their points and that we report a cautious interpretation that argues for a large, well-conducted, and transparent randomized trial.
Ernst E, Lee MS. Acupuncture during stroke rehabilitation. Stroke. 2010; 41: e549.
Antithrombotic Trialists’ Collaboration. Collaborative meta-analysis of randomised trials of antiplatelet therapy for prevention of death, myocardial infarction, and stroke in high risk patients. BMJ. 2002; 324: 71–86.
Bassler D, Briel M, Montori VM, Land M, Glasziou P, Zhou Q, Heels-Ansdell D, Walter SD, Guyatt GH, STOPIT-2 Study Group, Flynn DN, Elamin MB, Murad MH, Abu Elnour NO, Lampropulos JF, Sood A, Mullan RJ, Erwin PJ, Bankhead CR, Perera R, Ruiz Culebro C, You JJ, Mulla SM, Kaur J, Nerenberg KA, Schunemann H, Cook DJ, Lutz K, Ribic CM, Vale N, Malaga G, Akl EA, Ferreira-Gonzalez I, Alonso-Coello P, Urrutia G, Kunz R, Bucher HC, Nordmann AJ, Raatz H, da Silva SA, Tuche F, Strahm B, Djulbegovic B, Adhikari NK, Mills EJ, Gwadry-Sridhar F, Kirpalani H, Soares HP, Karanicolar PJ, Burns KE, Vandvik PO. Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 2010; 303: 1180–1187.