How to Improve the Quality of a Clinical Trial on Traditional Chinese Medicine for Stroke
To the Editor:
In the August issue of Stroke, Zhang and colleagues1 took on the tremendous task to present a systematic review of the efficacy of complex Traditional Chinese Medicine (TCM) for stroke. The authors have included 34 randomized, controlled trials and quasi-randomized, controlled trials, which had assessed the effect of complex TCM on motor dysfunction after a stroke. In a quite appropriate way, they have chosen death, activities of daily living, functional recovery, and quality of life as outcome criteria. They correctly stress the poor methodological quality of almost all the trials selected for the review. The participants involved in these trials were patients who had either an ischemic or hemorrhagic stroke. The time from stroke onset to the trial inclusion was highly variable. The heterogenous interventions included complex TCM, acupuncture, herbal medicine, Western medicine, and physical exercise. “Effective rate” was defined as a primary outcome measure in almost all the trials. All trials but one reported results in favor of complex TCM treatments. Only one study reported mild adverse events. The authors concluded that most of the studies available for the review were inadequately designed trials characterized by unknown dropout rates and definitional vagueness in outcomes measures.
How should we assess the quality of these clinical trials? There are at least several limitations in the design of trial included in this review. First, the term “randomized” is improperly used in most papers. The original contribution had not described the method of randomization and the allocation concealment. In almost all trials, as shown by Figure 2 of the review paper, the number of participants in the experimental group was higher, sometimes very much so, than that in the control groups. This fact raises doubts about whether these studies had really been constructed and run in a truly randomized fashion. In fact, incomplete reporting of research methods, both in the original paper and in later review of it, provides a misleading description of randomization procedures. Referring to a nonrandomized study like a randomized, controlled trial is a worldwide problem raising ethical concerns about research reporting. Second, criteria to include papers in the systematic review as well as the stroke diagnostic criteria applied in their choice were not appropriate. An additional shortcoming of the systematic review is the lack of discussion about the background of the researchers performing the trials, whether they were allopathic or traditional medicine practitioners, because the diagnosis and treatment of stroke is not identical in the 2 medical systems. It is of particular note, for example, that pharmacological studies have indicated that some TCMs may dilate the cerebral vessels and suppress the aggregation of platelets.2 Hence, TCM might increase the risk of hematoma extension for the patient with hemorrhagic stroke in the early stage. Moreover, the time of TCM treatment after stroke onset varied significantly among these trials (from a subacute state at 12 days to a very chronic state after 6 years). As now well established, 5 to 6 months after stroke is an appropriate time point at which to measure neurological and functional outcome. Spontaneous recovery does not reach a plateau until 5 to 6 months after stroke, especially in more severe strokes.3 It is almost impossible to improve the functional outcome for the patients after several years of stroke onset by using any drug. Maybe TCM can prevent stroke recurrence. So, the proper outcome measure for these patients should be stroke recurrence rather than functional status. Most of the trials evaluated the efficacy immediately after completing the treatment. The period of follow-up was not long enough to evaluate the long-term effect of TCM. Third, the testified intervention should be compared with current “gold standard treatment” rather than randomly chosen unproved treatment. Fourth, the outcome measure of most of the trials was defined as an “effective rate.” What is effectiveness? Which instrument was used to identify efficacy? How about the validity and reliability of the scale used for assessing the outcome? The primary outcome measure should be focused on the level of activities rather than a vague “effective rate.” Lastly, adverse effects are as important as the efficacy of the interventions in clinical trials. It should be noted that TCM is not side effect-free and caution should be exercised in its use.4
So, what lessons can we learn from this systematic review? As mentioned in our own previously published review,2 future trials of TCM should overcome these limitations; in particular, they should assure adequate concealment of allocation, appropriate participants, and use functional outcome as the primary outcome measured at long-term follow-up. Reports of the trials should conform to the recommendations of the Consolidated Standards of Reporting Trials (CONSORT) statement.4
- ↵Zhang J, Menniti-Ippolito F, Gao X, Firenzuoli F, Zhang B, Massari M, Shang H, Huang Y, Ferrelli R, Hu L, Fauci A, Guerra R, Raschetti R. Complex Traditional Chinese Medicine for poststroke motor dysfunction. A systematic review. Stroke. 2009; 40: 2797–2804.
- ↵Wu B, Liu M, Liu H, Li W, Tan S, Zhang S, Fang Y. Meta-analysis of traditional Chinese patent medicine for ischemic stroke. Stroke. 2007; 38: 1973–1979.
- ↵Duncan PW, Jorgensen HS, Wade DT. Outcome measures in acute stroke trials: a systematic review and some recommendations to improve practice. Stroke. 2000; 31: 1429–1438.
- ↵Feigin VL. Herbal medicine in stroke: does it have a future? Stroke. 2007; 38: 1734–1736.