(Stroke. 1995;26:498-502.)
© 1995 American Heart Association, Inc.
Articles |
The Cochrane Collaboration Stroke Review Group
Meeting the Need for Systematic Reviews in Stroke Care
From the Department of Clinical Neurosciences, Western General Hospital, Edinburgh, United Kingdom (C.C., C.W., P.S., H.F.), and the Department of Neurology, University of Utrecht (Netherlands) (J. van G.).
Correspondence to Dr Carl Counsell, Department of Clinical Neurosciences, Western General Hospital, Edinburgh EH4 2XU, UK.
 |
Abstract |
|---|
 Top Abstract Why Do We Need...Why Do We Need...The Systematic Review:...The Cochrane CollaborationReferences |
|---|
Background There is a pressing need to identify which interventions are definitely effective in the prevention of stroke and in the treatment and rehabilitation of stroke patients, which interventions are definitely ineffective, and which interventions require further research. This information is most reliably obtained from reviewing all the available evidence from randomized controlled trials in a systematic way.
Summary of Comment There have been many (at least 800) randomized controlled trials relevant to stroke. It would be difficult for any one individual to keep track of all these trials, and therefore most clinicians, therapists, and researchers are dependent, to some degree, on reviews of this literature. However, most current reviews are unsystematic and tend to be either incomplete or biased, so that their recommendations can be seriously flawed. Until now there has been no attempt to systematically identify all randomized controlled trials relevant to stroke (including subarachnoid hemorrhage), to review the data they contain, and to keep these reviews up-to-date in the light of new evidence. The Stroke Review Group has now been established within the Cochrane Collaboration to try to perform these tasks. There are presently 40 collaborators from 13 countries working on approximately 25 reviews.
Conclusions Identifying and reviewing all randomized controlled trials relevant to stroke should bring important benefits to patients and all those involved in purchasing or providing care for patients with stroke. The Cochrane Collaboration Stroke Review Group has started this process and would welcome help from anyone interested in collaborating in this enormous task.
Key Words: cerebrovascular disorders • clinical trials • meta-analysis
|
Why Do We Need to Review Randomized Controlled Trials in Stroke? |
|---|
|
TopAbstractWhy Do We Need...Why Do We Need...The Systematic Review:...The Cochrane CollaborationReferences |
|---|
We live in a time when financial resources for health care are limited, yet paradoxically there is ever-increasing clinical research in many areas, including stroke. It is therefore a good time to take stock and consider what we already know–based on the best available evidence rather than just anecdote–about interventions in the prevention, acute treatment, and rehabilitation of stroke patients.1 Only in this way can we hope to identify the best treatments for our patients as well as the most important research priorities. Moreover, with the increase in medicolegal proceedings against doctors, the best available evidence also needs to inform those who decide what is considered "adequate care." At present there is considerable variation between (and within) different countries in many aspects of care of patients with stroke, which suggests that there is uncertainty regarding what is considered "best care." For example, data from the International Stroke Trial show that nearly 50% of patients admitted with acute ischemic stroke in Italy received glycerol compared with less than 1% in the United Kingdom (P. Sandercock, unpublished data, 1994).
This begs the question of how to establish what the "best treatment" or "adequate care" is. The randomized controlled trial (RCT) is rightly regarded as the gold standard. Most beneficial interventions for patients with stroke, as with other conditions, are likely to have only moderate effects (ie, relative risk reductions on the order of 20%), which may be nonetheless worthwhile if the patient has a high absolute risk of a poor outcome. For instance, a 20% reduction of the relative risk of death in a condition that carried an absolute risk of death of 30% would result in 60 fewer deaths per 1000 patients treated. To detect these moderate effects reliably, moderate random errors and moderate biases need to be minimized.2 Trials that are truly randomized eliminate selection bias, and blinding reduces observation bias. When trials are large enough, random error (the play of chance) is also minimized.2 However, very large trials are time-consuming, expensive, and require large-scale organizational backup and therefore are relatively uncommon. The more usual state of affairs is to have several small RCTs of varying quality that give inconsistent or conflicting results. This creates confusion, which may lead to varying clinical practice and to further small trials to "retest the hypothesis." Stroke medicine provides a good example of this. The largest published trial relating to the treatment of acute stroke included only 1267 patients.3 Many of the potentially useful treatment modalities in acute stroke have been tested in several small trials instead of one definitive large trial. For instance, we have identified 22 completed RCTs of calcium antagonists in acute stroke, which included in total approximately 5000 patients. The largest trial included 1215 patients and the smallest 19. Ten trials (45%) included fewer than 100 patients. Ten reported a statistically significant beneficial effect from calcium antagonists on at least one outcome of interest, and 12 showed no effect or an adverse effect.
|
Why Do We Need to Review All Randomized Controlled Trials in Stroke? |
|---|
|
TopAbstractWhy Do We Need...Why Do We Need...The Systematic Review:...The Cochrane CollaborationReferences |
|---|
To make sense of such situations, the clinician needs to review all the available evidence from RCTs relating to a particular treatment. If only certain subsets of the trials are reviewed, particularly subsets that are easier to identify, the conclusions may be seriously biased. A comparison can be drawn with the need to follow up all patients in clinical trials. If large numbers of patients are lost to follow-up, the results of a trial may be inaccurate or biased, since patients who cannot be traced may be more likely to have a poor outcome, eg, be dead or disabled, than other patients. Similarly, trials with statistically null results tend to be less well known than those with statistically positive results because they are less likely to be published4 5 6 (publication bias) and less likely to be cited in reference lists7 (citation or reference bias). There may also be systematic differences in the results of trials published in English rather than another language (language bias). However, it is practically impossible for a single person to keep track of all the relevant RCTs in his/her specialty. During the past year we have identified more than 800 references to RCTs relevant to stroke that have been published in more than 150 journals or books in seven different languages, and we continue to identify approximately five RCTs per week. In the face of such a daunting task, most clinicians rely on the personal knowledge of a few "important" trials (the choice of which is likely to be biased by personal prejudice or access to information) and reviews of the literature published in journals or textbooks. Unfortunately, many of these reviews are unsystematic and of poor quality because reviewers do not apply the same rigorous methods to the writing of reviews as they do to their primary research.8 As a result, their recommendations can be misleading or frankly wrong. Antman et al9 showed that the recommendations of reviews in the cardiological literature frequently lagged several years behind the recommendations that would have arisen if systematic reviews of all RCTs available at that time had been performed. The probable result has been that effective treatments that could have saved thousands of lives, such as thrombolysis for acute myocardial infarction, did not become widely established as quickly as they should have. Furthermore, other treatments, such as the routine use of lidocaine after myocardial infarction, which systematic reviews would have shown to be ineffective and possibly harmful, continued to be given.9
Numerous similar examples probably exist in stroke medicine. One example is the use of heparin in acute ischemic stroke. A systematic review including all known RCTs was published in 1993 and showed that the benefits of heparin were unclear: the 95% confidence intervals suggested that there was the potential for it to save 66 lives per 1000 patients treated or to kill 36 patients per 1000 treated.10 However, recommendations from several recent nonsystematic reviews have varied from the routine administration of heparin in some situations11 12 13 to the advice that heparin not only be avoided but also excluded from further trials.14 The confusion this has produced is evident. In one study in the United States, 22% of stroke patients treated by neurologists were given heparin within 24 hours of stroke despite the fact that 48% of the neurologists recognized the lack of evidence for its efficacy.15 A recent survey in the United Kingdom showed that 10% of physicians who treated patients with stroke routinely gave some form of heparin to patients with ischemic stroke (R. Lindley, unpublished data, 1994). Another example comes from a recent review of the treatment of subarachnoid hemorrhage, which recommended the routine use of steroids and prophylactic anticonvulsants in patients with acute subarachnoid hemorrhage.16 This was unsupported because there have been no relevant RCTs evaluating their effects.17
|
The Systematic Review: Advantages and Disadvantages |
|---|
|
TopAbstractWhy Do We Need...Why Do We Need...The Systematic Review:...The Cochrane CollaborationReferences |
|---|
Clearly, therefore, there is a need for up-to-date systematic reviews of all the available evidence regarding the various aspects of care of stroke patients to ensure that treatments that are effective are widely recognized and used, treatments that have been proven to be ineffective are identified and discouraged, and treatments for which insufficient evidence exists are subjected to further research. Such systematic reviews should give explicit criteria for the inclusion of a study, use extensive search strategies to identify all the potentially eligible studies and, when possible, use appropriate statistical methods to pool the data to derive a "best estimate" of the treatment effect (meta-analysis).18 Deriving estimates of treatment effects in this way effectively increases the sample size and thereby reduces random errors, which can obscure treatment effects in small trials,2 narrow confidence intervals, and may also allow preliminary subgroup analyses.
The assessment of the effectiveness of stroke units provides a good example of the power of systematic reviews. The recognition that stroke units improve outcome is perhaps the most important advance in the treatment of stroke patients in recent years. None of the 10 small RCTs on their own had provided conclusive evidence of their benefit, and it was only when a systematic review was performed that a highly statistically significant 33% reduction in the odds of death or institutionalization at 1 year was discovered.19 20 Frequently the results of systematic reviews are not so clear-cut but are still useful. Often they will highlight the need for large RCTs to clarify treatment issues.10 At the very least, systematic reviews provide all the available data in a form that can be reanalyzed by others who may wish to omit certain RCTs, explore different outcomes, or concentrate on particular subgroups of patients. In the case of stroke trials they have also demonstrated the need for better and more uniform assessments of functional outcome. It is obviously important to know whether treatment of acute stroke reduces mortality, but it is also important to know whether acute treatment and subsequent rehabilitation reduce disability in survivors. However, functional outcome is frequently poorly recorded or uses one of many different measures, which makes analysis in systematic reviews difficult21 or impossible.10 22 In a survey of the first 123 acute stroke trials that we have identified, only 28 (23%) recorded a measure of functional outcome as opposed to measuring neurological impairment by means of a stroke scale. Neurological impairment is more frequently recorded but is clinically less important and is also difficult to analyze because of the wide variety of different stroke scales used.23 24 25
It is important to realize that although systematic reviews are a significant improvement on their nonsystematic counterparts, they too have their limitations. First, there are the logistic problems of identifying all the available randomized studies and synthesizing them into a single report. Potential publication bias means that efforts have to be taken to identify as many unpublished trials as possible.4 5 6 26 Even then, the usefulness of systematic reviews will depend on the amount of data available and the quality of the trials included. Systematic reviews cannot answer questions that were not posed by the original studies. They are also not very good at analyzing side effects associated with treatments because these are rare and often poorly reported in the original trials and because different trials use different definitions of major side effects. Technical problems also exist concerning which trials should be included in a particular review, which statistical technique should be used to derive the overall treatment effect, and how generalizable the results are.27 28 29 For example, the evidence from the Antiplatelet Trialists' Collaboration showed beyond all reasonable doubt the overall effectiveness of antiplatelet agents in reducing the risk of serious vascular events in high-risk patients.30 There is little disagreement about that particular conclusion, but debate continues on which patients are at sufficiently high risk to warrant therapy and on the optimum antiplatelet regimen.29 31 32 33 We would certainly agree that it is not always easy to generalize the results of a systematic review to an individual patient, but we also believe that the overwhelming advantage of such a review is that it does present all the information, and readers can then make their own judgments.
|
The Cochrane Collaboration |
|---|
|
TopAbstractWhy Do We Need...Why Do We Need...The Systematic Review:...The Cochrane CollaborationReferences |
|---|
Archie Cochrane (1909-1988), a respiratory epidemiologist and clinical trialist, pointed out earlier than anyone else the need for systematic reviews of healthcare interventions across all specialties that should be periodically updated as new evidence becomes available.34 In response to his challenge, the Cochrane Collaboration was established in 1992 to try to achieve this.35 It aims to bring together people with similar interests and enthusiasm from around the world to form collaborative review groups, which then go about the task of systematically reviewing the evidence in their chosen field. Ideally this evidence is from RCTs, but when this is lacking, other forms of data can also be reviewed. The organization, which is coordinated from several Cochrane centers (Australia, Canada, Denmark, Italy, United States, United Kingdom), is open to anyone who might contribute in any way. For instance, clinicians, therapists, and epidemiologists can collaborate on the production of reviews, and medical students or nonmedical volunteers can be trained to hand-search journals looking for trials. Others may simply want to offer help with funding.
The Cochrane Collaboration Stroke Review Group36 was one
of the first to register with the organization and already has 40
members from 13 countries. With the help of the Ottawa Stroke Trials
Registry,37 it is now in the process of developing a
complete register of RCTs in the prevention of stroke and in the
treatment and rehabilitation of stroke patients. Relevant trials are
then forwarded to the reviewers, who will use them to produce
systematic reviews, which will then be periodically updated as new
trials become available or old ones are discovered. By having a central
register of trials, reviewers are spared much of the onerous task of
trying to identify all the trials in their chosen field. Guidelines
have been drawn up by the Cochrane Collaboration to ensure that each
review is of a high quality, but it is ultimately up to individual
reviewers to decide what form their review will take. For instance,
most will be based on tabular data, but some will use individual
patient data. Completed reviews or protocols of planned reviews will
then be returned to an editorial board and forwarded to a Cochrane
center for publication and dissemination to as wide an audience as
possible (Figure
). The emphasis will be on electronic
publication, which will make it easier for the reviews to be regularly
updated (and corrected and improved in the light of valid criticisms).
Initially the reviews will be published twice yearly on computer disk
and CD-ROM, but it is hoped that eventually they will be available
on-line over INTERNET (Figure). Authors may also wish to publish their
reviews simultaneously in journals, and agreement has been reached with
several journals, such as the BMJ, the Lancet,
and the Annals of Internal Medicine, which will allow them
to do so.
|
This is an enormous task. We in the Stroke Review Group have only just
begun. The current list of reviews that are in progress does not nearly
cover all the relevant topics in stroke (Table). To
succeed, we will require continued and extended international
collaboration, both to identify all the published and unpublished
stroke RCTs worldwide and to help in the production of reviews. We
welcome any further help from people who believe they can collaborate.
The potential benefits to clinicians, researchers, and, most
importantly, patients are clear for all to see and certainly justify
the effort.
|
|
Acknowledgments |
|---|
This effort is supported by a Wellcome Training Fellowship in Clinical Epidemiology (C. Counsell); the Chief Scientist Office, Scottish Home and Health Department (H. Fraser); and the United Kingdom Medical Research Council (P. Sandercock).
Received October 25, 1994; revision received November 30, 1994; accepted November 30, 1994.
|
References |
|---|
|
TopAbstractWhy Do We Need...Why Do We Need...The Systematic Review:...The Cochrane CollaborationReferences |
|---|
1. Scheinberg P. Stroke: the way things really are. Stroke. 1994;25:1290-1294. [Abstract]
2. Peto R. Why do we need systematic overviews of randomized trials? Stat Med. 1987;6:233-244. [Medline] [Order article via Infotrieve]
3. Italian Acute Stroke Study Group. Haemodilution in acute stroke: results of the Italian Haemodilution Trial. Lancet. 1988;1:318-321. [Medline] [Order article via Infotrieve]
4. Chan SS, Sacks HS, Chalmers TC. The epidemiology of unpublished randomized control trials. Clin Res. 1982;30:234A. Abstract.
5. Dickersin K, Chan S, Chalmers TC, Sacks TC, Smith H Jr. Publication bias and clinical trials. Control Clin Trials. 1987;8:343-353. [Medline] [Order article via Infotrieve]
6. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991;337:867-872. [Medline] [Order article via Infotrieve]
7. Gøtzsche PC. Reference bias in reports of drug trials. BMJ. 1987;295:654-659.
8. Mulrow CD. The medical review article: state of the science. Ann Intern Med. 1987;106:485-488.
9.
Antman EM, Lau J, Kupelnick B, Mosteller F, Chalmers TC. A
comparison of results of meta-analyses of randomized control trials and
recommendations of clinical experts. JAMA. 1992;268:240-248.
10.
Sandercock PAG, van den Belt AGM, Lindley RI, Slattery J.
Antithrombotic therapy in acute ischaemic stroke: an overview of the
completed randomised trials. J Neurol Neurosurg Psychiatry. 1993;56:17-25.
11.
Estol CJ, Pessin MS. Anticoagulation: is there still a role in
atherothrombotic stroke? Stroke. 1990;21:820-824.
12.
Miller VT, Hart RG. Heparin anticoagulation in acute brain
ischemia. Stroke. 1988;19:403-406.
13. Sherman DG, Hart RG. Thromboembolism and antithrombotic therapy in cerebrovascular disease. J Am Coll Cardiol. 1986;8:88B-97B.
14.
Phillips SJ. An alternative view of heparin anticoagulation in
acute focal brain ischemia. Stroke. 1989;20:295-298.
15.
Marsh EE, Adams HP, Biller J, Wasek P, Banwart K, Mitchell V,
Woolson R. Use of antithrombotic drugs in the treatment of acute
ischaemic stroke: a survey of neurologists in practice in the United
States. Neurology. 1989;39:1631-1634.
16.
Kopitnik TA, Samson DS. Management of subarachnoid
haemorrhage. J Neurol Neurosurg Psychiatry. 1993;56:947-959.
17. Vermeulen M, Rinkel GJE. Management of subarachnoid haemorrhage. J Neurol Neurosurg Psychiatry. 1994;57:768-769.
18. L'Abbe KA, Detsky AS, O'Rourke K. Meta-analysis in clinical research. Ann Intern Med. 1987;107:224-233.
19. Langhorne P, Williams BO, Gilchrist W, Howie K. Do stroke units save lives? Lancet. 1993;342:395-398. [Medline] [Order article via Infotrieve]
20. Langhorne P, on behalf of the Stroke Unit Trialists' Collaboration. Specialist stroke unit care improves survival and outcomes: a statistical overview. Cerebrovasc Dis. 1994;4:258. Abstract.
21. Schleenbaker RE, Mainous AG. Electromyographic biofeedback for neuromuscular re-education in the hemiplegic stroke patient: a meta-analysis. Arch Phys Med Rehabil. 1993;74:1301-1304. [Medline] [Order article via Infotrieve]
22. Di Mascio R, Marchioli R, Tognoni G. From pharmacological promises to controlled clinical trials to meta-analysis and back: the case of nimodipine in cerebrovascular disorders. Clin Trials Meta-analysis. 1994;29:57-79. [Medline] [Order article via Infotrieve]
23. Asplund K. Hemodilution in acute stroke. Cerebrovasc Dis. 1991;1(suppl 1):129-138.
24. Mohr JP, Orgogozo JM, Harrison MJG, Hennerici M, Wahlgren NG, Gelmers JH, Martinez-Vila E, Dycka J, Tettenborn D. Meta-analysis of oral nimodipine trials in acute ischemic stroke. Cerebrovasc Dis. 1994;4:197-203.
25. Muir KW, Grosset DG, Lees KR. Interconversion of stroke scales: implications for therapeutic trials. Stroke. 1994;25:1366-1370. [Abstract]
26. Simes RJ. Confronting publication bias: a cohort design for meta-analysis. Stat Med. 1987;6:11-29. [Medline] [Order article via Infotrieve]
27. Thompson SG, Pocock SJ. Can meta-analyses be trusted? Lancet. 1991;338:1127-1130. [Medline] [Order article via Infotrieve]
28. Berlin JA, Laird NM, Sacks HS, Chalmers TC. A comparison of statistical methods for combining event rates from clinical trials. Stat Med. 1989;8:141-151. [Medline] [Order article via Infotrieve]
29. Dyken ML. Meta-analysis in the assessment of therapy for stroke prevention. Cerebrovasc Dis. 1992;2(suppl 1):35-40.
30.
Antiplatelet Trialists' Collaboration. Collaborative
overview
of randomised trials of antiplatelet therapy, I: prevention of death,
myocardial infarction, and stroke by prolonged antiplatelet therapy in
different categories of patients. BMJ. 1994;308:81-106.
31. Gelmers HJ, Tijssen JGP. Platelet antiaggregants in secondary prevention after stroke: does dipyridamole add to the effect of aspirin? J Stroke Cerebrovasc Dis. 1993;3:115-120.
32. Lowenthal A, Buyse M. Secondary prevention of stroke: does dipyridamole add to aspirin? Acta Neurol Belg. 1994;94:24-34. [Medline] [Order article via Infotrieve]
33. Algra A, van Gijn J. Low or high dose aspirin: equi-efficacy in secondary prevention. Cerebrovasc Dis. 1994;4:259. Abstract.
34. Cochrane AL. 1931-1971: a critical review with particular reference to the medical profession. In: Medicines for the Year 2000. London, England: Office of Health Economics; 1979:1-11.
35. Chalmers I, Dickersin K, Chalmers TC. Getting to grips with Archie Cochrane's agenda. BMJ. 1992;305:786-788.
36.
Counsell CE, Fraser H, Sandercock PAG. Archie Cochrane's
challenge: can periodically updated reviews of all randomised
controlled trials relevant to neurology and neurosurgery be produced?
J Neurol Neurosurg Psychiatry. 1994;57:529-533.
37. The OSTR Collaborative Group. The Ottawa Stroke Trials Registry Collaborative Group and the development of the Ottawa Stroke Trials Registry. Control Clin Trials. 1994;15:503-511.[Medline] [Order article via Infotrieve]
This article has been cited by other articles:
 |
|
 |
|
  P. Sandercock, H. Fraser, B. Thomas, A. McInnes, and S. Dixon Cochrane Stroke Group 10 Years On: Progress to Date and Future Challenges Stroke, October 1, 2003; 34(10): 2537 - 2539. [Full Text] [PDF]
|
|
|
|
 |
|
 J. M. Wardlaw Overview of Cochrane thrombolysis meta-analysis Neurology, September 1, 2001; 57(90002): S69 - 76. [Abstract] [Full Text]
|
|
|
|
|
|
S. C. Shiflett and K. Asplund Acupuncture and Stroke Rehabilitation Response Stroke, August 1, 2001; 32(8): 1934 - 1936. [Full Text] [PDF]
|
|
|
|
|
|
C. S. Kidwell, D. S. Liebeskind, S. Starkman, and J. L. Saver Trends in Acute Ischemic Stroke Trials Through the 20th Century Stroke, June 1, 2001; 32(6): 1349 - 1359. [Abstract] [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||