What Lessons Have We Learned in the Past 40 Years?
Before 1970, there was no journal devoted to cerebral vascular disease. Only hypertension had been identified as a potentially treatable risk factor for stroke, and there were no established specific therapies for preventing or treating stroke. Imaging was limited to x-ray or x-ray with contrast (angiography, pneumoencephalography, or ventriculography) and very primitive ultrasound.
In 1970, the journal Stroke was born and during the next 40 years many advancements occurred. CT, MRI, positron emission tomography, and ultrasound were introduced and further perfected as extremely effective diagnostic instruments. For the first time, we could visualize the nervous system and measure its function in vivo. The major risk factors were identified, and the significant effectiveness of preventive treatment was established for hypertension, tobacco use, lipidemia, and other potentially treatable risk factors. The value and the indications have been defined for platelet antiaggregating agents, thrombolytic therapy, statins, anticoagulant therapy for atrial fibrillation, carotid endarterectomy, and coiling for aneurysm and arteriovenous malformations. Genetics became a major tool in the investigation of cerebrovascular disease. Vascular dementia and its relationship to other dementias have been better identified. The lack of effectiveness of extracranial–intracranial bypass procedures in most ischemic vascular disease has been shown. Steady progress is being made in the understanding of the mechanisms of brain injury, repair, and plasticity. All these and more have been added to our knowledge.
These are not the lessons, but they are the results of the lessons we have learned. To address these lessons, we need to go back even further than the past 40 years. Although I cannot equal Dr Barnett’s 65 years, my 56 years as a physician have led to similar conclusions. So, forgive us if we overlap.
Like most of the older people in the field, our acceptance and crusade for the “lessons” were initiated and reinforced by personal experiences. During the 1950s through the early 1970s, it was not uncommon to go to meetings or read a journal in which almost all the presentations were anecdotal. It was disheartening to listen or read presentations and conclusions drawn by our learned elders that were all subjective and based on perceived experience. It was even more disheartening to find that equally respected elders seemed to draw entirely different conclusions from what appeared to be similar experiences. If we had not been indoctrinated with the scientific method before, then these experiences established its need.
As a personal example, these impressions were reinforced for me in 1955 when one of the lead articles in the Journal of the American Medical Association reported that 27 of 35 consecutive stroke patients dramatically improved after receiving cortisone. At the time, only severely ill stroke patients were admitted on the service I was covering. Most had trouble swallowing and handling their secretions and many died, and all I could do was support them. In the face of these very impressive statistics in consecutive patients, it seemed unethical not to go back to the wards and start cortisone immediately. But cortisone was and is a dangerous drug. So with great trepidation and the support of my staff and Dr Philip White, I designed a prospective, double-blind, controlled study that was published in 1956.1 The results were sobering. Because of a much higher mortality in the cortisone group, the study was aborted after entering only 36 patients. We were told by our statisticians that we could not prove the drug to be dangerous, but it was so extremely unlikely that it would be beneficial that it would be unethical to continue. Only in fairly recent years did I learn that the Cochrane reviews report this study to be the first prospective, controlled, double-blind study ever performed in stroke.2 Numerous experiences such as this reinforced the importance of certain key lessons—lessons that have resulted in critical advancements in the past 40 years and that will continue to lead us to the advancements of the future.
We have learned not to trust general impressions or consensus opinions until tested by well-designed studies.
We have learned that even though science begins with brilliant original concepts and observations that when studied meticulously give us great insight into the isolated “possible,” they are not generalizable until tested in properly designed studies of the projected populations to which they may apply.
We have learned that basic science and laboratory studies are important to develop and test concepts, but the results are frequently disappointing when applied clinically.
This should not have been unexpected because the laboratory can limit all variables to the issue being tested. In clinical studies of humans, one cannot control or even recognize all possible variables. Evolution has produced a very complex system with many checks and balances recognized and unrecognized. The lack of effectiveness does not disprove the isolated concept but indicates that it does not exist isolated when other systems interact in a living functioning human being. Still, it is extremely important to test concepts in the laboratory and with animal studies before applying them to humans. Knowing what works in the laboratory might not be applicable clinically, but it is still important to establish the safety and effect in animals before initiating human trials.
We have learned that clinical studies require adequate numbers representing the population in which the results will be applied. They must be prospective, randomized, and, ideally, double-blinded, with statistical tests to exclude type 1 and type II errors determined as part of the original design.
Ignoring this lesson may lead to very serious errors. Because of the complexities and unknown characteristics that may affect results, it is important to try by randomization to get them equally represented in the study and the control group.
In recent years, some have downplayed the importance of double-blinding. This is probably because some studies involve surgical or invasive procedures that cannot ethically be applied to controls. It is disturbing that some investigators have the impression that as long as randomization is successful, blinding is not necessary. But the impact of recognized or unrecognized bias, no matter how subtle, should be obvious.
As an extreme example, in the 1960s during an interview with a hematologist who was being recruited for a position at Indiana University, we discussed why 2 prospective, randomized, controlled studies of anticoagulation at his present institution produced quite different results. The first was a pilot study and the second was part of a multicenter trial. The results in favor of the anticoagulant group were excellent in the randomized pilot study and poor in the multicenter blinded study. The explanation was not surprising. Those patients in the pilot studies who were assigned warfarin were monitored very closely, examined immediately for any symptom, frequently followed-up, and all risk factors were treated vigorously. Those who were assigned a placebo were asked to call or return if they had any events and were sent back to their referring physicians. Obviously, the study was biased, not by the differences in the use of anticoagulation, but by the better general treatment of risk factors. Most bias is much more subtle than this. Too often, investigators excuse their lack of rigidity of design by assuming they are honest. The investigator who considers the possibility of cheating will set-up an experimental design so tight that cheating will be impossible.
Although trial methodology and statistical tests are becoming more and more sophisticated, all are designed to estimate the real response rate for a study group and to compare this to an estimate of the real rate for the population studied. Typically, differences are accepted as real when a small likelihood exists that they occur by chance. Commonly, one accepts 1 chance in 20 (α=0.05) or 1 in 100 (α=0.01). These are the probabilities of concluding that differences are real when actually they are by chance. However, if no statistically significant differences occur, then do we assume that no real differences exist? If we do and there really is a difference, then a type II error has been made. To decrease the magnitude of a type II error, before any study is initiated, investigators must estimate the true response rates and what changes in these would be of practical importance at a clinical level. Then they can design their trial so that if the results show no significant differences, then they have reduced the likelihood of type II error. Many reviews of randomized trials have concluded that most are underpowered, and much attention is being given to this.
Obviously, the types, methods, and time of applying statistical tests must be determined as part of the original study design. For example, if one accepted differences that would only occur by chance in 1 in 20 trials, by applying the tests at 20 intervals during the study would almost certainly yield a difference when no difference was present in the general population.
We have learned that failure to publish negative studies may result in accepting the results of 1 “positive” study as being significant when many others show no effect.
In the past, journals tended to be prejudiced against negative studies, and a positive study was much more likely to be published. Understandably, when a therapeutic trial is sponsored by the maker of the agent being tested, a negative result can be most disconcerting and publication of the results discouraged. As an example, in 1999 at an international stroke meeting, an investigator of an industry-funded clinical trial announced from the podium that he was not permitted by the sponsor to present the results of the study. Because at that time an estimated 54% of randomized trials were commercially sponsored, this was an excellent example of how the literature could be biased by commercial control. If many studies were performed and negative ones not published, then few positive studies would lead to acceptance of potentially ineffective therapy. For example, if 20 trials were performed, one would expect by chance that 1 would show a difference at P<0.05. This precipitated an invited Stroke editorial and a cover illustration, “Weighing the Evidence: The Scales Are Unbalanced,” concluding that authors as individuals should not enter into agreements that interfere with their control over the decision to publish. The editor of Stroke then established a formal policy requiring investigators to document the control by sponsors over study design, data collection, analysis, interpretation, and writing of the report.
We have learned that the results of studies cannot be extrapolated to groups not included in the study design.
For example, the early studies of thrombolysis initiated within 72 hours of stroke showed no clinical benefit and increased hemorrhage. For a long time, on the basis of these studies, thrombolytic agents were considered to be ineffective and likely dangerous, even though the authors raised the possibility that treatment initiated within hours might show a favorable difference. Eventually, studies designed to respond to all the lessons that we have learned were performed. These studies ultimately established the effectiveness of thrombolysis using a tissue plasminogen activator if instituted within the first 3 hours of the stroke.
We have learned that major advancements in prevention and treatment are not easily extrapolated to clinical practice.
For example, despite the established benefit of the treatment of hypertension, cessation of cigarette smoking, anticoagulation or cardioversion for atrial fibrillation, statins, and thrombolytic agents, most of the population are not receiving these effective therapies. For example, it is estimated that >70 million people in the United States have high blood pressure but only 70% are being treated and <50% have high blood pressure under control. More than one-fifth of the adult population in the United States still smoke. Although the percentages are increasing, surveys in the United States reveal that <5% of those interviewed could name 3 risk factors and <20% could name 3 warning signs for stroke or had ever heard about tissue plasminogen activator treatment. From this lesson, it is imperative that we develop more and better delivery systems.
We have learned that major breakthrough therapeutic trials will be published in premier, widely distributed, nondisease-oriented journals like the New England Journal of Medicine.
These are made possible by the many basic and preliminary studies that establish the rationale for the definitive studies that are published in Stroke. Also, Stroke publishes the posttrial studies that help define and clarify the application of the results.
And finally, we have learned that all these processes are accelerated by the input of a unique and superb editor, such as the one we have had during the past 10 years. Vladimir Hachinski and his associate editors have rigorously applied these lessons during their tenure, adjusted to the new age of electronic publishing, and made many innovations that have encouraged the advancement of our knowledge. We who have observed these contributions are not surprised that our high opinions are supported by objective evidence. For example, the impact factor of Stroke has steadily increased and, when last recorded, had reached a new record of 6.499 in 2008. Thank you, Vladimir.
In conclusion, I apologize for “carrying coals to New Castle.” What I have discussed is simple and elementary to those working primarily in the field of stroke but needs to be constantly reemphasized because it is the basis for acquisition of future knowledge. For example, most journals publishing randomized, controlled trials, including Stroke, require adherence to the CONSORT (Consolidated Standards of Reporting Trials) statement that incorporates many of the lessons we have learned in a checklist.3
We are in good hands, but these simple lessons must never be forgotten.
Presented at Stroke Journal Pre-Conference Symposium, As Stroke Turns 40—Committing to a Stroke Global Action Plan, February 26, 2010, San Antonio, Texas.
- Received March 9, 2010.
- Accepted March 10, 2010.