Epistemology, Parachutes, and “Yeah, but” Interventional Stroke Trials
Epistemology is a branch of philosophy concerned with the nature and scope of knowledge. The term was introduced by the Scottish philosopher Ferrier.1 It seeks to address questions like What is knowledge? How do we know that we know something? To what extent is it possible for a particular subject to be known? Broadly speaking, epistemological knowledge can be divided into 2 categories: knowledge that and knowledge how. Most animal behavior, for example, a newborn bird learning to fly or a child learning to ride a bike, are of the knowledge how category. In contrast, most of the subsequent discussion surrounds the knowledge that category. This addresses questions like How do we know that the sum of the squares of the 2 sides of a right-angled triangle equals the square of the hypotenuse? or How do we know that antibiotics work for meningitis?
Humans are probably alone in the capacity not only to know something but also to know that they know it, and to consider the foundations of this knowledge. Although most of us know it would be a bad idea to jump from an airplane in flight without a parachute, it is harder to justify how we know that with such certainty. Each of us has clearly not tested that out for ourselves and most would agree it is absurd to test this as a hypothesis. Rather, our knowledge comes from experience in relevant parallel situations (eg, seeing what happens when one jumps from 10 feet and extrapolating it to 10 000 feet), mechanistic understanding of gravity, and inborn instincts for self-preservation. Much of this knowledge is arguably innate: a 4-year-old child knows when it is not safe to jump off a height; so does a baby monkey. But this raises a classic question of particular importance to interpreting evidence-based medicine: What qualifies as a parachute?
Let us take the example of meningitis and antibiotics. Meningitis was initially well recognized, but poorly understood: high fever, neck stiffness, delirium, and rash likely leading to death, frequently occurring in outbreaks. As the germ theory of disease slowly took shape, someone decided to look for germs in the cerebrospinal fluid of patients. Along the way, scientists started working on ways to kill bacteria and the era of antibiotics emerged. Tests in a petri dish could show whether a particular antibiotic works with a particular bug, and soon an antibiotic that kills the bacteria causing meningitis is discovered and tried in patients with excellent clinical outcomes.2 All this predated modern statistical theory and modern clinical trial design. Our ways of knowing that we know have changed, yet no one would now insist on a randomized double-blind placebo controlled trial to test whether antibiotics are effective for meningitis. They have become a parachute.
How do we know that this was correct? We can cite a combination of multiple factors: decades of accumulated positive experience, biological plausibility of mechanism, similarity of response to other infections, but the epistemological question remains: What qualifies as a parachute? When in medicine can we say that we know something with certainty?
Recently, these questions have recently become more than intellectual stimulation in stroke research. New developments in the science of stroke diagnosis and treatment exemplify the importance of this discussion, and highlight the fact that although the parachute question may be old, it is far from answered. Diffusion-weighted MR images have rapidly become a parachute in stroke practice, despite no randomized trial showing improved clinical outcomes, the use and predictive value3 is such that few stroke neurologists would voluntarily pass up the information provided by diffusion-weighted images.
But even when we have evidence, it may be put aside, explained away, or rapidly rendered obsolete requiring further trials. There is no better exemplar of this phenomenon than intra-arterial intervention for acute stroke. In late 1990s, thrombolytics began to be delivered intracranially, at the face of a clot, via microcatheter. Some people start reporting the superiority of this approach in achieving recanalization. However, the technique is complex, requires additional infrastructure, and has potential for adverse complications. A group of scientists and clinicians work on initial safety and dose data (Interventional Management of Stroke [IMS]-I and II)4,5 and then embarked on an National Institutes of Health–funded, peer-reviewed trial (IMS-III). At the outset of the phase III trial, it was widely thought that a positive trial would show conclusive proof of the superiority of a combined intravenous/intra-arterial approach to treat large vessel acute ischemic stroke. The designers of the trial planned for changes in technology and many new advances were incorporated into the trial. The trial was stopped early because of futility after enrolling 656 patients. There was no difference in the proportion of patients with good outcome (modified Rankin Scale, ≤2 at 90 days), or in the frequency of intracerebral hemorrhages or mortality. Subgroup analyses of very severe or moderate-severe cohorts also show no differences.6 Interestingly, the trial has failed but many beliefs have not changed at all. There are long lists of potential explanations for the futility of the combined intravenous/intra-arterial approach in the trial. Some cite changes in imaging paradigms, newer devices, overselected patient sampling (the authors acknowledge that future studies should be done in environments where intra-arterial interventions are not available outside the trial), and changes in process (eg, door-to-recanalization time). But at the heart of the matter lies a belief, common among those that use it, that intra-arterial therapy is better.
The results of the IMS-III trial raise interesting epistemological questions, especially concerning the value of, and response to, large negative trials. Before the start of IMS-III, a randomized controlled trial (RCT) was felt to be the best way of obtaining knowledge. Now that the trial is negative, there is a movement to consider the results of what was supposed to be a definitive trial now as preliminary. Is doing a trial simply a formality to obtain proof for what we believe to be true? Will we keep doing trials until we have one that is positive? Certainly, if enough RCT’s are performed on a given topic, at least one of them will turn out to be positive. Evidence-based practitioners may say that intra-arterial intervention is futile and that future trials should be abandoned.
With many negative trials, people learn and move to a series of “yeah, but” trials: studies designed to overcome the limitations of previous negative studies (“yeah, but we can use better technology”—the newer stentrievers with superior recanalization; “yeah, but we can select better patients”—randomize all consecutive patients and not allow procedures in patients outside of trials; “yeah, but we can treat faster”—minimize time to treatment). Critics may dismiss these trials as a series of vain attempts to prove a belief, and argue that further efforts should be abandoned. However, although a negative trial may render an approach obsolete, it may also be the stimulus for innovation. The “yeah, but” trials can pave the way for new standards of care. Intravenous tissue-type plasminogen activator is perhaps the most directly obvious example of a successful “yeah, but” trial. After many negative thrombolytic trials, the knowledge gained helped to develop a “yeah, but” trial with strict inclusion criteria that finally resulted in a positive result and a new standard of care.7
Ironically, although negative trials can spark improvements and refinements, it is an even more interesting corollary that a positive RCT may stop innovation: safer or more effective thrombolytics than tissue-type plasminogen activator may exist, but it is virtually impossible to test in the 3- to 4.5-hour window when we have a clear standard of care. There have been no new medical treatments for hyperacute stroke in the past 17 years. It is far easier to show something is “better than nothing” but much harder to test “better than pretty good”.
Why are those that use intra-arterial approaches so convinced that the approach is correct that new trials are already being planned in the wake of IMS-III? Is it because we have seen enough Lazarus responses living in the trenches? Is that sufficient? Can enough good responses make intra-arterial therapy a parachute? These other ways of knowing (trial and error, case studies, case–control/cohort, expert opinion), all have value in generating hypotheses and facilitating innovation. But they do not eliminate a multitude of biases, and the good experiences of single centers (or individual operators) cannot be extrapolated to other sites. Ironically, clinicians getting the best results from intra-arterial intervention (precisely the ones who would be most suited to randomizing patients for an RCT) may have the greatest concerns about subjecting their patients to the possibility of being randomized to what they know to be an inferior treatment. Although if those clinicians opt out of trials, studies are performed by clinicians who have not been getting good results and hence, are unsure about the approach. RCTs with broad involvement, and without patient treatment outside the setting of the trial, have potential to address this, but those who fund research will not pay for an infinite series of expensive trials, even if each one results in new learning, without eventual patient impact.
This generates an epistemological disconnect: we cannot simply do one trial and stop without learning from it. Yet if we continue with “yeah, but” trials, Where does it stop? How do we draw the line between innovation and fruitless repetition? Do we keep doing trials until we prove our a priori beliefs? How much negative evidence is required to abandon an approach and how innovative should “yeah, but” trials be to proceed?
So what are the solutions to these problems? We in the stroke research community, along with funding agencies and policy makers, need to address the epistemological questions of equipoise, consecutive randomization and treatment outside of clinical trials head-on. In the current era of limited financial resources, we also will be forced to address the question when do we stop after negative trials and which trials do we use to continue to drive innovation? In the longer term, changes to medical education in which our trainees are more accustomed to the idea of equipoise, evidence-based medicine, RCTs and less reliant on expert opinion and the results of the last similar case may be required. Finally, more discussion with those in other fields—the input and guidance from philosophers (who both know about thinking and think about knowing), and from social scientists (who evaluate what we do with that knowledge) will be necessary to help answer these questions. The investment of limited research funds and efforts may be guided by these deliberations.
In the meantime, new “yeah, but” trials should be encouraged when the lessons of a previous trial or new advances in the field are large enough to significantly differentiate new trials from their predecessors. The recent successes of stent-retrievers for recanalization have once again provided hope for intra-arterial approaches. This, combined with lessons learned from IMS-III, justify cautious optimism for future trials, but it remains to be shown that vessels can be opened quickly enough and safely enough to result in improved patient outcomes. To be successful, future interventional stroke trials will require academic discipline and rigour, a high level of commitment to testing the equipoise, good leadership, and a bit of luck.
- Received February 26, 2013.
- Accepted April 17, 2013.
- © 2013 American Heart Association, Inc.
- Ferrier JF
- Tee GH
- Khatri P,
- Neff J,
- Broderick JP,
- Khoury JC,
- Carrozzella J,
- Tomsick T
- Tomsick T,
- Broderick J,
- Carrozella J,
- Khatri P,
- Hill M,
- Palesch Y,
- et al