Letter by Saver et al Regarding Article, “Guidelines for the Prevention of Stroke in Patients With Stroke and Transient Ischemic Attack: A Guideline for Healthcare Professionals From the American Heart Association/American Stroke Association”
To the Editor:
We are writing to request correction of an error in Class Assignment and recommendation wording made in the recently published 2014 secondary stroke prevention guideline.1 The guideline assigns to patent foramen ovale (PFO) device closure treatment a Class III assignment and wording. But the evidence reviewed by the Committee supports a Class IIb, not Class III, recommendation and wording.
We think the Committee was led astray by a recent change in American Heart Association (AHA) levels of evidence procedures. In the past, AHA/American College of Cardiology (ACC)/American Stroke Association (ASA) Class III, level of evidence (LOE) A recommendations were reserved for cases in which multiple clinical trials clearly demonstrated definite harm from an intervention. However, recently, the AHA/ACC/ASA revised its evidence table to also permit Class III, LOE A recommendations when evidence demonstrates that an intervention confers no benefit. Guideline writers are likely not yet familiar with the types of trials needed to support such a finding of absence of benefit. The evidence reviewed by the committee does not support a finding of No Benefit per the AHA/ACC/ASA evidentiary table.
The standard evidence-based medicine requirement for a No Benefit claim rising to LOE A levels of support would be ≥2 equivalence or nonsuperiority trials actively proving absence of benefit.2,3 Such trials specify a margin (delta) within which 2 treatment strategies would be considered equivalent or noninferior and then prove that not only the point estimate, but also the 95% confidence interval for a treatment effect fall within this delta. If there were 2 equivalence or nonferiority RCTs confirming no benefit of PFO closure, a Grade A, LOE III recommendation would be supported. However, the Committee in its review of the evidence did not mention any equivalence or nonsuperiority trials (and we are not aware of any).
Instead, the Committee seems to have succumbed to the common fallacy of interpreting a superiority trial that is formally nonpositive as demonstrating neutrality.4,5 This is an unfortunate error, always to be avoided when trials have neutral point estimates, but their 95% confidence intervals do not exclude potential benefit and absolutely to be avoided when, as in PFO treatment, trials have favorable point estimates as well as favorable 95% confidence intervals.
To guard against this fallacy, the AHA/ASA Writing Committee would have needed to prespecify before undertaking its review an equivalence margin and then apply this margin to the superiority trials under review. If the 95% confidence intervals of multiple individual trials, or of the trials pooled together, fell within this margin, a nonsuperiority conclusion could perhaps be reached. The Writing Committee did not take this approach. If it had, the results would have clearly prevented a Class III recommendation from being advanced. The upper 95% margin for treatment effect on primary end points in the 3 trials analyzed all include ranges of substantial degree of benefit, with odds ratios for reduction in events of 0.22 (Randomized Evaluation of Recurrent Stroke Comparing PFO Closure to Established Current Standard of Care Treatment [RESPECT]), 0.24 (Percutaneous Closure of Patent Foramen Ovale in Cryptogenic Stroke [PC]), and 0.45 (STARFlex Septal Closure System in Patients With a Stroke and/or Transient Ischemic Attack Due to Presumed Paradoxical Embolism Through a Patent Foramen Ovale [CLOSURE]).
In fact, the Committee’s error is even worse than simply failing to take into account the 95% bounds of trials results. For each of the 3 trials, the point estimates themselves, as stated by the Committee, indicated benefit, including odds ratio 0.49 (RESPECT), odds ratio 0.63 (PC), and odds ratio 0.78 (CLOSURE). Consequently, the best available estimates of treatment effect (the point estimates) indicate that closure therapy is conferring gain. It is an extreme violation of the norms of evidence-based medicine to conclude that there is definitely no benefit of an intervention when the best estimate of treatment effect available from RCTs actually suggests benefit.
The evidence summary provided by the Committee provides support for at least a Level IIB recommendation. The AHA/ASA evidence table indicates that Class IIB should be used when Benefit≥Risk—when the available evidence suggests the treatment may have some benefit but the possibility of neutrality has not been excluded. That is exactly the state of the evidence for PFO device closure, according to the Committee: “Although the point estimates favored device closure to various degrees in each trial, none of the studies demonstrated a statistically significant finding for their primary end point in an intention-to-treat analysis.”
Accordingly, using only the Writing Committee’s own summary of the evidence, the AHA/ACC/ASA evidentiary process requires a Class IIb rather than a Class III recommendation. We request the guideline be corrected to so state.
Jeffrey L. Saver, MD
Department of Neurology and Comprehensive Stroke Center
David Geffen School of Medicine at UCLA
Los Angeles, CA
John D. Carroll, MD
Department of Cardiology
University of Colorado
Richard Smalling, MD
Department of Cardiology
University of Texas
David Thaler, MD
Department of Neurology
All authors are members of the Steering Committee for the RESPECT Trial, and they or their institutions receive payments from St Jude Medical for their services as scientific consultants regarding trial design and conduct.
Stroke welcomes Letters to the Editor and will publish them, if suitable, as space permits. Letters must reference a Stroke published-ahead-of-print article or an article printed within the past 3 weeks. The maximum length is 750 words including no more than 5 references and 3 authors. Please submit letters typed double-spaced. Letters may be shortened or edited.
The American Heart Association/American Stroke Association Manuscript Oversight Committee reviewed this letter on October 20, 2014.
Guest Editor for this article was Bo Norrving, MD, PhD.
- © 2015 American Heart Association, Inc.
- Kernan WN,
- Ovbiagele B,
- Black HR,
- Bravata DM,
- Chimowitz MI,
- Ezekowitz MD,
- et al
- Head SJ,
- Kaul S,
- Bogers AJ,
- Kappetein AP
- Altman DG,
- Bland JM