Agreement Among Stroke Faculty and Fellows in Treating Ischemic Stroke Patients With Tissue-Type Plasminogen Activator and Thrombectomy
Background and Purpose—The aim of this study is to determine agreement among vascular neurology fellows and faculty in treating patients with acute ischemic stroke with intravenous tissue-type plasminogen activator and intra-arterial thrombectomy (IAT).
Methods—Patients were evaluated simultaneously by at least 2 vascular neurology. Agreement was determined using kappa (κ) and intraclass correlation coefficients.
Results—In 60 patients, agreement was substantial for tissue-type plasminogen activator (κ=0.75 [95% confidence interval, 0.57–0.92]) and IAT (κ=0.63 [95% confidence interval, 0.30–0.96]), with no difference between fellow–fellow versus fellow–faculty. Intraclass correlation coefficient for National Institutes of Health Stroke Scale was 0.94 (95% confidence interval, 0.90–0.97) and κ for Alberta Stroke Program Early CT Score was 0.53 (95% confidence interval, 0.20–0.78). Rapidly improving or mild deficits caused disagreement for both tissue-type plasminogen activator and IAT, whereas interpretation of computed tomographic perfusion led to disagreement for IAT.
Conclusions—We found substantial agreement between vascular neurology fellows and faculty in treating with tissue-type plasminogen activator or IAT. Areas for improvement include recognition of stroke mimics, consensus on treating less severe strokes, and use/interpretation of imaging.
The decision to treat acute ischemic stroke patients with tissue-type plasminogen activator (tPA) must be made quickly, without a definitive confirmatory test, but mild deficits or possible stroke mimics may pose a dilemma.
Intra-arterial thrombectomy (IAT) is also effective,1 depending on clot site and computed tomographic (CT) Alberta Stroke Program Early CT Score (ASPECTS); interpreting neuroimaging might also introduce treatment variability.
There are no data on agreement among vascular neurology (VNs) in treating acute ischemic stroke patients with tPA or IAT. Such data might establish a Quality Assessment benchmark for stroke on call systems
We prospectively determined agreement for tPA and IAT treatment between VN faculty and fellows and identified reasons for disagreement.
We evaluated patients presenting to our ED on weekdays from 7:00 am to 5:00 pm. A stroke code was paged to 10 fellows in our Accreditation Council for Graduate Medical Education (ACGME)-accredited VN fellowship program and 7 faculty with VN board certification on consecutive patients with suspected acute ischemic stroke within 6 hours of last known well. If CT showed no hemorrhage, the on call fellow sent an additional page and at least 1 faculty or additional fellow responded to the ED. Without delaying care, the second fellow/faculty observed the on call fellow’s history and examination, independently obtained additional history and examination if desired, assessed the CT (+/- CT angiogram), reviewed laboratory data, and documented their tPA and IAT treatment decision, reasons for excluding the patient from either therapy, and National Institutes of Health Stroke Scale (NIHSS) and ASPECTS. Each rater was blinded to the other raters’ decisions. Raters adhered to published guidelines for tPA or IAT,2 with consensus modifications of the VN program. Stroke mimic was identified after reviewing all clinical information, including magnetic resonance imaging if available.
the institutional review board approved this as a quality improvement project with exception from informed consent.
Means (SD) or medians (interquartile range) were reported for continuous and frequencies for categorical variables. The primary analytic population was patients who had at least 2 ratings by fellows, or a fellow and faculty when a second fellow was not available. Secondarily, we analyzed fellow–fellow, fellow–faculty, and 2 fellows and 1 faculty ratings. Inter-rater reliability for tPA, IAT, and ASPECTS was calculated by κ and 95% confidence interval (CI).3 The intraclass correlation coefficient with 95% CI measured agreement on NIHSS using 2-way mixed-effects models.4 Fisher exact test determined differences across clinical variables. We used STATA version 14 (StataCorp, College Station, TX). For agreement between 2 raters, we calculated that 60 patients would achieve 90% power to detect a true κ of 0.7 (significance 0.05 and null of 0.3).
More than 6 months, 60 consecutive patients were evaluated (median, NIHSS 5; interquartile range, 3–17). Twenty-nine had 3 ratings, whereas 31 had 2 (Figure). There were no differences between patients rated by different sets of raters (online-only Data Supplement). Our sample (41.7% ) was stroke mimics with no differences in baseline characteristics between true strokes and mimics.
Agreement With tPA and IAT
Positive and negative agreements and κ for the rating groups are in the Table (κ=0.41–0.60 represents moderate, κ=0.61–0.80 substantial, and κ=0.81–0.99 almost perfect agreements).5 Agreement for tPA between 2 VN was 88.3% (85.7% positive and 89.7% negative), κ=0.75 (95% CI, 0.57–0.92). Excluding mimics, κ=0.89 (95% CI, 0.73–1.0; n=35). κ's were similar for all rating groups. Agreement for IAT between 2 VN was 93.3% (80.0% positive and 94.5% negative), κ=0.63 (95% CI, 0.30–0.96).
Agreement Across Clinical and Demographic Variables
Age (≤57 and >57 years), NIHSS (≤5 and >5), sex, race, and ultimate diagnosis of mimic had no effect on agreement to treat with either therapy (online-only Data Supplement).
Frequency and Reasons for Disagreement With Treatment
Frequently, multiple reasons for excluding a patient from treatment were listed. Disagreement about tPA occurred in 7 cases; in 5 (71.4%), stroke mimic was suspected; rapidly improving or mild deficit was a reason in 4 (57.1%). Suspected intracerebral hemorrhage on imaging, history of intracerebral hemorrhage, late presentation, and unclear onset time were each a reason in 1 (14.3%). One of 17 patients who received tPA (6%) was later determined to be a mimic, but the raters all agreed on this case.
For IAT, disagreement occurred in 4 cases. Reasons included mild deficits (50%), absence of perfusion mismatch (50%), rapid improvement (25%), large infarct core on perfusion imaging (25%), low ASPECTS (25%), and late presentation (25%).
Inter-Rater Reliability in Determining NIHSS and ASPECTS
Agreement between 3 raters on NIHSS was almost perfect (intraclass correlation coefficient, 0.94; 95% CI, 0.90–0.97), and moderate for ASPECTS (agreement 85.3%; κ=0.53; 95% CI, 0.20–0.78).
In this first-of-its-kind analysis, we found substantial agreement among VN faculty and fellows to administer tPA or IAT to patients with acute ischemic stroke. Similar to studies measuring agreement on NIHSS and CT,6 our results might provide a benchmark for in-person or telemedicine tPA call programs.
Disagreement on tPA treatment occurred in 12% of cases. The most common reason was suspicion of stroke mimic. Although best identified before treatment, tPA is relatively safe in mimics.7
Study limitations include generalizing our results to all settings from a single academic center comprised VN faculty or fellows. Our cohort also included many stroke mimics. Consecutive patients were enrolled, representing a real-world sample evaluated at a comprehensive stroke center. Excluding mimics, agreement was almost perfect. Although our sample was sufficient to answer our primary aim, it was too small for subgroup analysis, and the 95% CI for κ include the possibility of less robust agreement. Finally, we cannot exclude some interaction among raters that may have influenced decisions, but such interaction, if present, was minimal.
In conclusion, we found substantial agreement among VN trainees and faculty in deciding to treat with tPA or IAT. Although reassuring, our data identify recognition of stroke mimics, weighting of minor stroke signs, and interpretation of brain imaging as topics for both additional research and quality improvement efforts.
We thank Tareq Almaghrabi, MD, Andrew Barreto, MD, Tiffany Cossey, MD, David Doan, MD, Kasey Gildersleeve, MD, Nicole Gonzales, MD, Amanda Jagolino, MD, Judy Jia, MD, Umair Saeed, MD, Anjail Sharrief, MD, MPH, and Yamin Shwe, MD, for their participation in data collection.
Guest Editor for this article was Eric E. Smith, MD, MPH.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.116.015214/-/DC1.
- Received August 23, 2016.
- Revision received September 24, 2016.
- Accepted October 14, 2016.
- © 2016 American Heart Association, Inc.
- Goyal M,
- Menon BK,
- van Zwam WH,
- Dippel DW,
- Mitchell PJ,
- Demchuk AM,
- et al
- Jauch EC,
- Saver JL,
- Adams HP Jr,
- Bruno A,
- Connors JJ,
- Demaerschalk BM,
- et al
- Smith EE,
- Fonarow GC,
- Reeves MJ,
- Cox M,
- Olson DM,
- Hernandez AF,
- et al
- et al
- Powers WJ,
- Derdeyn CP,
- Biller J,
- Coffey CS,
- Hoh BL,
- Jauch EC,
- et al
- Albers GW