Abstract W P281: Inter-Rater Reliability of Clinical Decision Making Based on Recorded Telemedicine Assessments and Case Histories
Background: Patient outcomes are improved by reducing time, and increasing access, to stroke-specialist consultation and treatment. Telemedicine may ensure earlier and wider access to stroke-specialist consultation. However, reliable and accurate decision making is necessary by various professional groups in the emergency department in order for the telemedicine assessment to be initiated.
Aim: To explore the inter-rater reliability of clinical decision making by various health professionals, when using telemedicine assessments in acute stroke.
Method: Twenty-six telemedicine assessments in patients with suspected stroke (final/gold standard diagnosis: 16 strokes and 10 non-strokes) were recorded, edited and then presented with case histories. Health professionals viewed 6, randomly selected assessments and rated: whether ‘stroke or not’; level of confidence with their diagnosis. Data were analysed using Cohen’s kappa (K) with 95% confidence interval (CI) calculated using bias-corrected bootstrapping in STATA with 10000 replicates. Sensitivity and specificity were estimated.
Results: Forty raters: 13 stroke physicians, 15 stroke nurses, 12 non-stroke specific staff (NSS) (including 3 ED consultants, 8 registrars and 1 neurologist). Median of 8 raters per assessment.
For ‘stroke or not’, agreement between all health professions was fair (K=0.36, CI 0.22-0.54); correctly diagnosing whether the patient was ‘stroke or not’ on 8 occasions, Sensitivity 77% (98/128), Specificity 67% (45/67).
Agreement within each health professional group ranged from K=0.37 (stroke physicians and nurses) to K=0.43 (NSS group), although CIs were wide. Sensitivity ranged from 74% (stroke physicians) to 81% (NSS group); specificity from 58% (stroke physicians) to 74% (NSS group).
Median confidence was highest in 4 of the 8 on which all raters agreed the ‘stroke or not’ diagnosis.
Conclusion: Agreement was ‘fair’ overall and was similar for all groups, although CIs were wide. Nurses’ agreement or diagnostic accuracy were similar to stroke physicians and NSS doctors when rating telemedicine consultations.
Author Disclosures: C.E. Lightbody: None. P. Davies: None. H. Emsley: None. J. Fitzgerald: None. G. Ford: None. L. Howell: None. M.J. Leathley: None. A. McLoughlin: None. C. May: None. C. Price: None. C. Sutton: None. C. Watkins: None.
- © 2014 by American Heart Association, Inc.