Convergent Validity and Interrater Reliability of Estimating the ABCD2 Score From Medical Records
Background and Purpose—The ABCD2 score is increasingly used for risk stratification of transient ischemic attack patients. We sought to determine the reliability and convergent validity of retrospective ABCD2 score estimation from medical records.
Methods—We compared ABCD2 scores that were prospectively determined by a vascular neurology attending to scores determined retrospectively from medical record review. Emergency department records and neurology consult notes for patients with acute transient ischemic attack were abstracted with explicit ABCD2 scoring redacted. Scores were estimated by 2 independent raters using these records. Estimated ABCD2 component scores, total scores, and risk category were compared both between retrospective raters and with prospectively obtained scores. Reliability was assessed using unweighted κ statistics.
Results—Interrater reliability was substantial with 72% exact agreement in total score between retrospective raters (κ=0.64) and nearly perfect with 82% agreement for ABCD2 category (κ=0.71). Interrater agreement was best for age and diabetes mellitus and poorest for clinical features and duration. Agreement between the retrospective raters and prospectively obtained score was >90% for age, blood pressure, and diabetes mellitus, but only ≈70% for clinical features and duration. Retrospectively, estimated total ABCD2 score exactly matched the prospective score in 58% of patients for rater 1 and 44% of patients for rater 2. Retrospectively, estimated ABCD2 category matched the prospectively scored category in 67% of patients for rater 1 and 71% of patients for rater 2.
Conclusions—The ABCD2 score can be abstracted from medical records with substantial interrater reliability but limited convergent validity. This may lead to misclassification of risk category in more than one third of patients.
- Received August 30, 2012.
- Revision received November 13, 2012.
- Accepted December 4, 2012.
- © 2013 American Heart Association, Inc.