| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2001;32:1091.)
© 2001 American Heart Association, Inc.
Original Contributions |
Improving the Reliability of Stroke Subgroup Classification Using the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) Criteria
From the Durham Veterans Affairs Medical Center (L.B.G., M.R.J., J.H., S.B.A., R.D.H.); Department of Medicine (Neurology [L.B.G., V.C.] and General Internal Medicine [D.B.M.]), Stroke Policy Program, Center for Clinical Health Policy Research (L.B.G., D.B.M.), Center for Cerebrovascular Disease (L.B.G., D.B.M., V.C., R.D.H.), Duke University, Durham, NC; and Department of Biostatistics, University of North Carolina, Chapel Hill (L.J.E.).
Correspondence to Larry B. Goldstein, MD, Duke Center for Cerebrovascular Disease, Stroke Policy Program, Center for Clinical Health Policy Research, Box 3651, Duke University Medical Center, Durham, NC 27710. E-mail golds004{at}mc.duke.edu
Background and Purpose—We sought to improve the reliability of the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) classification of stroke subtype for retrospective use in clinical, health services, and quality of care outcome studies. The TOAST investigators devised a series of 11 definitions to classify patients with ischemic stroke into 5 major etiologic/pathophysiological groupings. Interrater agreement was reported to be substantial in a series of patients who were independently assessed by pairs of physicians. However, the investigators cautioned that disagreements in subtype assignment remain despite the use of these explicit criteria and that trials should include measures to ensure the most uniform diagnosis possible.
Methods—In preparation for a study of outcomes and management practices for patients with ischemic stroke within Department of Veterans Affairs hospitals, 2 neurologists and 2 internists first retrospectively classified a series of 14 randomly selected stroke patients on the basis of the TOAST definitions to provide a baseline assessment of interrater agreement. A 2-phase process was then used to improve the reliability of subtype assignment. In the first phase, a computerized algorithm was developed to assign the TOAST diagnostic category. The reliability of the computerized algorithm was tested with a series of synthetic cases designed to provide data fitting each of the 11 definitions. In the second phase, critical disagreements in the data abstraction process were identified and remaining variability was reduced by the development of standardized procedures for retrieving relevant information from the medical record.
Results—The 4
physicians agreed in subtype diagnosis for only 2 of the 14 baseline
cases (14%) using all 11 TOAST definitions and for 4 of the 14 cases
(29%) when the classifications were collapsed into the 5 major
etiologic/pathophysiological groupings (
=0.42;
95% CI, 0.32 to 0.53). There was 100% agreement between
classifications generated by the computerized algorithm and the
intended diagnostic groups for the 11 synthetic cases. The
algorithm was then applied to the original 14 cases, and the
diagnostic categorization was compared with each of the 4
physicians’ baseline assignments. For the 5 collapsed subtypes, the
algorithm-based and physician-assigned diagnoses disagreed for 29% to
50% of the cases, reflecting variation in the abstracted data and/or
its interpretation. The use of an operations manual designed to guide
data abstraction improved the reliability subtype assignment (=0.54;
95% CI, 0.26 to 0.82). Critical disagreements in the abstracted data
were identified, and the manual was revised accordingly. Reliability
with the use of the 5 collapsed groupings then improved for both
interrater (=0.68; 95% CI, 0.44 to 0.91) and intrarater (=0.74;
95% CI, 0.61 to 0.87) agreement. Examining each remaining disagreement
revealed that half were due to ambiguities in the medical record
and half were related to otherwise unexplained errors in data
abstraction.
Conclusions—Ischemic stroke subtype based on published TOAST classification criteria can be reliably assigned with the use of a computerized algorithm with data obtained through standardized medical record abstraction procedures. Some variability in stroke subtype classification will remain because of inconsistencies in the medical record and errors in data abstraction. This residual variability can be addressed by having 2 raters classify each case and then identifying and resolving the reason(s) for the disagreement.
Editorial Comment
Classifying the Mechanisms of Ischemic Stroke
St James’s University Hospital, Beckett Street, Leeds, UK, j.m.bamford@leeds.ac.uk
This article has been cited by other articles:
 |
|
 |
|
  Y Iguchi, K Kimura, K Kobayashi, Y Ueno, K Shibazaki, and T Inoue Microembolic signals at 48 hours after stroke onset contribute to new ischaemia within a week J. Neurol. Neurosurg. Psychiatry, March 1, 2008; 79(3): 253 - 259. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H. Ay, T. Benner, E. Murat Arsava, K. L. Furie, A. B. Singhal, M. B. Jensen, C. Ayata, A. Towfighi, E. E. Smith, J. Y. Chong, et al. A Computerized Algorithm for Etiologic Classification of Ischemic Stroke: The Causative Classification of Stroke System Stroke, November 1, 2007; 38(11): 2979 - 2984. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. Saleh, M. Schroeter, A. Ringelstein, H.-P. Hartung, M. Siebler, U. Modder, and S. Jander Iron Oxide Particle-Enhanced MRI Suggests Variability of Brain Inflammation at Early Stages After Ischemic Stroke Stroke, October 1, 2007; 38(10): 2733 - 2737. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. G. Wiklund, W. M. Brown, T. G. Brott, B. Stegmayr, R. D. Brown Jr, S. Nilsson-Ardnor, J. A. Hardy, B. M. Kissela, A. Singleton, D. Holmberg, et al. Lack of aggregation of ischemic stroke subtypes within affected sibling pairs Neurology, February 6, 2007; 68(6): 427 - 431. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Y Iguchi, K Kimura, K Kobayashi, Y Ueno, and T Inoue Ischaemic stroke with malignancy may often be caused by paradoxical embolism J. Neurol. Neurosurg. Psychiatry, December 1, 2006; 77(12): 1336 - 1339. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. B. Goldstein and D. L. Simel Is This Patient Having a Stroke? JAMA, May 18, 2005; 293(19): 2391 - 2402. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C. Jackson and C. Sudlow Are Lacunar Strokes Really Different?: A Systematic Review of Differences in Risk Factor Profiles Between Lacunar and Nonlacunar Infarcts Stroke, April 1, 2005; 36(4): 891 - 901. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. C. Johnston Ischemic Preconditioning From Transient Ischemic Attacks?: Data From the Northern California TIA Study Stroke, November 1, 2004; 35(11_suppl_1): 2680 - 2682. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. B. Goldstein, G. P. Samsa, D. B. Matchar, and R. D. Horner Charlson Index Comorbidity Adjustment for Ischemic Stroke Outcome Studies Stroke, August 1, 2004; 35(8): 1941 - 1945. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
U.G.R. Schulz, E. Flossmann, and P.M. Rothwell Heritability of Ischemic Stroke in Relation to Age, Vascular Risk Factors, and Subtypes of Incident Stroke in Population-Based Studies Stroke, April 1, 2004; 35(4): 819 - 824. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. K. Lovett, A. J. Coull, and P. M. Rothwell Early risk of recurrence by subtype of ischemic stroke in population-based incidence studies Neurology, February 24, 2004; 62(4): 569 - 573. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
German Stroke Study Collaboration Predicting outcome after acute ischemic stroke: An external validation of prognostic models Neurology, February 24, 2004; 62(4): 581 - 585. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L.B. Goldstein, D.B. Matchar, J. Hoff-Lindquist, G.P. Samsa, and R.D. Horner VA Stroke Study: Neurologist care is associated with increased testing but improved outcomes Neurology, September 23, 2003; 61(6): 792 - 796. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
U.G.R. Schulz and P.M. Rothwell Differences in Vascular Risk Factors Between Etiological Subtypes of Ischemic Stroke: Importance of Population-Based Studies Stroke, August 1, 2003; 34(8): 2050 - 2059. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. Wassertheil-Smoller, S. Hendrix, M. Limacher, G. Heiss, C. Kooperberg, A. Baird, T. Kotchen, J. D. Curb, H. Black, J. E. Rossouw, et al. Effect of Estrogen Plus Progestin on Stroke in Postmenopausal Women: The Women's Health Initiative: A Randomized Trial JAMA, May 28, 2003; 289(20): 2673 - 2684. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. B. Goldstein, D. B. Matchar, J. Hoff-Lindquist, G. P. Samsa, R. D. Horner, and E. J. Kenton III Veterans Administration Acute Stroke (VASt) Study: Lack of Race/Ethnic-Based Differences in Utilization of Stroke-Related Procedures or Services * Diagnostic Disparities: Still Exist? Stroke, April 1, 2003; 34(4): 999 - 1004. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. D. Horner, J. W. Swanson, H. B. Bosworth, and D. B. Matchar Effects of Race and Poverty on the Process and Outcome of Inpatient Rehabilitation Services Among Stroke Patients Stroke, April 1, 2003; 34(4): 1027 - 1031. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Atiya, T. Kurth, K. Berger, J. E. Buring, and C. S. Kase Interobserver Agreement in the Classification of Stroke in the Women's Health Study Stroke, February 1, 2003; 34(2): 565 - 567. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
E. Z. Oddone, R. D. Horner, D. C.C. Johnston, K. Stechuchak, L. McIntyre, A. Ward, L. G. Alley, J. Whittle, L. Kroupa, and J. Taylor Carotid Endarterectomy and Race: Do Clinical Indications and Patient Preferences Account for Differences? Stroke, December 1, 2002; 33(12): 2936 - 2943. [Abstract] [Full Text] [PDF]
|
|