| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2003;34:565.)
© 2003 American Heart Association, Inc.
Research Report |
From the Division of Preventive Medicine, Department of Medicine, Brigham and Womens Hospital (M.A., T.K., K.B., J.E.B.) and Department of Ambulatory Care and Prevention (J.E.B.), Harvard Medical School, Boston, Mass; Department of Epidemiology, Harvard School of Public Health, Boston, Mass (T.K., J.E.B.); Institute of Epidemiology and Social Medicine (K.B.), University of Muenster, Muenster, Germany; and Department of Neurology, Boston University Medical Center, Boston, Mass (C.S.K.).
Correspondence to Tobias Kurth, MD, MSc, Brigham and Womens Hospital, Division of Preventive Medicine, 900 Commonwealth Ave E, Boston, MA 02215-1204. E-mail tkurth{at}rics.bwh.harvard.edu
Abstract
Background and Purpose Accurate classification of stroke events is essential in large cohort studies of risk factor assessment and in treatment trials such as the Womens Health Study (WHS).
Methods Based on medical record review, we assessed interrater reliability in stroke classification and disability status in the WHS using the
statistic.
Results During 7.0 years of follow-up, 271 incident strokes occurred, of which 133 were reclassified. There was excellent interrater agreement in the diagnosis of major stroke types, hemorrhagic subtypes, and degree of disability, as well as substantial agreement in the definition of the vascular territory involved. Only moderate agreement was reached in the classification of ischemic subtypes.
Conclusions Major stroke types, degree of disability, and vascular territory involved can be reliably classified on the basis of a review of medical records in the WHS, whereas classification of ischemic stroke subtypes needs further refinement of diagnostic criteria.
Key Words: classification diagnosis interobserver variation stroke
Avalid and reproducible diagnosis of stroke is essential in the assessment of risk factors in observational studies. In addition, the differentiation of stroke subtypes and the determination of degree of disability are important factors in the evaluation of therapeutic agents to properly assess their effects.
Large epidemiological studies that have sufficient power to detect meaningful associations cannot include direct patient examinations and often have to rely solely on medical record information. We evaluated the interobserver reliability for stroke subtypes and degree of disability on the basis of medical record review in the Womens Health Study (WHS).
Methods
Subjects were participants in the WHS, an ongoing clinical trial of low-dose aspirin and vitamin E in the primary prevention of cardiovascular disease and cancer.1 Study participants were 39 876 female health professionals
45 years at baseline. Information was collected by mailed questionnaire asking about newly diagnosed conditions, including stroke.
Stroke Classification
We asked participants who self-reported stroke for permission to obtain their medical records. Stroke was defined as a focal neurological deficit of vascular mechanism lasting >24 hours and confirmed by an end points committee. After extracting the information from the medical records, which included medical history, clinical information, and results of diagnostic tests, we classified strokes using a standardized stroke card. For stroke subtypes, the modified Trial of Org 10172 in Acute Stroke Treatment (TOAST) criteria2 were used. In addition, vascular territory was determined from clinical information or imaging results. Degree of disability was measured with the modified Rankin Scale3 from information from the hospital discharge summary. We randomly selected 134 of 271 stroke cases, of which 1 medical record was not available at the time of this study, leaving 133 strokes for this study. A second neurologist (M.A.), blinded to the first classification (by C.S.K.), independently reclassified those cases using the same medical records and standardized diagnostic card.
Statistical Analysis
We used Cohens
statistic4 to measure the agreement between the 2 raters. To measure the agreement of ordinal scales, we estimated quadratic-weighted
.5 This measure accounts for differences in the classification by giving partial agreement weights for near but not exact agreement. Generally,
>0.80 represents excellent agreement; 0.80
>0.60 is thought to be substantial agreement; 0.60
>0.40, moderate agreement; 0.40
>0.20, fair agreement; and 
0.20, slight or poor agreement.6
Results
After a mean follow-up of 7.0 years, 271 incident strokes occurred, of which 133 were reclassified. Information found in the medical records was heterogeneous, ranging from self-reports only to extensive diagnostic work-up and clinical information. Of 133 cases, 111 (83%) had CT studies, 77 (58%) had MRI studies, 54 (41%) had contrast angiography or MR angiography, 101 (76%) had vascular Doppler studies, and 57 (43%) had echocardiograms.
The interrater agreement on major stroke subtypes (ischemic, hemorrhagic, unknown type) was excellent, with
=0.96. Only 1 of 27 hemorrhagic strokes was reclassified as stroke of unknown type, whereas 1 of 2 strokes of unknown type was reclassified as hemorrhagic.
Table 1 shows the interobserver agreement in subtypes of ischemic stroke with only moderate overall agreement (
=0.49). The
value varied widely with substantial agreement in cardioembolic subtype (
=0.74) and moderate agreement in atherothrombotic and atheroembolic (
=0.59) and unknown mechanism (
=0.48) types. In contrast, agreement in hemorrhagic subtypes of stroke was excellent, with
=0.94.
|
Table 2 shows the agreement in the classification of the involved vascular territory. The initial category of stem and branches of the main arteries was collapsed into 1 category, which slightly increased the
value from 0.68 to 0.73. The highest disagreement was found in the basilar artery category, for which 3 of 7 strokes originally diagnosed in the basilar artery territory were attributed to the posterior cerebral artery territory.
|
Table 3 represents the interrater agreement on the degree of disability. Agreement was found in 58% of cases with a weighted
=0.89. Collapsing the modified Rankin Scale into categories of independent (0 to 1), moderately dependent (2 to 3), and severely dependent (4 to 5) yielded a weighted
of 0.86.
|
Discussion
We have shown that stroke reclassification based on medical record information in the WHS can be done reliably with an excellent interrater agreement in major stroke types, hemorrhagic stroke subtypes, and degree of disability. Additionally, we found substantial agreement in the vascular territory involved. In contrast, the agreement for the various ischemic stroke subtypes was only moderate.
Diagnosis of hemorrhagic stroke is greatly facilitated by the generally unequivocal CT features of its subtypes.7,8 The difficulty in classifying ischemic stroke might be explained by several factors. The classification of lacunar strokes depends heavily on interpretation of clinical features because CT neuroimaging may be negative. The lower reliability of clinical data alone is well known.9 The 2 physicians might have weighted discrepant information differently, a problem also encountered by others.10 Some variability will always remain because of inconsistencies in medical records and errors in data abstractions, even after reliability is improved through the use of a computerized algorithm to guide data abstraction.10 Considering the results of 2 prospective studies that showed different interrater agreement for neurological signs,11,12 our findings are not surprising.
Categorization of ischemic stroke is difficult because it still has to rely on poorly defined data such as stroke pathogenesis and vascular territory involved.2,13 Our findings are in line with those of the Physicians Health Study7 and others that showed variable interrater agreement and emphasize the need to refine the definitions. Because misclassification is likely, studies investigating ischemic stroke subtypes should be interpreted with caution.
Classification of stroke according to vascular territory involved showed substantial agreement. This is likely due to the high frequency of neuroimaging-based diagnoses in this group of subjects. The diagnostic value of neuroimaging data has been shown when both a vascular territory classification and the modified TOAST criteria are used.14
Our findings of interrater agreement in degree of disability confirm previous results.7 Patients with severe deficits and those with minor impairment can be classified more accurately than those in the intermediate range of disability. Similar results with the use of a 6-item modified Rankin Scale have been reported by other investigators3,9
In summary, our study demonstrates excellent interrater agreement on major stroke types and degree of disability, and substantial agreement on vascular territory involved using a classification system based on review of medical records. Classification of ischemic stroke subtypes is more difficult, leading to moderate levels of interobserver reliability. Further refinement of diagnostic criteria and widespread use of modern neuroimaging data are required for a reliable classification of ischemic stroke subtypes in observational studies and controlled clinical trials.
Acknowledgments
The study is supported by the National Institutes of Health (CA-47988 from the National Cancer Institute and HL-43851 from the National Heart, Lung, and Blood Institute). We are indebted to the participants in the WHS for their outstanding commitment and cooperation and to the entire WHS staff for their expert and unfailing assistance.
Received August 29, 2002; accepted September 12, 2002.
References
This article has been cited by other articles:
![]() |
R. P. Gelber, J. M. Gaziano, E. J. Orav, J. E. Manson, J. E. Buring, and T. Kurth Measures of Obesity and Cardiovascular Risk Among Men and Women J. Am. Coll. Cardiol., August 19, 2008; 52(8): 605 - 615. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Schurks, R. Y.L. Zee, J. E. Buring, and T. Kurth Interrelationships among the MTHFR 677C>T polymorphism, migraine, and cardiovascular disease Neurology, August 12, 2008; 71(7): 505 - 513. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kurth, M. Schurks, G. Logroscino, J M. Gaziano, and J. E Buring Migraine, vascular risk, and cardiovascular events in women: prospective cohort study BMJ, August 7, 2008; 337(aug07_1): a636 - a636. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Ay, E. M. Arsava, M. Vangel, B. Oner, M. Zhu, O. Wu, A. Singhal, W. J. Koroshetz, and A. G. Sorensen Interexaminer Difference in Infarct Volume Measurements on MRI: A Source of Variance in Stroke Research Stroke, April 1, 2008; 39(4): 1171 - 1176. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. Ay, T. Benner, E. Murat Arsava, K. L. Furie, A. B. Singhal, M. B. Jensen, C. Ayata, A. Towfighi, E. E. Smith, J. Y. Chong, et al. A Computerized Algorithm for Etiologic Classification of Ischemic Stroke: The Causative Classification of Stroke System Stroke, November 1, 2007; 38(11): 2979 - 2984. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kurth, B. M. Everett, J. E. Buring, C. S. Kase, P. M. Ridker, and J. M. Gaziano Lipid levels and the risk of ischemic stroke in women Neurology, February 20, 2007; 68(8): 556 - 562. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. M. Everett, T. Kurth, J. E. Buring, and P. M. Ridker The Relative Strength of C-Reactive Protein and Lipid Levels as Determinants of Ischemic Stroke Compared With Coronary Heart Disease in Women J. Am. Coll. Cardiol., December 5, 2006; 48(11): 2235 - 2242. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kurth, J. M. Gaziano, N. R. Cook, G. Logroscino, H.-C. Diener, and J. E. Buring Migraine and risk of cardiovascular disease in women. JAMA, July 19, 2006; 296(3): 283 - 291. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kurth, S. C. Moore, J. M. Gaziano, C. S. Kase, M. J. Stampfer, K. Berger, and J. E. Buring Healthy lifestyle and the risk of stroke in women. Arch Intern Med, July 10, 2006; 166(13): 1403 - 1409. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Di Legge, J. Fang, G. Saposnik, and V. Hachinski The impact of lesion side on acute stroke treatment Neurology, July 12, 2005; 65(1): 81 - 86. [Abstract] [Full Text] [PDF] |
||||
![]() |
I-M. Lee, N. R. Cook, J. M. Gaziano, D. Gordon, P. M Ridker, J. E. Manson, C. H. Hennekens, and J. E. Buring Vitamin E in the Primary Prevention of Cardiovascular Disease and Cancer: The Women's Health Study: A Randomized Controlled Trial JAMA, July 6, 2005; 294(1): 56 - 65. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kurth, J. M. Gaziano, K. M. Rexrode, C. S. Kase, N. R. Cook, J. E. Manson, and J. E. Buring Prospective Study of Body Mass Index and Risk of Stroke in Apparently Healthy Women Circulation, April 19, 2005; 111(15): 1992 - 1998. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. T. L. Wilson, A. Hareendran, A. Hendry, J. Potter, I. Bone, and K. W. Muir Reliability of the Modified Rankin Scale Across Multiple Raters: Benefits of a Structured Interview Stroke, April 1, 2005; 36(4): 777 - 781. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. M Ridker, N. R. Cook, I-M. Lee, D. Gordon, J. M. Gaziano, J. E. Manson, C. H. Hennekens, and J. E. Buring A Randomized Trial of Low-Dose Aspirin in the Primary Prevention of Cardiovascular Disease in Women N. Engl. J. Med., March 31, 2005; 352(13): 1293 - 1304. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kurth, M. A. Slomke, C. S. Kase, N. R. Cook, I.-M. Lee, J. M. Gaziano, H.-C. Diener, and J. E. Buring Migraine, headache, and the risk of stroke in women: A prospective study Neurology, March 22, 2005; 64(6): 1020 - 1026. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. S. Bowman, H. D. Sesso, J. Ma, T. Kurth, C. S. Kase, M. J. Stampfer, and J. M. Gaziano Cholesterol and the Risk of Ischemic Stroke Stroke, December 1, 2003; 34(12): 2930 - 2934. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Kurth, C. S. Kase, K. Berger, J. M. Gaziano, N. R. Cook, and J. E. Buring Smoking and Risk of Hemorrhagic Stroke in Women Stroke, December 1, 2003; 34(12): 2792 - 2795. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2003 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |