This Article Abstract Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by D'Olhaberriague, L. Articles by Mansbach, H. H. Search for Related Content PubMed PubMed Citation Articles by D'Olhaberriague, L. Articles by Mansbach, H. H.

(Stroke. 1996;27:2331-2336.)
© 1996 American Heart Association, Inc.

Articles

A Reappraisal of Reliability and Validity Studies in Stroke

Luis D'Olhaberriague, MD, PhD; Irene Litvan, MD; Panayiotis Mitsias, MD Harry H. Mansbach, MD

the Center for Stroke Research, Department of Neurology, Henry Ford Hospital and Health Sciences Center, Detroit, Mich (L.D'O., P.M., H.H.M.), and the Neuroepidemiology Branch, National Institute of Neurological Disorders and Stroke, National Institutes of Health, Bethesda, Md (L.D'O., I.L.).

Correspondence to Luis D'Olhaberriague, MD, PhD, Center for Stroke Research, Department of Neurology, K-11, Henry Ford Hospital and Health Sciences Center, 2799 W Grand Blvd, Detroit, MI 48202. E-mail ldrda@aol.com.

	Abstract

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 
Background The emergence of prophylactic and therapeutic interventions in stroke has been accompanied by the widespread use of stroke classifications and scales that measure deficit (stroke scales) or resulting long-term handicap (handicap and disability scales). Although the accuracy of some scales and classifications has been studied, there is no updated systematic review appraising all of them.

Review We reviewed the literature and selected 21 studies on classifications and scales. The International Classification of Diseases, 10th revision, achieved the highest interobserver agreement among seven stroke classifications. The National Institutes of Health Stroke Scale, the Canadian Neurological Scale, and the European Stroke Scale had the highest reliability across items among nine stroke scales. The Barthel Index was the most reliable disability scale.

Conclusions The identification of the most reliable stroke classifications and scales should encourage their use in selection of homogeneous populations of patients for clinical research studies and to improve communication among scientists. Further research is needed to investigate neglected aspects of the neurological examination and the validity of stroke classifications.

Key Words: cerebrovascular disorders • epidemiology • stroke assessment • stroke classification

	Introduction

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 
Stroke scales and classifications were developed to deal with the intrinsic difficulties in studying stroke by standardizing its study. The necessity of formalization is especially conspicuous in the era of acute stroke trials, when we need to include homogeneous populations of patients, to collaborate with multiple centers to achieve sufficient statistical power, or to compare results across studies. Furthermore, stroke is a difficult syndrome to study because of four variable characteristics: manifestations, prognosis, etiology, and recovery.¹ Clinical manifestations are addressed using the neurological examination or stroke scales, stroke classifications formalize the plurality of cause, and handicap and disability scales quantify prognosis and recovery.

To obtain "hard" clinical data, Feinstein^{2 3 4} developed the idea of clinimetrics.⁵ The main requirements of scientific quality in data management are reliability, internal consistency, and validity. Reliability includes interobserver and intraobserver agreement, ie, faithful repetition by different observers and by the same observer. Reliability is best tested with values.^{6 7 8} ranges between -1 for complete disagreement and +1 for perfect agreement (0 would be chance agreement) and can be categorized as poor (<0), slight (0<0.2), fair (0.2<0.4), moderate (0.4<0.6), substantial (0.6<0.8), and almost perfect (0.8<1).⁹ Internal consistency measures the variation within the assessment, ie, the degree to which the items measure one construct. It depends on the number of items in the scale and their correlation. Consistency is measured using the Kuder-Richardson formula or, when the number of responses is greater than two, with Cronbach's (a correlation coefficient).⁹ Consistency is considered good if >0.8 and excellent if >0.9.⁹ Low values may indicate that more than one construct is present in the scale. High values may signify good internal consistency, but they also appear when there is redundancy. Redundancy results in unnecessarily long scale administration, and it artificially raises consistency because items that measure the same thing logically highly correlate with each other. Reliability analyses do not detect validity problems, since a measurement could have good interobserver agreement, irrespective of whether it is valid. Thus, validity analyses are necessary. Validity is examined according to criterion, construct, and content. Criterion validity is the demonstration of the accuracy of an assessment compared with a particular "gold standard,"^{9 10 11} and it is measured with sensitivity, specificity, and predictive values. However, it is not clear which should be the gold standard for stroke. Neuropathology, the usual criterion, is rarely available, since stroke is seldom lethal in the acute phase.^{12 13} Brain CT and MRI are useful for differentiating between ischemic and hemorrhagic stroke,¹ but it is debatable whether certain neuroradiological patterns can discriminate among ischemic stroke subtypes.^{14 15} Construct validity is proved by examining the relationships between a newly created instrument of measurement and a previously existing one to show that both measure the same construct. Construct validity is especially useful when there is no gold standard and is measured using correlation coefficients. Content validity measures the extent to which an instrument of measure includes all relevant dimensions of what is being measured. Its assessment relies on expert opinions.

The proliferation of classifications and scales^{16 17 18 19 20 21 22 23 24 25 26 27 28} calls for a clarification,²⁹ more so in view of new and promising therapeutic advances in the field of stroke.^{30 31 32 33 34} In their excellent review on scales, Lyden and Lau⁹ pointed out the lack of explicit testing of accuracy of many scales, thus restricting their use. Currently, the problem is to select those classifications and scales with the greater accuracy.

	Materials, Methods, and Bibliography

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 
We reviewed the literature by using a computerized Medline search (1966 through October 1995; key words: stroke diagnosis, epidemiological determinants, interobserver agreement) and the collection of the journal Cerebrovascular Diseases (January 1991 through October 1995) not included in Medline. From 64 references, we included only those studies that submitted stroke classifications and scales to formal analysis. We excluded articles that did not undertake formal analyses. Tables 1 and 2 summarize the 17 studies on reliability of classifications and scales.^{35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51}

View this table: [in this window] [in a new window]   Table 1. Interrater Agreement for Stroke Classifications

View this table: [in this window] [in a new window]   Table 2. Interrater Agreement for Stroke and Disability Scales

	Studies on Stroke Classifications

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 
Reliability of Stroke Classifications
The reliability of seven stroke classifications was reported in five studies^{35 36 37 38 39} (Table 1). Four of them^{35 36 37 39} analyzed their own classifications (the TOAST ischemic stroke subtypes, the PSDB stroke subtypes, the Oxfordshire Community Stroke Project ischemic stroke subtypes, and the Physicians Health Study stroke subtypes), and one study³⁰ compared the ICD-9, the ICD-10, and the German Classification of Neurological Diseases. The classifications did not include the same outcome measures: the TOAST classification,³⁵ the PSDB,³⁶ the German Classification of Neurological Diseases,³⁸ and the Physicians Health Study³⁹ analyzed etiologic categories, the ICD-9 and ICD-10³⁸ etiologic and topographic categories, and the Oxfordshire Community Stroke Project³⁷ only topographic categories, whether the infarcts were lacunae or affected the anterior or posterior circulation. Reliability varied among seven classifications (Table 1), with ranging from fair (Physicians Health Study)³⁹ to substantial (ICD-10),³⁸ indicating that the ICD-10 was the most reliable. There is no obvious difference between the methods of study adopted, ie, clinical vignettes^{35 38} versus direct patient examination.^{36 37 39} Perhaps, for multicenter trials or in the acute phase of stroke, when evolution of clinical phenomena is so rapid, it may be more suitable to use vignettes or videos.⁵² However, in the chronic phase, when changes in the clinical status are not expected, direct patient examination may be preferred.

In an attempt to improve reliability, some researchers proposed eliminating etiologic diagnosis from their classifications. However, when we compared the different classifications, a purely topographical classification³⁷ was not more reliable than many etiologic classifications.^{35 36 38} Moreover, eliminating etiologic diagnosis is not useful because various treatments of proven value or under testing are dependent on stroke mechanisms.^{30 31 32 33 34} Brain CT and MRI resolved the issue of differential diagnosis between ischemic and hemorrhagic stroke,¹ but it is debatable whether conventional MRI improves the sensitivity of CT in the acute phase.^{53 54}

All but two^{20 39} of the classifications^{17 18 19 21 22 35 36 38} converge into five ischemic stroke subtypes: atherothrombotic, cardioembolic, small-vessel disease, infarction of unknown pathogenesis, and other. Thus, it seems reasonable to develop a consensus on these five etiologic categories. In the meantime, we encourage the adoption of the ICD-10 because it is the most reliable.

Validity of Stroke Classifications
Three studies evaluated validity of two stroke classifications.^{55 56 57} The TOAST researchers⁵⁵ described that in 35% of patients the initial diagnosis changed with the results of ancillary investigations. When these investigators⁵⁶ used the final diagnosis (3 months after stroke) as a gold standard for their five ischemic stroke subtypes (see above), the initial diagnosis changed in 38% of cases. The sensitivity and specificity for the diagnosis varied according to the subtype. The sensitivity ranged from 18% for uncommon causes of stroke to 81% for small-vessel disease, whereas specificity ranged from 18% for small-vessel disease to 100% for uncommon causes of stroke. It is important to note the frequency of initial misdiagnosis. While these investigators detected small-vessel disease strokes, they also made many false-positive misdiagnoses. They detected few patients with uncommon causes of stroke but made no false-positive misdiagnoses.

When the Lausanne Stroke Registry classification⁵⁷ was evaluated, the initial topographic diagnosis was revised in 4% of the patients with the results of full radiological investigations, and the initial clinical diagnosis was revised in 3% of the patients with the results of full ancillary investigations.

	Studies on Stroke Scales and Neurological Examination

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 
Reliability and Consistency
Nine studies^{37 39 40 41 42 43 44 45 46 47 49 50 51} evaluated the reliability of nine stroke scales and neurological examination using direct patient examination (Table 2). (Not included here is the study of Edwards et al,⁵⁸ since it mainly refers to patients with diagnoses other than ischemic stroke.) All but two studies measured agreement with ,^{37 40 41 42 43 44 46} using the index of crude agreement⁴⁷ and Kendall's coefficient of agreement.⁴⁵ With all studies combined, 21 features of the neurological examination were evaluated (Table 3). Ten elements, evaluated by at least five studies, included language, weakness (face, arm, and leg), level of consciousness, speech, cerebellar signs, visual fields, extraocular movements, and hand sensory loss. Substantial to almost-perfect agreement was found in three of these 10 features (language and motor power in the arm and leg) (Table 3) in three of the nine studies. Agreement for each feature varied in the different studies. Agreement for facial weakness ranged from slight to near perfect; for level of consciousness, agreement ranged from fair to perfect; and overall agreement on sensory function, speech, cerebellar signs, visuospatial dysfunction, and visual fields ranged from fair to moderate. The differences in overall interobserver agreement between the studies of Brott et al⁴¹ (0.69) and of Goldstein et al⁴² (0.48) are likely related to the administration of different versions of the NIHSS, since sample size (24 versus 20), number of items in the scale (15 versus 13), and raters' expertise (a senior neurologist, a resident, and two nurses versus four stroke fellows) cannot explain the differences. In the study of Brott et al, raters had almost perfect (=0.95) agreement in the pupillary response, thus raising the overall value of that version of the NIHSS; this item and the plantar response, whose was 0.67, were eliminated in the NIHSS version presented in the study of Goldstein et al. Agreement on pure motor syndrome⁴⁰ was slight (=0.10), suggesting that some lacunar syndromes were difficult to agree on.^{59 60 61 62 63 64 65} Testing of vascular findings and tone is strikingly lacking. Agreement in carotid auscultation was assessed in only two studies.^{40 47} Some studies found a low degree of carotid bruit detection in patients with Doppler ultrasound–proved severe carotid stenosis.^{22 66 67 68 69 70 71} Tone was tested only with the Unified Neurological Stroke Scale,⁴⁵ despite the prognostic importance of hypotonia.

View this table: [in this window] [in a new window]   Table 3. Interrater Agreement of Different Stroke Scales

The internal consistency of the Hemispheric Stroke Scale (=0.88) was greater than that of either the Toronto Stroke Scale (=0.72) or the Mathew Scale (=0.54).⁷² Internal consistencies of the CNS and European Stroke Scale were also high.⁴⁴ Scales with higher agreement through items and higher consistency were the NIHSS,⁴¹ the CNS,⁴⁴ and the European Stroke Scale.⁴⁶

Validity Studies of Stroke Scales
Stroke scales measure the status of the patient after the ictus. They are useful to monitor acute status, evolution,⁷³ and treatment. Because stroke scales do not give a mathematically continuous measurement of the patient condition, nonparametric statistics should be used for their analysis.

Validity studies were performed on the NIHSS,⁴¹ CNS,^{44 74} European Stroke Scale,⁴⁶ and Scandinavian Stroke Scale.⁴⁸ The initial NIHSS correlated highly with infarction volume on the day-7 CT scan (r=.78) and with the 3-month outcome (r=.71).⁴¹

The CNS score correlated highly with neurological examination (r=.77)⁴⁴ and was predictive of the 6-month outcome: low initial scores correlated with higher mortality, greater incidence of recurrent vascular events, and lower independence at 6 months.⁴⁴ To evaluate the construct validity of the CNS and Glasgow Coma Scale, both were compared with the neurological examination.⁴⁴ The CNS correlated better with the examination (r=.76) than the Glasgow Coma Scale (r=.56).⁴⁴ Content validity of the CNS showed that an ideal scale should (1) use simple and nonambiguous definitions of each item, (2) have a minimal number of grades per modality, (3) be relevant for modalities that are frequently impaired by stroke, (4) be easy to use by observers with different degrees of medical training, (5) be brief, and (6) be practical and simple.⁷⁴

The European Stroke Scale⁴⁶ has good construct validity, and there is a high correlation between the European Stroke Scale and other scales (r=.93 for the Middle Cerebral Artery Neurological Scale, r=.95 for CNS, and r=.94 for Scandinavian Stroke Scale). The Scandinavian Stroke Scale also has good construct validity when compared with the Fugl-Mayer Scale, the Mathew Scale, the Toronto Scale, and the BI,⁴⁸ and it correlates with the other scales (r=.95 for Fugl-Mayer Scale, r=.94 for Mathew Scale, r=.92 for Toronto Scale, and r=.94 for BI).

We thus encourage the use of the NIHSS, the CNS, or the European Stroke Scale, which are the most accurate.

	Studies on Reliability of Disability and Handicap Scales

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 
Measurement of the poststroke status involves impairment (measured by the stroke scales), disability, and handicap.^{16 75 76 77 78} Disability refers to the capacity to perform a task and entails performance and capacity. Finally, handicap is a disadvantage that limits or prevents the fulfillment of a role that was normal for that individual. Therefore, there are performance scales (measuring what the person does); capacity scales (scales of activities of daily living that measure what the person can do); and handicap scales (including comorbidity and social factors).

Table 2 summarizes four studies^{39 49 50 51} on the reliability of the disability scales. Three studies used direct patient examination,^{39 49 51} and one used vignettes.⁵⁰ All studies^{39 49 50 51} evaluated reliability of a performance scale (the Stroke Severity Scale or the Rankin Scale), and one⁵¹ also assessed the BI of activities of daily living.⁷⁹ The reliability of the Stroke Severity Scale was substantial,³⁹ and that of the Rankin Scale score ranged from substantial to near perfect.^{49 50 51} When the BI and the Rankin score were directly compared, the interobserver agreement of the BI was greater (0.88 versus 0.75).⁵¹ In addition, the internal consistency () of the BI was high (0.96),⁷² and the BI highly correlated with poststroke status as measured by the Mathew, Toronto, and Hemispheric Stroke Scales.⁷²

It remains unclear whether a single scale will convey as much information as some combination of a disability and a handicap scale.^{80 81 82} An innovative approach^{34 83} consists of the generation of a scale that integrates a stroke scale, a scale of activities of daily living, and a performance scale to generate global statistics.

The BI is the most reliable disability scale and highly correlates with the immediate poststroke condition.⁵¹

	Conclusion

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 
A systematic review enabled us to select the ICD-10, the CNS, the NIHSS, the European Stroke Scale, and the BI as the most reliable instruments of clinical measurement in stroke. Thus, we encourage their use because the universal application of stroke scales will allow us to select homogeneous populations of patients for clinical trials and will improve communication among scientists. Further research is still needed to investigate the validity of most stroke classifications and some neglected aspects of the neurological examination. Sorting out instruments of clinical measure should be considered a priority in stroke research. However, the definitive way of doing this is by prospectively evaluating these instruments in a sample of patients that is representative of both those entering clinical trials and the entire stroke population.

	Selected Abbreviations and Acronyms

 

BI = Barthel Index CNS = Canadian Neurological Scale ICD-10 = International Classification of Diseases, 10th revision ICD-9 = International Classification of Diseases, 9th revision NIHSS = National Institutes of Health Stroke Scale PSDB = Pilot Stroke Data Bank TOAST = Trial of ORG 10172 in Acute Stroke Treatment

	Acknowledgments

 
This work was supported in part by NIH grant NS 23393; L. D'Olhaberriague receives the Jacob K. Javits NIH fellowship.

After the completion of this article, Muir et al compared the predictive validity of the NIHSS, CNS, and Middle Cerebral Artery Neurological Score (Muir KW, Weir CJ, Murray GD, Povey C, Lees KR. Comparison of neurological scales and scoring systems for acute stroke prognosis. Stroke. 1996;27:1817-1820).

Received May 29, 1996; revision received August 7, 1996; accepted August 7, 1996.

	References

Top Abstract Introduction Materials, Methods, and... Studies on Stroke... Studies on Stroke Scales... Studies on Reliability of... Conclusion References

 

1. Bamford J. Clinical examination in diagnosis and subclassification of stroke. Lancet. 1992;339:400-402.[Medline] [Order article via Infotrieve]
2. Feinstein AR. An additional basic science for clinical medicine, I: the constraining fundamental paradigms. Ann Intern Med. 1983;99:393-397.
3. Feinstein AR. An additional basic science for clinical medicine, III: the challenges of comparison and measurement. Ann Intern Med. 1983;99:705-712.
4. Feinstein AR. An additional basic science for clinical medicine, IV: the development of clinimetrics. Ann Intern Med. 1983;99:843-848.
5. Asplund K. Clinimetrics in stroke research. Stroke. 1987;18:528-530.[Free Full Text]
6. Posner KL, Sampson PD, Caplan RA, Ward RJ, Cheney FW. Measuring interrater reliability among multiple raters: an example of methods for nominal data. Stat Med. 1990;9:1103-1115.[Medline] [Order article via Infotrieve]
7. Cyr L, Francis K. Measures of clinical agreement for nominal and categorical data: the kappa coefficient. Comput Biol Med. 1992;22:239-246.[Medline] [Order article via Infotrieve]
8. Gross ST. The kappa coefficient of agreement for multiple observer when the number of subjects is small. Biometrics. 1986;42:883-893.[Medline] [Order article via Infotrieve]
9. Lyden PD, Lau GT. A critical appraisal of stroke evaluation and rating scales. Stroke. 1991;22:1345-1352.[Abstract/Free Full Text]
10. Mittl RL, Broderic M, Carpenter JP, Goldberg HI, Listerud J, Mishkin MM, Berkowitz HD, Atlas SW. Blinded-reader comparison of magnetic resonance angiography and duplex ultrasonography for carotid artery bifurcation stenosis. Stroke. 1994;25:4-10.[Abstract]
11. Alexandrov AV, Bladin CF, Maggisano R, Norris JW. Measuring carotid stenosis: time for a reappraisal. Stroke. 1993;24:1292-1296.[Abstract/Free Full Text]
12. Bamford J, Dennis M, Sandercock P, Burn J, Warlow CP. The frequency, causes, and timing of death within 30 days of a first stroke: the Oxfordshire Community Stroke Project. J Neurol Neurosurg Psychiatry. 1990;53:824-829.[Abstract]
13. Silver FL, Norris JW, Lewis AJ, Hachinski VC. Early mortality following stroke: a prospective review. Stroke. 1984;15:492-496.[Abstract/Free Full Text]
14. Ringelstein EB, Koschorke S, Holling A, Thron A, Lambertz H, Minale C. Computed tomographic patterns of proven embolic brain infarctions. Ann Neurol. 1989;26:759-765.[Medline] [Order article via Infotrieve]
15. D'Olhaberriague L, Welch KMA, Nagesh V, Gymnopoulos C, Boska MD, Levine SR, Ewing JR, Schultz LR, Chopp M. Diffusion weighted and T2 MRI can discriminate embolic from non-embolic stroke in humans. Stroke. 1996;27:180. Abstract.
16. Van Gijn J. Measurement of outcome in stroke prevention trials. Cerebrovasc Dis. 1992;2(suppl 2):23-34.
17. Special Report from the National Institute of Neurological Disorders and Stroke: Classification of Cerebrovascular Diseases III. Stroke. 1990;21:637-676.[Free Full Text]
18. Hachinski VC. Classification of stroke for clinical trials. Stroke. 1990;21(suppl II):II-27-II-29.
19. Mohr JP, Sacco RL. Classification of ischemic strokes. In: Barnett HJM, Stein BM, Mohr JP, Yatsu FM, eds. Stroke: Pathophysiology, Diagnosis, and Management. 2nd ed. New York, NY: Churchill Livingstone Inc; 1992:271-283.
20. Bamford J, Sandercock P, Dennis M, Burn J, Warlow C. Classification and natural history of clinically identifiable subtypes of cerebral infarction. Lancet. 1991;337:1521-1526.[Medline] [Order article via Infotrieve]
21. WHO Task Force on Stroke and Other Cerebrovascular Diseases. Recommendations on stroke prevention, diagnosis, and therapy. Stroke. 1989;20:1407-1431.[Free Full Text]
22. Bogousslavsky J, Van Melle G, Regli F. The Lausanne Stroke Registry: analysis of 1000 consecutive patients with first stroke. Stroke. 1988;19:1083-1092.[Abstract/Free Full Text]
23. Boysen G. The Scandinavian Neurological Scale. In: Candelise L, ed. Stroke scores and scales. Cerebrovasc Dis. 1992;4:239-240.
24. Brott T. Utility of the NIH stroke scale. In: Candelise L, ed. Stroke scores and scales. Cerebrovasc Dis. 1992;4:241-242.
25. Orgogozo JM. Scaling neurological impairment after middle cerebral artery infarction. In: Candelise L, ed. Stroke scores and scales. Cerebrovasc Dis. 1992;4:242-243.
26. Cote R, Hachinski VC. The Canadian Neurological Scale. In: Candelise L, ed. Stroke scores and scales. Cerebrovasc Dis. 1992;4:243-244.
27. Rankin J. Cerebral vascular accidents in patients over the age of 60, II: prognosis. Scott Med J. 1957;2:200-215.[Medline] [Order article via Infotrieve]
28. Orgogozo JM, Dartigues JF. Methodology of clinical trials in acute cerebral ischemia: survival, functional and neurological outcome measures. Cerebrovasc Dis. 1991;1(suppl 1):100-111.
29. Van Gijn J, Warlow C. Down with stroke scales. In: Candelise L, ed. Stroke scores and scales. Cerebrovasc Dis. 1992;4:244-246.
30. EAFT (European Atrial Fibrillation Trial Study Group). Secondary prevention in non-rheumatic atrial fibrillation after transient ischemic attack or minor stroke. Lancet. 1993;342:1255-1262.[Medline] [Order article via Infotrieve]
31. Albers GW, Sherman DG, Gress DR, Paulseth JE, Petersen P. Stroke prevention in nonvalvular atrial fibrillation: a review of prospective randomized trials. Ann Neurol. 1991;30:511-518.[Medline] [Order article via Infotrieve]
32. North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med. 1991;325:445-453.[Abstract]
33. Hass WK, Easton JD, Adams HP, Pryse-Phillips W, Molony BA, Anderson S, Kamm B. A randomized trial comparing ticlopidine hydrochloride with aspirin for the prevention of stroke in high-risk patients. N Engl J Med. 1989;321:501-507.[Abstract]
34. National Institute of Neurological Disorders and Stroke rt-PA Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med. 1995;333:1581-1587. (This work was published after the bibliographical search but is included because of relevance.)[Abstract/Free Full Text]
35. Gordon DL, Bendixen BH, Adams HP Jr, Clarke W, Kappelle LJ, Woolson RF, and the TOAST investigators. Interphysician agreement in the diagnosis of subtypes of acute ischemic stroke. Neurology. 1993;43:1021-1027.[Abstract/Free Full Text]
36. Gross CR, Shinar D, Mohr JP, Hier DB, Caplan LR, Price TR, Wolf PA, Kase CS, Fishman IG, Carlingo S, Kunitz SC. Interobserver agreement in the diagnosis of stroke types. Arch Neurol. 1986;43:893-898.[Abstract]
37. Lindley RI, Warlow CP, Wardlaw JM, Dennis MS, Slattery J, Sandercock PAG. Interobserver reliability of a clinical classification of acute cerebral infarction. Stroke. 1993;24:1801-1804.[Abstract/Free Full Text]
38. Kessler C, Freyberger HJ, Dittmann V, Ringelstein EB. Interrater reliability in the assessment of neurovascular diseases. Cerebrovasc Dis. 1991;1:41-43.
39. Berger K, Case CS, Buring JE. Interobserver agreement in the classification of stroke in the Physicians Health Study. Stroke. 1996;27:238-242. (This work was published after the bibliographical search but is included because of relevance.)[Abstract/Free Full Text]
40. Shinar D, Gross CR, Mohr JP, Caplan LR, Price TR, Wolf PA, Hier D, Kase CS, Fishman IG, Wolf C, Kunitz SC. Interobserver variability in the assessment of neurologic history and examination in Stroke Data Bank. Arch Neurol. 1985;42:557-565.[Abstract]
41. Brott T, Adams HP Jr, Olinger CP, Marler JR, Barsan WG, Biller J, Spilker J, Holleran R, Eberle R, Herztberg V, Rorick M, Moonaw CJ, Walker M. Measurement of acute cerebral infarction: a clinical examination scale. Stroke. 1989;20:864-870.[Abstract/Free Full Text]
42. Goldstein LB, Bertels C, Davis JN. Interrater reliability of the NIH stroke scale. Arch Neurol. 1989;46:660-662.[Abstract]
43. Gelmers HJ, Gorter K, de Weerdt CJ, Wiezer HJA. Assessment of interobserver variability in a Dutch multicenter study on acute ischemic stroke. Stroke. 1988;19:709-711.[Abstract/Free Full Text]
44. Cote R, Battista RN, Wolfson C, Boucher J, Adam J, Hachinski VC. The Canadian Neurological Scale: validation and reliability assessment. Neurology. 1989;39:638-643.[Abstract/Free Full Text]
45. Treves TA, Karepov VG, Aronovich AY, Gorbulev AY, Bornstein NM, Korczyn AD. Interrater agreement in evaluation of stroke patients with the Unified Neurological Stroke Scale. Stroke. 1994;25:1263-1264.[Abstract]
46. Hanston L, De Weerdt W, De Keyser J, Diener HC, Franke C, Palm R, Van Osrhoven M, Schoonderwalt H, De Klippel N, Herroelen L, Feys H. The European Stroke Scale. Stroke. 1994;25:2215-2219.[Abstract]
47. Tomassello F, Mariani F, Fieschi C, Argentino C, Bono G, De Zanche L, Inzitari D, Martini A, Perrone P. Assessment of inter-observer differences in the Italian multicenter study on reversible cerebral ischemia. Stroke. 1982;13:32-35.[Abstract]
48. Roden-Jullig A, Britton M, Gustaffson C, Fugl-Meyer A. Validation of four scales for the acute stage of stroke. J Intern Med. 1994;236:125-136.[Medline] [Order article via Infotrieve]
49. Van Swieten JC, Koudstaal PJ, Visser MC, Schouten HJA, Van Gijn J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1988;19:604-607.[Abstract/Free Full Text]
50. Bamford JM, Sandercock PAG, Warlow CP, Slattery J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1989;20:828. Letter.[Medline] [Order article via Infotrieve]
51. Wolfe CDA, Taub NA, Woodrow EJ, Burney PGJ. Assessment of scales of disability and handicap for stroke patients. Stroke. 1996;22:1242-1244.[Abstract/Free Full Text]
52. Lyden P, Brott T, Tilley B, Welch KMA, Mascha EJ, Levine S, Haley EC, Grotta J, Marler J, and the NINDS TPA Stroke Study Group. Improved reliability of the NIH Stroke Scale using video training. Stroke. 1994;25:2220-2226.[Abstract]
53. Bryan RN, Levy LM, Whitlow WD, Killiam JM, Preziosi TJ, Rosario JA. Diagnosis of acute cerebral infarction: comparison of CT and MR imaging. AJNR Am J Neuroradiol. 1991;12:611-620.[Abstract]
54. Mohr JP, Biller J, Hilal SK, Yuh WTC, Tatemichi TK, Hedges S, Tali E, Nguyen H, Mun I, Adams HP Jr, Grimsamn K. Magnetic resonance versus computed tomography imaging in acute stroke. Stroke. 1995;26:807-812.[Abstract/Free Full Text]
55. Adams HP Jr, Bendixen BH, Kappelle LJ, Biller J, Love B, Lee Gordon D, Marsh EE III, and the TOAST investigators. Classification of subtype of acute ischemic stroke. Stroke. 1993;24:35-41.[Abstract/Free Full Text]
56. Madden KP, Karanjia PN, Adams HP Jr, Clarke WR, and the TOAST investigators. Accuracy of initial stroke subtype diagnosis in the TOAST study. Neurology. 1995;45:1975-1979. (This work was published after the bibliographical search but is included because of relevance.)[Abstract]
57. Bogousslavsky J, Regli F, Besson G, Pinho e Melo T, Nater B. Early clinical diagnosis of stroke subtype. Cerebrovasc Dis. 1993;3:39-44.
58. Edwards DF, Chen YW, Diringer MN. Unified Neurological Stroke Scale is valid in ischemic and hemorrhagic stroke. Stroke. 1995;26:1852-1858.[Abstract/Free Full Text]
59. Mohr JP, Foulkes MA, Polis AT, Kase CS, Price TR, Tatemichi TK, Wolf PA. Infarct topography and hemiparesis profiles with cerebral convexity infarction: the Stroke Data Bank. J Neurol Neurosurg Psychiatry. 1993;56:344-351.[Abstract]
60. Fisher CM. Lacunar strokes and infarcts: a review. Neurology. 1982;32:871-876.[Abstract/Free Full Text]
61. Chamorro A, Sacco RL, Mohr JP, Foulkes MA, Kase CS, Tatemichi T, Wolf PA, Price TR, Hier DB. Clinical–computed tomographic correlations of lacunar infarction in the stroke data bank. Stroke. 1991;22:175-181.[Abstract/Free Full Text]
62. Bogousslavsky J. Brain or nosology lacunes. Cerebrovasc Dis. 1991;1:305. Editorial.
63. Arboix A, Marti-Vilalta JL, Garcia JH. Clinical study of 227 patients with lacunar infarcts. Stroke. 1990;21:842-847.[Abstract/Free Full Text]
64. Lodder J, Bamford J, Kappelle LJ, Boiten J. What causes false clinical predictors of small deep infarcts? Stroke. 1994;25:86-91.[Abstract]
65. Millikan C, Futrell N. The fallacy of the lacune hypothesis. Stroke. 1990;21:1251-1257.[Abstract/Free Full Text]
66. D'Olhaberriague L, Hernandez-Vidal A, Molina L, Soler-Singla LL, Marrugat J, Pons S, Moral A, Pou-Serradell A. A prospective study of atrial fibrillation and stroke. Stroke. 1989;20:648-652.
67. Bogousslavsky J, Hachinski VC, Boughner DR, Fox AJ, Vinuela F, Barnett HJM. Cardiac and arterial lesions in carotid transient ischemic attacks. Arch Neurol. 1986;43:223-228.[Abstract]
68. Bogousslavsky J, Hachinski VC, Boughner DR, Fox AJ, Vinuela F, Barnett HJM. Clinical predictors of cardiac and arterial lesions in carotid transient ischemic attacks. Arch Neurol. 1986;43:229-233.[Abstract]
69. Spitzer K, Becker V, Thie A, Kunze K. The Hamburg stroke data bank: goals, design and preliminary results. J Neurol. 1989;236:139-144.[Medline] [Order article via Infotrieve]
70. Foulkes MA, Wolf PA, Price TR, Mohr JP, Hier DB. The pilot stroke data bank: design, methods, and baseline characteristics. Stroke. 1988;19:547-554.[Abstract/Free Full Text]
71. Mohr JP, Caplan LR, Melski JW, Goldstein RJ, Duncan GW, Kistler JP, Pessin MS, Bleich HL. The Harvard Cooperative Stroke Registry: a prospective registry. Neurology. 1978;28:754-762.[Abstract/Free Full Text]
72. Brown EB, Tiejten GE, Deveshar RK, Ramadan NM, Levine SR, Dietrich KL, Nazareno DF, Welch KMA. Clinical stroke scales: an intra- and inter-scale evaluation. Neurology. 1990;40(S1):352. Abstract.
73. Wityk RJ, Pessin MS, Kaplan RF, Caplan LR. Serial assessment of acute stroke using the NIH Stroke Scale. Stroke. 1994;25:362-365.[Abstract]
74. Cote R, Hachinski VC, Shurvell BL, Norris JW, Wolfson C. The Canadian Neurological Scale: a preliminary study in acute stroke. Stroke. 1986;17:731-737.[Abstract/Free Full Text]
75. Gresham GE. Rehabilitation of the stroke survivor. In: Barnett HJM, Stein BM, Mohr JP, Yatsu FM, eds. Stroke: Pathophysiology, Diagnosis, and Management. 2nd ed. New York, NY: Churchill Livingstone Inc; 1992:1189-1201.
76. Task Force on Stroke Impairment. Task Force on Stroke Disability and Task Force on Stroke Handicap: symposium recommendations for methodology in stroke outcome research. Stroke. 1990;21(suppl II):II-68-II-73.
77. Roth EJ, Mueller K, Green D. Stroke rehabilitation outcome: impact of coronary artery disease. Stroke. 1988;19:42-47.[Abstract/Free Full Text]
78. Parikh RM, Robinson RG, Lipsey JR, Starkstein SE, Fedoroff P, Price T. The impact of poststroke depression on recovery in activities of daily living over a 2-year follow-up. Arch Neurol. 1990;47:785-789.[Abstract]
79. Mahoney FJ, Barthel DW. Functional evaluation: the Barthel Index. Md State Med J. 1965;14:61-65.[Medline] [Order article via Infotrieve]
80. Burn JPS. Reliability of the modified Rankin scale. Stroke. 1992;23:438. Letter.[Free Full Text]
81. Bloch RF. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1988;19:1448. Letter.[Free Full Text]
82. Hanston L, De Keyser J. Neurological scales in the assessment of cerebral infarction. Cerebrovasc Dis. 1994(suppl 2):7-14.
83. Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, Boysen G, Bluhmki E, Hoxter G, Mahagne MH, Hennerici M, for the ECASS Study Group. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke: the European Cooperative Acute Stroke Study (ECASS). JAMA. 1995;274:1017-1026.[Abstract]

This article has been cited by other articles:

				F. Song, C. Jerosch-Herold, R. Holland, M. d. L. Drachler, K. Mares, and I. Harvey Statistical methods for analysing Barthel scores in trials of poststroke interventions: a review and computer simulations Clinical Rehabilitation, April 1, 2006; 20(4): 347 - 356. [Abstract] [PDF]

				W.-C. Shyu, S.-Z. Lin, C.-C. Lee, D. D. Liu, and H. Li Granulocyte colony-stimulating factor for acute ischemic stroke: a randomized controlled trial Can. Med. Assoc. J., March 28, 2006; 174(7): 927 - 933. [Abstract] [Full Text] [PDF]

				P. J. Hand, J. Kwan, R. I. Lindley, M. S. Dennis, and J. M. Wardlaw Distinguishing Between Stroke and Mimic at the Bedside: The Brain Attack Study Stroke, March 1, 2006; 37(3): 769 - 775. [Abstract] [Full Text] [PDF]

				F. B. Young, C. J. Weir, K. R. Lees, and for the GAIN International Trial Steering Committe Comparison of the National Institutes of Health Stroke Scale With Disability Outcome Measures in Acute Stroke Trials Stroke, October 1, 2005; 36(10): 2187 - 2192. [Abstract] [Full Text] [PDF]

				M. Uyttenboogaart, R. E. Stewart, P. C.A.J. Vroomen, J. De Keyser, and G.-J. Luijckx Optimizing Cutoff Scores for the Barthel Index and the Modified Rankin Scale for Defining Outcome in Acute Stroke Trials Stroke, September 1, 2005; 36(9): 1984 - 1987. [Abstract] [Full Text] [PDF]

				J. F. Toole, M. R. Malinow, L. E. Chambless, J. D. Spence, L. C. Pettigrew, V. J. Howard, E. G. Sides, C.-H. Wang, and M. Stampfer Lowering Homocysteine in Patients With Ischemic Stroke to Prevent Recurrent Stroke, Myocardial Infarction, and Death: The Vitamin Intervention for Stroke Prevention (VISP) Randomized Controlled Trial JAMA, February 4, 2004; 291(5): 565 - 575. [Abstract] [Full Text] [PDF]

				PROGRESS Effects of a Perindopril-Based Blood Pressure-Lowering Regimen on Disability and Dependency in 6105 Patients With Cerebrovascular Disease: A Randomized Controlled Trial Stroke, October 1, 2003; 34(10): 2333 - 2338. [Abstract] [Full Text] [PDF]

				N U Weir, C E Counsell, M McDowall, A Gunkel, and M S Dennis Reliability of the variables in a new set of models that predict outcome after stroke J. Neurol. Neurosurg. Psychiatry, April 1, 2003; 74(4): 447 - 451. [Abstract] [Full Text] [PDF]

				M. Atiya, T. Kurth, K. Berger, J. E. Buring, and C. S. Kase Interobserver Agreement in the Classification of Stroke in the Women's Health Study Stroke, February 1, 2003; 34(2): 565 - 567. [Abstract] [Full Text] [PDF]

				D. L. Tirschwell, W.T. Longstreth Jr, K. J. Becker, R. E. Gammans Sr, L. A. Sabounjian, S. Hamilton, and L. B. Morgenstern Shortening the NIH Stroke Scale for Use in the Prehospital Setting Stroke, December 1, 2002; 33(12): 2801 - 2806. [Abstract] [Full Text] [PDF]

				M. J Bailey, M J. Riddoch, and P. Crome Treatment of Visual Neglect in Elderly Patients With Stroke: A Single-Subject Series Using Either a Scanning and Cueing Strategy or a Left-Limb Activation Strategy Physical Therapy, August 1, 2002; 82(8): 782 - 797. [Abstract] [Full Text] [PDF]

				C. Weimar, T. Kurth, K. Kraywinkel, M. Wagner, O. Busse, R. L. Haberl, and H.-C. Diener Assessment of Functioning and Disability After Ischemic Stroke Stroke, August 1, 2002; 33(8): 2053 - 2059. [Abstract] [Full Text] [PDF]

				B. C. Meyer, T. M. Hemmen, C. M. Jackson, and P. D. Lyden Modified National Institutes of Health Stroke Scale for Use in Stroke Clinical Trials: Prospective Reliability and Validity Stroke, May 1, 2002; 33(5): 1261 - 1266. [Abstract] [Full Text] [PDF]

				P. D. Lyden, M. Lu, S. R. Levine, T. G. Brott, J. Broderick, and R. Cote A Modified National Institutes of Health Stroke Scale for Use in Stroke Clinical Trials : Preliminary Reliability and Validity Editorial Comment : The NIH Stroke Scale: Is Simpler Better? Stroke, June 1, 2001; 32(6): 1310 - 1317. [Abstract] [Full Text] [PDF]

				D. Buck, A. Jacoby, A. Massey, and G. Ford Evaluation of Measures Used to Assess Quality of Life After Stroke Stroke, August 1, 2000; 31(8): 2004 - 2010. [Abstract] [Full Text] [PDF]

P. W. Duncan, H. S. Jorgensen, and D. T. Wade Outcome Measures in Acute Stroke Trials : A Systematic Review and Some Recommendations to Improve Practice Stroke, June 1, 2000; 31(6): 1429 - 1438. [Abstract]