| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2007;38:62.)
© 2007 American Heart Association, Inc.
Original Contributions |
From the Division of Neurology, Department of Neurosciences (M.D.C.), University of the Witwatersrand, Johannesburg, South Africa; the Division of Neurology, Department of Neurosciences (G.M.), University of the Witwatersrand, Johannesburg, South Africa; and the Department of Clinical Neurosciences (C.P.W.), Western General Hospital, Edinburgh, United Kingdom.
Correspondence to Dr Myles D. Connor, Division of Neurology, Department of Neurosciences, University of the Witwatersrand, 7 York Road, Parktown, 2193, South Africa. E-mail mconnor{at}mighty.co.za; connormd@medicine.wits.ac.za
| Abstract |
|---|
|
|
|---|
Methods— We assessed the stroke scores and the clinicians blind assessment of pathologic stroke type in consecutive black patients with stroke included in the Johannesburg Hospital Stroke Register over 23 months. We calculated the accuracy of the scores and clinicians compared with computed tomography brain scan (sensitivity, specificity, positive predictive value, likelihood ratio,
statistic).
Results— Two hundred twenty-two patients were scanned and assessed within 15 days. Sixty-two (28%) had cerebral hemorrhage and nine (4%) subarachnoid hemorrhage. Neither the Siriraj (sub-Saharan Africa) nor Guys Hospital score was accurate or offered much advantage over clinician assessment (sensitivity 0.60 and 0.34, specificity 0.88 and 0.95 for intracranial hemorrhage in the Siriraj Stroke Score and Guys Hospital Stroke Score, respectively; sensitivity 0.70 and 0.71, specificity 0.84 and 0.74, respectively, for ischemic stroke). Although the scores were more accurate when we used new cutoff points, they then failed to diagnose over 80% of stroke types.
Conclusions— The Siriraj Stroke Score and Guys Hospital Stroke Score are not sufficiently accurate for use in either epidemiologic studies or to guide clinical management in sub-Saharan Africa at present.
Key Words: Africa south of the Sahara South Africa cerebral hemorrhage cerebral infarction cerebrovascular accident diagnosis subarachnoid hemorrhage
| Introduction |
|---|
|
|
|---|
We cannot directly extrapolate the results from validation studies undertaken in populations outside SSA. First, because many of the scores used to distinguish pathologic stroke type use "atheroma markers" such as angina, intermittent claudication, diabetes, and myocardial infarction,6,7 our population does not have a high prevalence of extracranial atherosclerotic disease either in the general population or in patients with stroke.8,9 Second, our prevalence of intracerebral hemorrhage may be very different.10
Although there are several scores available, we chose to use the SSS and Guys Hospital Stroke Score (GHSS) because these required the least ancillary testing and investigation and appeared to be the simplest to use.6,7 The SSS only requires a history and examination, whereas the GHSS requires a chest x-ray and electrocardiogram in addition. We felt these scores were the most likely to succeed in settings where resources are scarce.
Our aim was to assess the accuracy of the GHSS and SSS in distinguishing between intracranial hemorrhage (cerebral hemorrhage and subarachnoid hemorrhage) and ischemic stroke in black patients with stroke in South Africa. Later in the study, we assessed whether the scores were more accurate than clinicians in distinguishing pathologic stroke type.
| Methods |
|---|
|
|
|---|
Case Ascertainment and Assessment
The JHSR was advertised around the hospital and at academic meetings, and every weekday, a dedicated stroke register assistant asked doctors and nurses in the emergency department, admission wards, and medical wards if they had admitted any patients with stroke or transient ischemic attack in the previous 24 hours or if any patients had had a stroke while in the hospital. Every weekday, she also reviewed ward admission books and the files of patients who had died during the previous 24 hours. Over weekends, a JHSR clinician searched the emergency wards and admission records for acute strokes. Physicians from the Division of Neurology then assessed all patients in detail, and one of the authors (M.C.) reviewed the vast majority of cases personally and all JHSR questionnaires. We defined stroke according to the World Health Organization criteria as "rapidly developing signs of focal (or global) disturbance of cerebral function, leading to death or lasting longer than 24 hours, with no apparent cause other than vascular."12
As part of the assessment of a patient with stroke, the register clinician documented the information required for the scores but did not calculate them. During the second period of the register (2001 to 2002), clinicians documented what they thought the likely pathologic stroke type was and whether they had made this assessment without knowledge of the brain imaging result. We did not provide the clinicians with diagnostic criteria or aids for assessing stroke type, because we wanted to find out if the scores would add to the bedside assessment of stroke type made by a South African clinician who did not have access to brain imaging. Thus, although the author (M.C.) confirmed the stroke diagnosis and decided on the inclusion of patients onto the JHSR, we used the initial stroke register clinicians assessment of stroke type in our analysis. Stroke cases were included if their self-defined ethnicity was black, if they had sufficient data available to calculate the stroke scores, and had undergone CT within 15 days of onset of their stroke to minimize the misdiagnosis of resolving small hemorrhages as ischemic stroke.13
Analysis
We included recurrent stroke, as have other studies.14 We calculated both scores as first described.6,7 As has been done in previous studies validating clinical stroke scores, we combined subarachnoid hemorrhage and cerebral hemorrhage as "intracranial hemorrhage."14,15
We calculated the scores using STATA16 according to the technique provided in the original descriptions and used the same cutoff points for diagnosing intracranial hemorrhage and ischemic stroke.7,8 We analyzed the distribution of the scores using SSC-stat.17 We compared the population with and without CT brain scans using a
2 test for homogeneity using STATA. We calculated the sensitivity and specificity of the scores and the clinicians assessment and
scores as a measure of agreement between the scores and the CT brain scan gold standard. We used Confidence Interval Analysis software to calculate CIs.18 We included the "uncertain" results, ie, the results that did not yield a definite answer of intracranial hemorrhage or ischemic stroke, in the analysis, because both sensitivity and specificity may be increased by excluding these results.14 We only used the clinicians evaluation of pathologic stroke type if they were not aware of the brain scan result at the time of their assessment.
Ethics Approval
Ethics approval for the study was granted by the University of the Witwatersrand Human Ethics Research Committee (M00/03/7).
| Results |
|---|
|
|
|---|
Figure 1 shows the distribution of scores using the SSS for stroke patients with cerebral infarcts and intracranial hemorrhages, and Figure 2 shows the equivalent information for the GHSS. We have indicated the cutoff points used in the original studies for diagnosing intracranial hemorrhage and ischemic stroke in the figures.6,7 The SSS gave an uncertain result in 44 cases (20%) and the GHSS in 65 (29%) of 222 patients with stroke. Although we analyzed the performance of the scores using the original cutoff points, we also considered whether new cutoff points might be useful in our population given the spread of the scores in patients with confirmed intracranial hemorrhage and ischemic stroke (Figures 1 and 2
). The SSS identified intracranial hemorrhage reasonably well with a score over 5 but did not accurately identify ischemic stroke at any cutoff point. The GHSS diagnosed ischemic stroke reasonably well below a score of 10 but did not identify intracranial hemorrhage well. However, using these new cutoff points the scores would only identify pathologic stroke type in very few patients; for example, 57 (81%) of intracranial hemorrhages would not be scored as such with the SSS and 135 (89%) of true infarcts with the GHSS.
|
|
We assessed whether the patients in this validation study who had CT brain scans and those excluded because they had not had a CT brain scan were as likely to have a stroke score diagnosis of intracranial hemorrhage. Sixty-nine patients had complete SSS and GHSS scores but no CT brain scan. A smaller proportion of patients without a CT scan had intracranial bleeds as assessed by the SSS and GHSS than those who did have a scan (15% versus 29% for the SSS and 4% versus 14% for the GHSS). This may be because intracranial hemorrhage has a more dramatic presentation than ischemic stroke and clinicians were therefore more likely to refer the patient for a CT brain scan. Although stroke severity was similar in those scanned (median National Institutes of Health Stroke Scale [NIHSS] 10; 95% CI 8 to 14) and not scanned (median NIHSS 11; 95% CI 8 to 13), the patients without a CT scan were older (mean age 57; SD 12; 95% CI 54 to 60) and more likely to be female (40 of 69 [58%]) than those scanned (mean age 48; SD 15; 95% CI 46 to 50) (female:104 of 222 [47%]). However, when we compared the results of the scores in patients with and without CT brain scans, we did not find a significant difference (P=0.07 for the SSS and P=0.06 for the GHSS).
The clinicians assessed stroke type in 199 consecutive patients. We excluded their assessment in 123 patients (80 did not have a CT brain scan and in 43, they knew the CT result before their assessment). Despite making every attempt to blind themselves to the CT scan result, the referring physician, the patient, or some other source inadvertently informed the clinician of the scan result before some assessments. They assessed the pathologic stroke type as ischemic stroke in 52 (68%) patients, intracranial hemorrhage in 16 (21%), and "uncertain" in 8 (11%) patients, and assessed the pathologic stroke type accurately in 56 (74%) of the 76 patients. Trainees in neurology assessed the stroke type in all but one patient in whom a specialist neurologist made the assessment.
Table 1 shows the sensitivity, specificity, positive predictive value, likelihood ratios, and
statistic for the SSS, the GHSS, and the clinicians assessment of intracranial hemorrhage; Table 2 shows the equivalent data for ischemic stroke. The likelihood ratio incorporates both the sensitivity and specificity of the test and provides a direct estimate of how much a positive test will change the odds of having a disease or, in this case, stroke type.20,21 The
statistic assesses agreement between the score/clinician diagnosis and the CT scan findings (0.0 to 0.2: slight agreement; 0.2 to 0.4: fair agreement; 0.4 to 0.6: moderate agreement; 0.6 to 0.8: good/substantial agreement; 0.8 to 1.0: almost perfect agreement).21
|
|
We recalculated the sensitivity, specificity, positive predictive values, likelihood ratios, and
statistic for the scores in patients who had a scan within 3 days of admission because scans done later may have missed some small hemorrhages. The results were almost identical in the 161 patients we scanned within 3 days and those shown in Tables 1 and 2
(data available from the authors).
| Discussion |
|---|
|
|
|---|
This study is the first prospective study of scores used to distinguish pathologic stroke type in SSA patients with stroke. There have, however, been retrospective assessments of the Siriraj Score and World Health Organization score from Nigeria and Ethiopia.4,5,22 We have compared our findings with those found in Nigeria4,22 and Ethiopia5 and in other regions of the world in Tables 3 and 4
.5,7,14,15,22–31
|
|
We found a higher sensitivity, specificity, and positive predictive value for the SSS in detecting intracranial hemorrhage than was found in Nigeria and Ethiopia. This was despite the high proportions of intracranial hemorrhage in these two studies (47% to 59%)4,5 compared with our study (70 of 222 [32%]). This probably reflects patient selection or case mix and the limitations of a retrospective study design rather than a true difference in the performance of the SSS in black populations in SSA. Our findings are well within the range of findings in populations with a varying prevalence of cerebral hemorrhage (Table 3). The most appropriate study for comparison with our study is from Malaysia.28 It too was a prospective study that included "uncertain" results in the analysis. We found the SSS had a slightly higher sensitivity for detecting intracranial hemorrhage than in the Malaysian study, but the specificity and positive predictive values were similar in the two studies. It has been suggested that the SSS has failed to impress in high-income white populations because it was developed in Thailand in a population with a high prevalence of cerebral hemorrhage.14 However, the score did not do much better in our population who also have a high prevalence of cerebral hemorrhage.
Although the GHSS has shown a high specificity in the diagnosis of intracranial hemorrhage and a fair positive predictive value, the sensitivity has been more variable (Table 3). Together with studies from Italy23 and New Zealand,14 we found the lowest sensitivity (0.31 to 0.38) for detecting intracranial hemorrhage using the GHSS. It has been suggested that the GHSS was developed with relatively young patients with stroke (under the age of 76 years) and therefore in a population with a higher prevalence of intracranial hemorrhage,14 yet the score failed to perform well in our very young stroke population.
In our study, the SSS was more accurate at diagnosing ischemic stroke than in Nigeria and Ethiopia (Table 4). Our findings for sensitivity, specificity, and positive predictive value are remarkably consistent with those found in prospective studies from Pakistan, India, and Hong Kong.25–27
The accuracy of the GHSS in diagnosing ischemic stroke in our study was poor compared with previous studies (Table 4). The GHSS score places far more emphasis on measures of atherosclerosis than the SSS does. Perhaps it performs better in populations in which atherosclerosis is an important cause of ischemic stroke rather than in populations such as ours in which atherosclerotic disease is currently uncommon.8,9
Previous studies have shown that the bedside assessment of pathologic stroke type by clinicians is not accurate.6,32,33 In Stockholm,33 clinicians made an accurate bedside diagnosis in 69% of 206 stroke cases, a similar proportion to our clinicians who accurately diagnosed 56 (74%) of 76 patients with stroke. Clinicians at Guys Hospital were less accurate than the GHSS, but still diagnosed stroke type accurately in 84% of patients. In a recent study from Ethiopia, stroke type was accurately diagnosed in 37 (76%) of 49 patients with stroke, far more often than by the SSS (18 of 41 patients [44%]).5
Thus, our clinicians performed about as well as other clinicians around the world, although they assessed rather few patients. In our study, we used the stroke type allocated by trainees in neurology in all but one case in an attempt to generalize our findings to other clinical settings. Despite this, clinicians without neurology training would possibly not perform as well. This would limit the generalizability of our findings to hospitals and clinics in remote areas of South Africa.
Although we did not find a significant difference between the score results in those patients who had and those that did not have a CT scan and stroke severity was similar in the two groups, patients who did not have a scan were older and more likely to be female. Therefore, our low scan rate may have influenced our findings. Ideally, a validation study should have an almost 100% scan rate, preferably in a community-based study, to assess the stroke score in patients with stroke with a wide range of severity.
We agree with others who have suggested that stroke scores are not sufficiently accurate in either epidemiologic studies or clinical management.14,24,28,30 Only CT or magnetic resonance scanning will do. Unfortunately, stroke incidence is likely to increase across SSA, a region with very limited access to CT or magnetic resonance scanners. For great parts of SSA, thrombolytics and anticoagulation for the few patients that need them are not feasible therapeutic options.34 However, a score that excluded significant intracranial hemorrhage would encourage doctors and nurses in remote areas to initiate aspirin therapy early. We need further analysis and refinement of these scores or development of a new score by multivariate analysis of variables, which are less dependent on extracranial atherosclerosis, using data from prospective stroke cases in SSA. Any new score will require validation in a new population. However, it is unlikely that any score will replace brain imaging and we would encourage investment in CT scanners. Where this is impossible and the treatment of stroke therefore limited, health services should emphasize strategies to reduce the risk of stroke in their population.
| Acknowledgments |
|---|
Sources of Funding
This study was funded by the Southern African Hypertension Society and the University of the Witwatersrand (H E Griffin Charitable Trust and Captain "Sailor Malan" award).
Disclosures
None.
Received May 24, 2006; revision received June 24, 2006; accepted August 29, 2006.
| References |
|---|
|
|
|---|
2. Yusuf S, Reddy S, Ounpuu S, Anand S. Global burden of cardiovascular diseases: part I: general considerations, the epidemiologic transition, risk factors, and impact of urbanization. Circulation. 2001; 104: 2746–2753.
3. El Khamlichi AMD. African neurosurgery: current situation, priorities, and needs. Neurosurgery. 2001; 48: 1344–1347.[CrossRef][Medline] [Order article via Infotrieve]
4. Ogun SA, Oluwole O, Fatade B, Ogunseyinde AO, Ojini FI, Odusote KA. Comparison of Siriraj Stroke Score and the WHO criteria in the clinical classification of stroke subtypes. Afr J Med Med Sci. 2002; 31: 13–16.[Medline] [Order article via Infotrieve]
5. Zenebe G, Asmera J, Alemayehu M. How accurate is Siriraj Stroke Score among Ethiopians? A brief communication. Ethiop Med J. 2005; 43: 35–38.[Medline] [Order article via Infotrieve]
6. Allen C. Clinical diagnosis of acute stroke syndrome. QJM. 1983; 52: 515–523.
7. Poungvarin N, Viriyavejakul A, Komontri C. Siriraj Stroke Score and validation study to distinguish supratentorial intracerebral hemorrhage from infarction. BMJ. 1991; 302: 1565–1567.
8. van der Horst RL. The frequency of different types of heart disease in South African blacks at a cardiac referral center. Int J Cardiol. 1984; 6: 240–243.[CrossRef][Medline] [Order article via Infotrieve]
9. Mollentze WF. Epidemiological aspects in South Africa. South African Journal of Diabetes and Vascular Disease. 2004; 1: 55–56.
10. Rosman KD. The epidemiology of stroke in an urban black population. Stroke. 1986; 17: 667–669.
11. Gauteng Department of Health, Johannesburg Hospital, 2005 [Cited July 24, 2005]. Available at: http://www.johannesburghospital.org.za/. Accessed June 24, 2006.
12. Hatano S. Experience from a multicentre stroke register: a preliminary report. Bull World Health Organ. 1976; 54: 541–553.[Medline] [Order article via Infotrieve]
13. Dennis MS, Bamford JM, Molyneux AJ, Warlow CP. Rapid resolution of signs of primary intracerebral haemorrhage in computed tomograms of the brain. BMJ (Clin Res Ed). 1987; 295: 379–381.
14. Hawkins GC, Bonita R, Broad JB, Anderson NE. Inadequacy of clinical scoring systems to differentiate stroke subtypes in population-based studies. Stroke. 1995; 26: 1338–1342.
15. Sandercock PA, Allen CM, Corston RN, Harrison MJ, Warlow CP. Clinical diagnosis of intracranial haemorrhage using Guys Hospital score. BMJ (Clin Res Ed). 1985; 291: 1675–1677.
16. Stata Statistical Software release 7.0. College Station TX: Stata Corp, 2001.
17. SSC-Stat. UK: The University of Reading, 2004.
18. Bryant TN. Computer software for calculating confidence intervals. In: Altman DG, Machin D, Bryant TN, Gardner MJ, eds. Statistics With Confidence. Bristol, UK: BMJ Publishing Group; 2000: 208–213.
19. City of Johannesburg Local Government. City of Johannesburg: city development plan 2001–2002, 2001 [Cited April 15, 2006]. Available at: http://www.johannesburgnews.co.za/budget_2001/develop_plan/dev_overview.html. Accessed April 15, 2006.
20. Altman DG, Machin D, Bryant TN, Gardner MJ. Statistics With Confidence, 2nd ed. Bristol: BMJ Publishing Group, 2000.
21. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology: A Basic Science for Clinical Medicine, 2nd ed. Boston: Little, Brown and Co, 1991.
22. Ogun SA, Oluwole SOA, Oluremi A, Fatade AO, Ojini F, Odusote KA. Accuracy of the Siriraj Stroke Score in differentiating cerebral haemorrhage and infarction in African Nigerians. African Journal of Neurological Sciences. 2001; 20: 21–26.
23. Celani MG, Righetti E, Migliacci R, Zampolini M, Antoniutti L, Grandi FC, Ricci S. Comparability and validity of two clinical scores in the early differential diagnosis of acute stroke. BMJ. 1994; 308: 1674–1676.
24. Badam P, Solao V, Pai M, Kalantri SP. Poor accuracy of the Siriraj and Guys Hospital stroke scores in distinguishing haemorrhagic from ischaemic stroke in a rural, tertiary care hospital. Natl Med J India. 2003; 16: 8–12.[Medline] [Order article via Infotrieve]
25. Kochar DK, Joshi A, Agarwal N, Aseri S, Sharma BV, Agarwal TD. Poor diagnostic accuracy and applicability of Siriraj Stroke Score, Allen score and their combination in differentiating acute haemorrhagic and thrombotic stroke. J Assoc Physicians India. 2000; 48: 584–588.[Medline] [Order article via Infotrieve]
26. Shah FU, Salih M, Saeed MA, Tariq M. Validaty of Siriraj Stroke Scoring. J Coll Physicians Surg Pak. 2003; 13: 391–393.[Medline] [Order article via Infotrieve]
27. Hui AC, Wu B, Tang AS, Kay R. Lack of clinical utility of the Siriraj Stroke Score. Intern Med J. 2002; 32: 311–314.[CrossRef][Medline] [Order article via Infotrieve]
28. Kan CH, Lee SK, Low CS, Velusamy SS, Cheong I. A validation study of the Siriraj Stroke Score. Int J Clin Pract. 2000; 54: 645–646.[Medline] [Order article via Infotrieve]
29. Akpunonu BE, Mutgi AB, Lee L, Khuder S, Federman DJ, Roberts C. Can a clinical score aid in early diagnosis and treatment of various stroke syndromes? Am J Med Sci. 1998; 315: 194–198.[CrossRef][Medline] [Order article via Infotrieve]
30. Weir CJ, Murray GD, Adams FG, Muir KW, Grosset DG, Lees KR. Poor accuracy of stroke scoring systems for differential clinical diagnosis of intracranial haemorrhage and infarction. Lancet. 1994; 344: 999–1002.[CrossRef][Medline] [Order article via Infotrieve]
31. Huang JA, Wang PY, Chang MC, Chia LG, Yang DY, Wu TC. Allen score in clinical diagnosis of intracranial hemorrhage. Zhonghua Yi Xue Za Zhi (Taipei). 1994; 54: 407–411.[Medline] [Order article via Infotrieve]
32. Allen CM. Clinical diagnosis of stroke. Lancet. 1984; 1: 1357–1358.[Medline] [Order article via Infotrieve]
33. von Arbin M, Britton M, de Faire U, Helmers C, Miah K, Murray V. Accuracy of bedside diagnosis in stroke. Stroke. 1981; 12: 288–293.
34. Heller RF, Langhorne P, James E. Improving stroke outcome: the benefits of increasing availability of technology. Bull World Health Organ. 2000; 78: 1337–1343.[Medline] [Order article via Infotrieve]
This article has been cited by other articles:
![]() |
M. D. Connor, G. Modi, and C. P. Warlow Differences in the Nature of Stroke in a Multiethnic Urban South African Population: The Johannesburg Hospital Stroke Register Stroke, February 1, 2009; 40(2): 355 - 362. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2007 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |