(Stroke. 2007;38:62.)
© 2007 American Heart Association, Inc.
Original Contributions |
From the Division of Neurology, Department of Neurosciences (M.D.C.), University of the Witwatersrand, Johannesburg, South Africa; the Division of Neurology, Department of Neurosciences (G.M.), University of the Witwatersrand, Johannesburg, South Africa; and the Department of Clinical Neurosciences (C.P.W.), Western General Hospital, Edinburgh, United Kingdom.
Correspondence to Dr Myles D. Connor, Division of Neurology, Department of Neurosciences, University of the Witwatersrand, 7 York Road, Parktown, 2193, South Africa. E-mail mconnor{at}mighty.co.za; connormd@medicine.wits.ac.za
| Abstract |
|---|
|
|
|---|
Methods— We assessed the stroke scores and the clinicians blind assessment of pathologic stroke type in consecutive black patients with stroke included in the Johannesburg Hospital Stroke Register over 23 months. We calculated the accuracy of the scores and clinicians compared with computed tomography brain scan (sensitivity, specificity, positive predictive value, likelihood ratio,
statistic).
Results— Two hundred twenty-two patients were scanned and assessed within 15 days. Sixty-two (28%) had cerebral hemorrhage and nine (4%) subarachnoid hemorrhage. Neither the Siriraj (sub-Saharan Africa) nor Guys Hospital score was accurate or offered much advantage over clinician assessment (sensitivity 0.60 and 0.34, specificity 0.88 and 0.95 for intracranial hemorrhage in the Siriraj Stroke Score and Guys Hospital Stroke Score, respectively; sensitivity 0.70 and 0.71, specificity 0.84 and 0.74, respectively, for ischemic stroke). Although the scores were more accurate when we used new cutoff points, they then failed to diagnose over 80% of stroke types.
Conclusions— The Siriraj Stroke Score and Guys Hospital Stroke Score are not sufficiently accurate for use in either epidemiologic studies or to guide clinical management in sub-Saharan Africa at present.
Key Words: Africa south of the Sahara South Africa cerebral hemorrhage cerebral infarction cerebrovascular accident diagnosis subarachnoid hemorrhage
| Introduction |
|---|
|
|
|---|
We cannot directly extrapolate the results from validation studies undertaken in populations outside SSA. First, because many of the scores used to distinguish pathologic stroke type use "atheroma markers" such as angina, intermittent claudication, diabetes, and myocardial infarction,6,7 our population does not have a high prevalence of extracranial atherosclerotic disease either in the general population or in patients with stroke.8,9 Second, our prevalence of intracerebral hemorrhage may be very different.10
Although there are several scores available, we chose to use the SSS and Guys Hospital Stroke Score (GHSS) because these required the least ancillary testing and investigation and appeared to be the simplest to use.6,7 The SSS only requires a history and examination, whereas the GHSS requires a chest x-ray and electrocardiogram in addition. We felt these scores were the most likely to succeed in settings where resources are scarce.
Our aim was to assess the accuracy of the GHSS and SSS in distinguishing between intracranial hemorrhage (cerebral hemorrhage and subarachnoid hemorrhage) and ischemic stroke in black patients with stroke in South Africa. Later in the study, we assessed whether the scores were more accurate than clinicians in distinguishing pathologic stroke type.
| Methods |
|---|
|
|
|---|
Case Ascertainment and Assessment
The JHSR was advertised around the hospital and at academic meetings, and every weekday, a dedicated stroke register assistant asked doctors and nurses in the emergency department, admission wards, and medical wards if they had admitted any patients with stroke or transient ischemic attack in the previous 24 hours or if any patients had had a stroke while in the hospital. Every weekday, she also reviewed ward admission books and the files of patients who had died during the previous 24 hours. Over weekends, a JHSR clinician searched the emergency wards and admission records for acute strokes. Physicians from the Division of Neurology then assessed all patients in detail, and one of the authors (M.C.) reviewed the vast majority of cases personally and all JHSR questionnaires. We defined stroke according to the World Health Organization criteria as "rapidly developing signs of focal (or global) disturbance of cerebral function, leading to death or lasting longer than 24 hours, with no apparent cause other than vascular."12
As part of the assessment of a patient with stroke, the register clinician documented the information required for the scores but did not calculate them. During the second period of the register (2001 to 2002), clinicians documented what they thought the likely pathologic stroke type was and whether they had made this assessment without knowledge of the brain imaging result. We did not provide the clinicians with diagnostic criteria or aids for assessing stroke type, because we wanted to find out if the scores would add to the bedside assessment of stroke type made by a South African clinician who did not have access to brain imaging. Thus, although the author (M.C.) confirmed the stroke diagnosis and decided on the inclusion of patients onto the JHSR, we used the initial stroke register clinicians assessment of stroke type in our analysis. Stroke cases were included if their self-defined ethnicity was black, if they had sufficient data available to calculate the stroke scores, and had undergone CT within 15 days of onset of their stroke to minimize the misdiagnosis of resolving small hemorrhages as ischemic stroke.13
Analysis
We included recurrent stroke, as have other studies.14 We calculated both scores as first described.6,7 As has been done in previous studies validating clinical stroke scores, we combined subarachnoid hemorrhage and cerebral hemorrhage as "intracranial hemorrhage."14,15
We calculated the scores using STATA16 according to the technique provided in the original descriptions and used the same cutoff points for diagnosing intracranial hemorrhage and ischemic stroke.7,8 We analyzed the distribution of the scores using SSC-stat.17 We compared the population with and without CT brain scans using a
2 test for homogeneity using STATA. We calculated the sensitivity and specificity of the scores and the clinicians assessment and
scores as a measure of agreement between the scores and the CT brain scan gold standard. We used Confidence Interval Analysis software to calculate CIs.18 We included the "uncertain" results, ie, the results that did not yield a definite answer of intracranial hemorrhage or ischemic stroke, in the analysis, because both sensitivity and specificity may be increased by excluding these results.14 We only used the clinicians evaluation of pathologic stroke type if they were not aware of the brain scan result at the time of their assessment.
Ethics Approval
Ethics approval for the study was granted by the University of the Witwatersrand Human Ethics Research Committee (M00/03/7).
| Results |
|---|
|
|
|---|
Figure 1 shows the distribution of scores using the SSS for stroke patients with cerebral infarcts and intracranial hemorrhages, and Figure 2 shows the equivalent information for the GHSS. We have indicated the cutoff points used in the original studies for diagnosing intracranial hemorrhage and ischemic stroke in the figures.6,7 The SSS gave an uncertain result in 44 cases (20%) and the GHSS in 65 (29%) of 222 patients with stroke. Although we analyzed the performance of the scores using the original cutoff points, we also considered whether new cutoff points might be useful in our population given the spread of the scores in patients with confirmed intracranial hemorrhage and ischemic stroke (Figures 1 and 2
). The SSS identified intracranial hemorrhage reasonably well with a score over 5 but did not accurately identify ischemic stroke at any cutoff point. The GHSS diagnosed ischemic stroke reasonably well below a score of 10 but did not identify intracranial hemorrhage well. However, using these new cutoff points the scores would only identify pathologic stroke type in very few patients; for example, 57 (81%) of intracranial hemorrhages would not be scored as such with the SSS and 135 (89%) of true infarcts with the GHSS.
|
|
We assessed whether the patients in this validation study who had CT brain scans and those excluded because they had not had a CT brain scan were as likely to have a stroke score diagnosis of intracranial hemorrhage. Sixty-nine patients had complete SSS and GHSS scores but no CT brain scan. A smaller proportion of patients without a CT scan had intracranial bleeds as assessed by the SSS and GHSS than those who did have a scan (15% versus 29% for the SSS and 4% versus 14% for the GHSS). This may be because intracranial hemorrhage has a more dramatic presentation than ischemic stroke and clinicians were therefore more likely to refer the patient for a CT brain scan. Although stroke severity was similar in those scanned (median National Institutes of Health Stroke Scale [NIHSS] 10; 95% CI 8 to 14) and not scanned (median NIHSS 11; 95% CI 8 to 13), the patients without a CT scan were older (mean age 57; SD 12; 95% CI 54 to 60) and more likely to be female (40 of 69 [58%]) than those scanned (mean age 48; SD 15; 95% CI 46 to 50) (female:104 of 222 [47%]). However, when we compared the results of the scores in patients with and without CT brain scans, we did not find a significant difference (P=0.07 for the SSS and P=0.06 for the GHSS).
The clinicians assessed stroke type in 199 consecutive patients. We excluded their assessment in 123 patients (80 did not have a CT brain scan and in 43, they knew the CT result before their assessment). Despite making every attempt to blind themselves to the CT scan result, the referring physician, the patient, or some other source inadvertently informed the clinician of the scan result before some assessments. They assessed the pathologic stroke type as ischemic stroke in 52 (68%) patients, intracranial hemorrhage in 16 (21%), and "uncertain" in 8 (11%) patients, and assessed the pathologic stroke type accurately in 56 (74%) of the 76 patients. Trainees in neurology assessed the stroke type in all but one patient in whom a specialist neurologist made the assessment.
Table 1 shows the sensitivity, specificity, positive predictive value, likelihood ratios, and
statistic for the SSS, the GHSS, and the clinicians assessment of intracranial hemorrhage; Table 2 shows the equivalent data for ischemic stroke. The likelihood ratio incorporates both the sensitivity and specificity of the test and provides a direct estimate of how much a positive test will change the odds of having a disease or, in this case, stroke type.20,21 The
statistic assesses agreement between the score/clinician diagnosis and the CT scan findings (0.0 to 0.2: slight agreement; 0.2 to 0.4: fair agreement; 0.4 to 0.6: moderate agreement; 0.6 to 0.8: good/substantial agreement; 0.8 to 1.0: almost perfect agreement).21
|
|
We recalculated the sensitivity, specificity, positive predictive values, likelihood ratios, and
statistic for the scores in patients who had a scan within 3 days of admission because scans done later may have missed some small hemorrhages. The results were almost identical in the 161 patients we scanned within 3 days and those shown in Tables 1 and 2
(data available from the authors).
| Discussion |
|---|
|
|
|---|
This study is the first prospective study of scores used to distinguish pathologic stroke type in SSA patients with stroke. There have, however, been retrospective assessments of the Siriraj Score and World Health Organization score from Nigeria and Ethiopia.4,5,22 We have compared our findings with those found in Nigeria4,22 and Ethiopia5 and in other regions of the world in Tables 3 and 4
.5,7,14,15,22–31
|
|
We found a higher sensitivity, specificity, and positive predictive value for the SSS in detecting intracranial hemorrhage than was found in Nigeria and Ethiopia. This was despite the high proportions of intracranial hemorrhage in these two studies (47% to 59%)4,5 compared with our study (70 of 222 [32%]). This probably reflects patient selection or case mix and the limitations of a retrospective study design rather than a true difference in the performance of the SSS in black populations in SSA. Our findings are well within the range of findings in populations with a varying prevalence of cerebral hemorrhage (Table 3). The most appropriate study for comparison with our study is from Malaysia.28 It too was a prospective study that included "uncertain" results in the analysis. We found the SSS had a slightly higher sensitivity for detecting intracranial hemorrhage than in the Malaysian study, but the specificity and positive predictive values were similar in the two studies. It has been suggested that the SSS has failed to impress in high-income white populations because it was developed in Thailand in a population with a high prevalence of cerebral hemorrhage.14 However, the score did not do much better in our population who also have a high prevalence of cerebral hemorrhage.
Although the GHSS has shown a high specificity in the diagnosis of intracranial hemorrhage and a fair positive predictive value, the sensitivity has been more variable (Table 3). Together with studies from Italy23 and New Zealand,14 we found the lowest sensitivity (0.31 to 0.38) for detecting intracranial hemorrhage using the GHSS. It has been suggested that the GHSS was developed with relatively young patients with stroke (under the age of 76 years) and therefore in a population with a higher prevalence of intracranial hemorrhage,14 yet the score failed to perform well in our very young stroke population.
In our study, the SSS was more accurate at diagnosing ischemic stroke than in Nigeria and Ethiopia (Table 4). Our findings for sensitivity, specificity, and positive predictive value are remarkably consistent with those found in prospective studies from Pakistan, India, and Hong Kong.25–27
The accuracy of the GHSS in diagnosing ischemic stroke in our study was poor compared with previous studies (Table 4). The GHSS score places far more emphasis on measures of atherosclerosis than the SSS does. Perhaps it performs better in populations in which atherosclerosis is an important cause of ischemic stroke rather than in populations such as ours in which atherosclerotic disease is currently uncommon.8,9
Previous studies have shown that the bedside assessment of pathologic stroke type by clinicians is not accurate.6,32,33 In Stockholm,33 clinicians made an accurate bedside diagnosis in 69% of 206 stroke cases, a similar proportion to our clinicians who accurately diagnosed 56 (74%) of 76 patients with stroke. Clinicians at Guys Hospital were less accurate than the GHSS, but still diagnosed stroke type accurately in 84% of patients. In a recent study from Ethiopia, stroke type was accurately diagnosed in 37 (76%) of 49 patients with stroke, far more often than by the SSS (18 of 41 patients [44%]).5
Thus, our clinicians performed about as well as other clinicians around the world, although they assessed rather few patients. In our study, we used the stroke type allocated by trainees in neurology in all but one case in an attempt to generalize our findings to other clinical settings. Despite this, clinicians without neurology training would possibly not perform as well. This would limit the generalizability of our findings to hospitals and clinics in remote areas of South Africa.
Although we did not find a significant difference between the score results in those patients who had and those that did not have a CT scan and stroke severity was similar in the two groups, patients who did not have a scan were older and more likely to be female. Therefore, our low scan rate may have influenced our findings. Ideally, a validation study should have an almost 100% scan rate, preferably in a community-based study, to assess the stroke score in patients with stroke with a wide range of severity.
We agree with others who have suggested that stroke scores are not sufficiently accurate in either epidemiologic studies or clinical management.14,24,28,30 Only CT or magnetic resonance scanning will do. Unfortunately, stroke incidence is likely to increase across SSA, a region with very limited access to CT or magnetic resonance scanners. For great parts of SSA, thrombolytics and anticoagulation for the few patients that need them are not feasible therapeutic options.34 However, a score that excluded significant intracranial hemorrhage would encourage doctors and nurses in remote areas to initiate aspirin therapy early. We need further analysis and refinement of these scores or development of a new score by multivariate analysis of variables, which are less dependent on extracranial atherosclerosis, using data from prospective stroke cases in SSA. Any new score will require validation in a new population. However, it is unlikely that any score will replace brain imaging and we would encourage investment in CT scanners. Where this is impossible and the treatment of stroke therefore limited, health services should emphasize strategies to reduce the risk of stroke in their population.
| Acknowledgments |
|---|
Sources of Funding
This study was funded by the Southern African Hypertension Society and the University of the Witwatersrand (H E Griffin Charitable Trust and Captain "Sailor Malan" award).
Disclosures
None.
Received May 24, 2006; revision received June 24, 2006; accepted August 29, 2006.
| References |
|---|
|
|
|---|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2007 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |