CT and Diffusion-Weighted MR Imaging in Randomized Order
Diffusion-Weighted Imaging Results in Higher Accuracy and Lower Interrater Variability in the Diagnosis of Hyperacute Ischemic Stroke
Background and Purpose— Diffusion-weighted MRI (DWI) has become a commonly used imaging modality in stroke centers. The value of this method as a routine procedure is still being discussed. In previous studies, CT was always performed before DWI. Therefore, infarct progression could be a reason for the better result in DWI.
Methods— All hyperacute (<6 hours) stroke patients admitted to our emergency department with a National Institutes of Health Stroke Scale (NIHSS) score >3 were prospectively randomized for the order in which CT and MRI were performed. Five stroke experts and 4 residents blinded to clinical data judged stroke signs and lesion size on the images. To determine the interrater variability, we calculated κ values for both rating groups.
Results— A total of 50 patients with ischemic stroke and 4 patients with transient symptoms of acute stroke (median NIHSS score, 11; range, 3 to 27) were analyzed. Of the 50 patients, 55% were examined with DWI first. The mean delay from symptom onset until CT was 180 minutes; that from symptom onset until DWI was 189 minutes. The mean delay between DWI and CT was 30 minutes. The sensitivity of infarct detection by the experts was significantly better when based on DWI (CT/DWI, 61/91%). Accuracy was 91% when based on DWI (CT, 61%). Interrater variability of lesion detection was also significantly better for DWI (CT/DWI, κ=0.51/0.84). The assessment of lesion extent was less homogeneous on CT (CT/DWI, κ=0.38/0.62). The differences between the 2 modalities were stronger in the residents’ ratings (CT/DWI: sensitivity, 46/81%; κ=0.38/0.76).
Conclusions— CT and DWI performed with the same delay after onset of ischemic stroke resulted in significant differences in diagnostic accuracy. DWI gives good interrater homogeneity and has a substantially better sensitivity and accuracy than CT even if the raters have limited experience.
- computed tomography
- magnetic resonance imaging, diffusion-weighted
- sensitivity and specificity
- stroke, ischemic
Cerebral ischemia is the leading cause of the acute onset of severe neurological symptoms in stroke patients. Fibrinolysis has become an accepted treatment of cerebral ischemia during the first 3 hours after symptom onset and up to 6 hours in selected cases.1–4⇓⇓⇓ The goal of fibrinolytic therapy is to recanalize an occluded basal brain artery and to prevent infarction in a hypoperfused tissue at risk.5 Before initiation of this treatment, however, other causes of acute symptoms such as seizure, encephalitis, and cerebral hemorrhage must be excluded. CT is the imaging modality most commonly used to exclude cerebral hemorrhage. It was also used in the European Cooperative Acute Stroke Study (ECASS) to exclude patients with extensive infarctions from recanalization therapy to reduce the incidence of intracranial hemorrhage (ICH).6 The main advantage of CT is its widespread availability in many hospitals 24 hours a day. The sensitivity of CT during the first 6 hours of cerebral ischemia was 64% in the ECASS reading panel with an accuracy of 67%. The local investigators of the ECASS trial reached only a 40% sensitivity (accuracy, 45%).7
According to the present literature, stroke MRI seems to be superior to CT as an imaging modality.8–11⇓⇓⇓ All previous studies, however, lack 1 or more of the following: sufficient patient numbers, blinding of scan interpretation, relevant time window <6 hours, preselection of patients not based on the therapeutic target group, and most importantly a negative bias against CT, which always was performed before MRI with a mean time delay averaging 2 hours.12–14⇓⇓
In this study, we prospectively compared CT and diffusion-weighted MR imaging (DWI) in hyperacute stroke patients potentially eligible for thrombolytic therapy according to time window and baseline severity of stroke [National Institutes of Health Stroke Scale (NIHSS)]. To avoid any bias against CT resulting from delay after symptom onset, we prospectively randomized our patients to both imaging modalities (either stroke MRI or CT first followed by the other). The scans were read by neurologists and neuroradiologists with extensive experience in stroke imaging. We also investigated the accuracy of stroke imaging in the real-life clinical setting. Therefore, residents from the departments of neurology and neuroradiology read the images separately. Accuracy and sensitivity were calculated for each rater. Finally, we analyzed the interrater variability for both groups in both modalities.
Materials and Methods
In a single-center, prospective trial conducted from February 2000 through March 2001, we screened 1320 patients with clinical signs of acute stroke, including transient ischemic attack (TIA). Inclusion criteria were anterior territory ischemic stroke within 6 hours of symptom onset with an NIHSS score of >3 and time interval between imaging modalities of <90 minutes. Baseline stroke severity was assessed with the NIHSS score.1 After clinical examination, patients were prospectively randomized for the sequence in which the imaging modalities were performed. Sealed randomization envelopes were kept in the neurological emergency ward. CT and DWI were performed by the junior radiology resident and/or the technician on call. Of the patients, 63% arrived outside the 6-hour time window (488 patients remaining), 48% had a TIA or an NIHSS score of <4, and 18% had an ICH; some of these patients were randomized to imaging (n=20) but were not included into this study (166 patients remaining). Of these 166 patients, 29 had signs of posterior territory stroke; 13 did not give consent for randomization to imaging sequence; 10 had a time interval of >90 minutes between the 2 modalities; 27 were screened, imaged, and included in acute pharmaceutical trials; and 13 had MRI contraindications such as pacemakers, loss of vigilance, or recurrent vomiting. Twenty seemingly eligible patients were not included. For 11 of these patients, there was no adequate information available in our stroke data bank to explain why. In 9 theoretically eligible patients, MRI was not available because of maintenance procedures or nonavailability of the technician and/or physician (eg, because of a simultaneously performed neurointerventional procedure).
In total, images from 54 patients were presented to the readers: 50 with acute brain infarctions and 4 with symptoms of TIA. No images from studies before February 20005,11,14–16⇓⇓⇓⇓ were included to avoid any methodological pitfall according to changes in clinical setting (eg, dealing with stroke patients in the scanner increasingly becoming a routine procedure).
Age and sex were typically distributed for a stroke cohort (Table 1). Patients presented with a median NIHSS score of 11 (range, 3 to 27). In all patients, the time of symptom onset could be precisely defined. The correct diagnosis was established by the clinical course and follow-up CT or MRI examination (6 patients refused another MRI or CT examination). Follow-up examinations served as reference for validation analysis.
The study protocol was approved by the institutional ethics committee, which recommended the examination of all eligible patients even if they were aphasic. Only patients who refused the initial workup and patients with general contraindications for MRI were excluded. The scans were read by neurologists and neuroradiologists with considerable experience in stroke imaging. Furthermore, we investigated the accuracy of stroke imaging in a real-life clinical setting. Therefore, residents from the departments of neurology and neuroradiology read the images separately. Early stroke signs on CT, hyperintensities on DWI, and lesion extent were judged. Accuracy and sensitivity were calculated for each rater. Stroke types were categorized as negative, lacunar, territorial, or hemodynamic. Finally, we analyzed the interrater variability for both groups in both modalities.
One neurologist and 4 neuroradiologists, all experienced in stroke imaging (J.B.F., S.H., M.H., W.H., and M.K. were the “experts”), and 4 second- or third-year residents (J.B., E.J., J.O., P.S. were the “novices”) read all images. All residents took part in a daily clinical conference on neuroradiology. The reviewers were unaware of the symptoms and signs that the patients had presented with, but they knew that the cohort was an ischemic stroke population. In the first session, all reviewers individually analyzed either the CT or DWI of each patient. In the second session, the corresponding images were read in a different order. All readers were asked to identify and lateralize early stroke signs on CT. Presence or absence of loss of the insular ribbon, hypodensity of the lentiform nucleus, focal cortical swelling, and hyperdensity of the middle cerebral artery (MCA) were assessed. The extent of infarction was estimated in thirds of the MCA territory, as lacunar, or as negative. On DWI, the impaired side and the anatomic localization of areas of abnormal, high signal were judged. Again, the readers quantified the extent of the lesion in relation to the MCA territory. The 4 novices had read the introduction to early stroke signs on CT by von Kummer et al17 and were instructed as to how to deal with the DWI; images from a former study served as examples for typical findings and artifacts.11,14⇓
None of the CT images were contrast enhanced, and all were obtained with a slice thickness of 4 mm for the posterior fossa and 8 mm for the cerebrum [PQ 2000, Marconi (formerly Picker), or Volume Zoom, Siemens]. MRI was performed with a 1.5-T scanner with improved gradient hardware (Edge, Marconi). Patient head movement was restricted, and ear plugs were used to diminish noise discomfort. For DWI, we used an isotropic, diffusion-weighted, spin-echo, echo-planar imaging sequence (repetition time, 2000 ms; echo time, 106 ms; number of averages, 1; slice thickness, 6 mm; field of view, 17×31 cm2; number of slices 19; matrix size, 64×256; b=1000 s/mm2). A complete MRI protocol, described elsewhere, was used to examine all patients.11
The description of interobserver agreement includes unweighted κ values for blinded interpretation of CT and DWI studies according to Fleiss18 and Landis and Koch.19 Data from the 2 groups of raters were analyzed separately. In agreement with others, we considered κ values <0.2 as slight, 0.21 to 0.4 as fair, 0.41 to 0.6 as moderate, 0.61 to 0.8 as substantial agreement beyond chance, and 0.81 to 1 as almost perfect.19–22⇓⇓⇓ Sensitivity, specificity, positive and negative predictive values, and accuracy were calculated.
In 54.5% of the study population, DWI was performed first. The mean delay between symptom onset and CT was 180 minutes (SD, 85.3 minutes; range, 62 to 360 minutes). DWI was performed 189 minutes after symptom onset (SD, 83.0 minutes; range, 69 to 356 minutes). Mean delay between the 2 examinations was 29 minutes (SD, 17.8 minutes; range, 8 to 89 minutes). Demographic characteristics were similar in the 2 patient groups (Table 1). All images were of adequate diagnostic quality with typical variation according to patient compliance. All infarctions were in the MCA territory; 66% were left sided. The other 4 patients suffered from migraine (n=1), postictal hemiparesis with fast recovery (n=1), or TIA (n=2).
All the experts reliably identified infarction on 20 CT scans. Early stroke signs were identified by at least 4 expert readers on early CT in 7 additional patients. The 4 CT scans of the patients with transient symptoms, migraine, or postictal paresis were judged normal by all experts. In 5 additional cases, the final decision (at least 4 readers) was negative. The mean CT sensitivity of the 5 experts was 61%, with a mean specificity of 65% (Table 2). The accuracy of lesion detection reached 61%. On the basis of early stroke signs, the interrater variability for identifying infarctions was κ=0.51. Detecting the 4 early signs of infarction each resulted in an interrater variability of κ=0.42 to 0.52; the estimation of lesion size, κ=0.38. Differentiation between the various stroke types gave κ=0.42 (Table 3).
On DWI, all of the experts identified infarcted sites in 40 cases. Infarction was identified by at least 4 observers on 7 additional DWI studies. One expert detected a hyperintense signal in a patient with normal findings on follow-up scan. The median DWI sensitivity of the experts was 91%, with a median specificity of 95%. An accuracy of 91% was found for lesion detection. The interrater variability was κ=0.84 for detecting infarction in general and κ=0.86 for identifying hyperintensities in the insular cortex. Rating of hyperintensities of the internal capsule or basal ganglia yielded κ=0.62 and 0.60, respectively (Table 3). Agreement in stroke type identification gave κ=0.52. The positive predictive value was excellent for both modalities (CT/DWI, 96%/100%). For CT, the negative predictive value was 12% (range, 9% to 17%); for DWI, 47% (range, 38% to 57%).
The agreement of stroke detection on DWI increased with the delay from symptom onset. For images made up to 120 minutes after ictus, κ=0.76. It increased for the period between 120 and 240 minutes to κ=0.79 and reached κ=1 for the fifth and sixth hours after symptom onset. For CT, the values were κ=0.52 (0 to 2 hours) and κ=0.58 (3 to 4 hours) but only κ=0.36 during the fifth and sixth hours after symptom onset. The major estimation of lesion size gave a clear result for 42 DWI and 26 CT. Findings were negative on 8 CT and 5 DWI studies. Lacunar lesions were not identified on any CT scans but were identified on 4 DWI images. Lesions smaller than one third of the MCA territory were detected on 15 CT scans and 25 DWI images.
Four infarcts were larger than one third and 1 was larger than two thirds of the MCA territory on CT. On DWI, 10 patients had large lesions. In 5 patients, the infarction was larger than one third, and in 3 additional patients, the lesion exceeded two thirds of the MCA territory, which were rated in agreement. In 2 additional cases, the raters judged disconcordantly whether the lesions were only larger than one third or larger than two thirds of the MCA territory.
The novices reached a mean stroke sensitivity of 46% (range, 32% to 64%) on early CT, with a specificity of 56%. Accuracy ranged from 35% to 61% (mean, 46%). The negative predictive value differed by between 5% and 11% (mean, 7%). Mean positive predictive value was 93% (range, 90% to 96%). Detection of stroke signs showed a low agreement (κ=0.38). Estimation of lesion size was slightly more concordant (κ=0.44). In 12 patients, all novices detected signs of infarction in the same hemisphere. Eight CTs were concordantly judged as negative; 19 cases were rated the same by 3 of 4 raters.
Readings of the DWI yielded a sensitivity of 78% to 86% (mean, 81%). The agreement of lesion detection was high (κ=0.76). The novices reached a specificity of 100%. Mean positive predictive value was 100%. The estimation of lesion size was moderately concordant (κ=0.55): 35 lesions were detected by all junior raters on DWI. They also judged 5 examinations in full agreement as negative.
Of the 5 lesions judged to be extensive infarctions on CT by the experts, the novices rated 25% as smaller. On DWI, 10 patients showed infarction larger than one third of the MCA territory. In 5%, the novices rated them falsely as smaller. The limited (less than one third of the MCA territory) infarctions identified by the experts on CT were judged as extended lesions in 13.3% by the novices. Based on DWI, the false-positive rate for extensive lesions was 3.4%.
This study was designed to evaluate CT and DWI in hyperacute stroke under identical clinical conditions. Therefore, we randomized eligible patients suffering from hyperacute stroke with respect to the sequence of the imaging modalities. Powers23 and Zivin and Holloway24 have addressed the need for a routine means of comparing imaging modalities in ischemic stroke. The large number of patients screened and recruited in a 14-month period and the short delay between the 2 examinations indicate that we have established a routine procedure for dealing with those patients. However, some patients were not included in the study because of individual medical, infrastructural, or logistical reasons (eg, angiographic procedures with other patients during the hyperacute phase).
Lansberg and coworkers13 presented the results of a prospective trial of 19 cerebral infarctions with CT and DWI examination during the first 7 hours after stroke onset. They observed a higher accuracy of DWI for identifying acute infarction and a good sensitivity for detection of infarctions compared with CT. The median delay between CT and DWI was 2.5 hours, and CT was always done first. Therefore, the higher detection rate of DWI was based not only on a different imaging modality but also on the increase in edema over time. In a letter of correspondence, von Kummer and Gahn25 criticized the small number of patients in that study and the broad overlap of confidence intervals between the results of CT and DWI.
Sensitivity and accuracy of our CT rating (experts) were similar to the results of the experts’ rating according to von Kummer et al7,22⇓ and as precise as the ECASS II CT reading panel. The assessment of the novices in our study showed a wide variety of sensitivity. The mean sensitivity was slightly higher than that of the ECASS local investigators. The experts’ ratings of early stroke signs were as moderately concordant as those observed by von Kummer and coworkers.22 The κ value of stroke diagnosis based on CT was also moderate. The judgment of lesion size in this study showed a fairly good agreement, beyond chance. The sensitivity of DWI was ≈91% by the expert raters. No rater reached a 100% detection rate as reported by Lansberg et al13 and Gonzáles et al.12 Our recently published data from a former cohort of 31 patients with infarctions show κ=1 in lesion detection.14 The slightly lower κ value and sensitivity in the new cohort seem to reflect clinical reality in a routine setting. Even though DWI changes appear seconds after vessel occlusion in animal models26 and cortical hyperintensity was observed in patients 20 minutes after symptom onset, ischemia might not cause changes on DWI if the hypoperfusion allows structural metabolism. Furthermore, the lesions may also transitorily, partially, or permanently disappear on DWI.27,28⇓
The accuracy values in the 2 modalities prove the superiority of DWI. The interrater variability in infarct detection was excellent for DWI. In a larger cohort such as ours, the statistical result is slightly lower than reported in earlier studies. On the other hand, the statistical power with a total number of 54 patients seems to be sufficient. Furthermore, the interrater variability in detecting lesions is significantly better based on DWI.
Concerning the judgment of lesion size, DWI shows a better κ value than CT. In contrast to a previous study, we not only analyzed whether the lesion was larger than one third of the MCA territory but also distinguished 5 different categories. DWI gave a larger number of concordant ratings, and lacunar lesions were also detected more often. As observed in a previous study, the same lesions appear larger on DWI.14 However, before treatment of ischemic stroke can be considered, it is important to detect ischemic lesions as soon as possible and to estimate the potential infarct size because thrombolytic therapy in patients with large infarctions increases the rate of secondary, symptomatic ICH.29 Compared with the expert opinion as gold standard, the novices falsely judged large infarcts to be smaller than one third of the MCA territory in 25%, thus potentially giving recombinant tissue plasminogen activator (rtPA) to patients in whom it is contraindicated because of an excessive risk of ICH based on CT. This was substantially higher than if based on DWI (5%). Conversely, in 13.4% of the patients, the novices falsely judged a small infarction to be larger than one third of the MCA territory based on CT compared with only 3.4% if based on DWI. Therefore, novices are much more likely to either give rtPA to patients who should not be treated or withhold rtPA from patients who should be treated when their decision making is based on CT rather than MRI.
Based on DWI, the threshold of increased risk for symptomatic ICH may be larger then one third of the MCA territory. This hypothesis should be investigated in a multicenter study so that a sufficiently large number of patients is reached. In future studies, the Alberta Stroke Program Early CT Score (ASPECTS) would be a useful tool to describe the ischemic lesions in a more detailed way according to vascular anatomy.30,31⇓ ASPECTS was established for CT within the first 3 hours of ischemic stroke and causes a higher agreement of different raters compared with the one third of the MCA rule. Before implemented into routine use, it should be validated for the 6-hour time window because efforts are being made to extend the rigid therapeutic time frame of 3 hours for thrombolytic therapy on the basis of improved imaging criteria.4
Recently, several groups published results of reversible DWI and apparent diffusion coefficient changes that did not lead to infarction on follow-up images.32,33⇓ Because these groups also investigated hyperacute stroke, this discrepancy between their data and ours cannot be explained by different clinical stages of ischemia. However, one difference between the results of those studies and ours with persistent lesions could be how the data were analyzed. We used a standardized presentation of the images with a low window level with a narrow width. This means of presenting data could render the extent of a lesion smaller and show sharp margins between healthy tissue and the core of infarction.
One drawback of our study is that follow-up imaging was not available in 6 patients. All of these, however, had a persistent focal neurological deficit consistent with the initial diagnosis of ischemic stroke.
Comparison between our novices’ results and the judgment of the local ECASS II investigators revealed a similar sensitivity. In daily routine, stroke patients should be treated as quickly as possible. In contrast to a study reading panel, no department is able to offer full expert assessment 24 hours a day, 7 days a week as part of clinical routine. Most patients are examined by interns or residents in emergency units nights and weekends. Under those circumstances the most reliable imaging modality should be used. When a daily stroke MRI routine is established, this modality leads to a short delay between the initial clinical examination and the initiation of therapy.15
Therefore, DWI should be used instead of CT as the modality of choice in stroke imaging. Despite the ongoing discussion about reversible changes, DWI shows the infarcted core in nearly every ischemic stroke patient and indicates the actual minimum extension of infarction at the time of imaging in nearly every patient.
- Received November 1, 2001.
- Revision received February 20, 2002.
- Accepted February 27, 2002.
- ↵Hacke W, Kaste M, Fieschi C, Toni D, Lesaffre E, von Kummer R, Boysen G, Bluhmki E, Hoexter G, Mahagne MH, Hennerici M. Intravenous thrombolysis with recombinant tissue plasminogen activator for acute hemispheric stroke: the European Cooperative Acute Stroke Study. JAMA. 1995; 274: 1017–1025.
- ↵Hacke W, Kaste M, Fieschi C, von Kummer R, Davalos A, Meier D, Larrue V, Bluhmki E, Davis S, Donnan G, Schneider D, Diez-Tejedor E, Trouillas P. Randomised double-blind placebo-controlled trial of thrombolytic therapy with intravenous alteplase in acute ischaemic stroke (ECASS II). Lancet. 1998; 352: 1245–1251.
- ↵Tong DC, Yenari MA, Albers GW, M OB, Marks MP, Moseley ME. Correlation of perfusion- and diffusion-weighted MRI with NIHSS score in acute (<6.5 hour) ischemic stroke. Neurology. 1998; 50: 864–870.
- ↵Barber PA, Darby DG, Desmond PM, Yang Q, Gerraty RP, Jolley D, Donnan GA, Tress BM, Davis SM. Prediction of stroke outcome with echoplanar perfusion- and diffusion-weighted MRI. Neurology. 1998; 51: 418–426.
- ↵Baird AE, Warach S. Magnetic resonance imaging of acute stroke. J Cereb Blood Flow Metab. 1999; 18: 583–609.
- ↵Lansberg MG, Albers GW, Beaulieu C, Marks MP. Comparison of diffusion-weighted MRI and CT in acute stroke. Neurology. 2000; 54: 1557–1561.
- ↵Schellinger P, Jansen O, Fiebach J, Pohlers O, Ryssel H, Heiland S, Steiner T, Hacke W, Sartor W. Feasibility and practicality of MR imaging of stroke in the management of hyperacute cerebral ischemia. Am J Neuroradiol. 2000; 21: 1184–1189.
- ↵Schellinger P, Jansen O, Fiebach J, Heiland S, Steiner T, Schwab S, Pohlers O, Ryssel H, Sartor K, Hacke W. Monitoring intravenous recombinant tissue plasminogen activator thrombolysis for acute ischemic stroke with diffusion and perfusion MRI. Stroke. 2000; 31: 1118–1128.
- ↵von Kummer R, Bozzao L, Manelfe C, eds. Early CT Diagnosis of Hemispheric Brain Infarction. 1st ed. Berlin, Germany: Springer-Verlag; 1995: 1–95.
- ↵Landis J, Koch G. The measurement of observer agreement for categorical data. Biometrics. 1977; 86: 974–977.
- ↵Tomsick TA, Brott TG, Chambers AA, Fox AJ, Caskill MF, Lukin RR, Pleatman CW, Wiot JG, Bourekas E. Hyperdense middle cerebral artery sign on CT: efficacy in detecting middle cerebral artery thrombosis. AJNR Am J Neuroradiol. 1990; 11: 473–477.
- ↵von Kummer R, Holle R, Grzyska U, Hofmann E, Jansen O, Petersen D, Schumacher M, Sartor K. Interobserver agreement in assessing early CT signs of middle cerebral artery infarction. AJNR Am J Neuroradiol. 1996; 17: 1743–1748.
- ↵Powers W. Testing a test: a report card for DWI in acute stroke. Neurology. 2000; 54: 1549–1551.
- ↵Zivin JA, Holloway RG. Weighting the evidence on DWI: caveat emptor. Neurology. 2000; 54: 1552.
- ↵von Kummer R, Gahn G. Comparison of diffusion-weighted MRI and CT in acute stroke. Neurology. 2000; 55: 1760–1761.
- ↵Rother J, de Crespigny AJ, D’Arceuil H, Iwai K, Moseley ME. Recovery of apparent diffusion coefficient after ischemia-induced spreading depression relates to cerebral perfusion gradient. Stroke. 1996; 27: 980–986.
- ↵Berger C, Fiorelli M, Steiner T, Schabitz WR, Bozzao L, Bluhmki E, Hacke W, von Kummer R. Hemorrhagic transformation of ischemic brain tissue: asymptomatic or symptomatic? Stroke. 2001; 32: 1330–1335.
- ↵Pexman J, Barber P, Hill M, Sevick R, Demchuk A, Hudon M, Hu W, Buchan A. Use of the Alberta Stroke Program Early CT Score (ASPECTS) for assessing CT scans in patients with acute stroke. AJNR Am J Neuroradiol. 2001; 22: 1534–1542.
- ↵Kidwell CS, Alger JR, Di Salle F, Starkman S, Villablanca P, Bentson J, Saver JL. Diffusion MRI in patients with transient ischemic attacks. Stroke. 1999; 30: 1174–1180.