Published online before print June 12, 2003, doi: 10.1161/01.STR.0000078840.96473.20
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2003;34:1681.)
© 2003 American Heart Association, Inc.
Original Contributions |
Reliability of Assessing Percentage of Diffusion-Perfusion Mismatch
From the Seaman Family MR Research Centre, Foothills Medical Centre, Calgary Health Region (S.B.C., J.E.S., A.I.T., J.R.M., R.F., A.M.D.), and Departments of Clinical Neurosciences (S.B.C., J.E.S., A.I.T., P.A.B., M.E.H., J.R.M., R.F., M.E., A.M.B., A.M.D.), Community Health Sciences (M.E.), and Radiology (J.C., M.E.H., J.R.M., R.F.), University of Calgary, Calgary, Alberta, Canada.
Correspondence to Dr Shelagh Coutts, Seaman Family MR Centre, Foothills Hospital, 1403 29th St NW, Calgary, Alberta T2N 2T9, Canada. E-mail shelagh.coutts{at}calgaryhealthregion.ca
 |
Abstract |
|---|
 Top Abstract IntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Background and Purpose— Emergent neurovascular imaging holds promise in identifying new and optimum target populations for thrombolysis in stroke. Recent research has focused on patients with diffusion-weighted MRI (DWI)-perfusion-weighted MRI (PWI) mismatch as a marker of tissue at risk of infarction and a means to select the most suitable candidates for thrombolysis. The present study sought to estimate the reliability of assessing the percentage of DWI-PWI mismatch.
Methods— Thirteen patients with acute strokes had DWI and PWI within 7 hours of symptom onset. Six raters independently created relative mean transit time (rMTT) maps and then compared them with DWI images to assess the percentage of mismatch (PWI>DWI) in 10% increments. The MR scans were reassessed by 4 raters, tracing around the lesions to calculate the volume percentage of mismatch.
Results— Visual assessment had an interrater reliability of 0.68 (95% CI, 0.52 to 1.0; SEM=21.6%) and an intrarater reliability of 0.80 (95% CI, 0.47 to 1.0; SEM=16.9%). Hand-drawn assessment had an interrater reliability of 0.66 (95% CI, 0.45 to 1.0; SEM=26.2%) and an intrarater reliability of 0.94 (95% CI, 0.81 to 1.0; SEM=10.9%).
Conclusions— Results from the present study suggest that quantifying mismatch by the human eye is reproducible but not reliable among observers. This raises doubts about using mismatch for clinical decision making and clinical trial enrollment.
Key Words: magnetic resonance imaging, diffusion-weighted • magnetic resonance imaging, perfusion-weighted • penumbra • stroke, acute
|
Introduction |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Currently, the only effective treatment for ischemic stroke is intravenous tissue plasminogen activator (tPA) given within 3 hours of symptom onset.1 Unfortunately, most ischemic stroke patients do not reach the emergency department within this time window.2 Furthermore, not all patients treated within 3 hours benefit from thrombolytic therapy.1 Current research is focused on identifying imaging markers that may identify patients most likely to benefit from tPA.3 For example, identification of the ischemic penumbra4 may allow safer treatment with tPA by reducing the risk of intracerebral hemorrhage and may extend the time window for therapeutic intervention.5
Diffusion-weighted MRI (DWI) and perfusion-weighted MRI (PWI) are often used to identify salvageable tissue. DWI, which shows change in the local diffusion of water, allows visualization of early tissue damage, while PWI provides semiquantitative information on cerebral perfusion. When the acute PWI lesion volume is larger than the DWI lesion volume (referred to as mismatch), often the stroke
See Editorial Comment, page 1683
evolves and grows.6–8 Tissue with hemodynamic change but without DWI ischemic lesions may be salvageable. Thus, mismatch has come to be used in patient selection for thrombolysis.9 Visual estimation of the 3-dimensional mismatch is difficult. The aim of the present study was to examine the interrater and intrarater reliability and variability of visually estimating the percentage of mismatch. The study parameters were chosen to mimic a clinical real-time assessment of mismatch that would be used to inform clinical decision making regarding thrombolysis.
|
Subjects and Methods |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Patients
The present study was part of an institutionally approved protocol of emergency MRI of acute stroke patients performed from October 1999 to February 2002. Thirteen patients were selected retrospectively from the acute stroke MR database. An investigator who was not a rater selected study subjects. All patients were scanned within 7 hours of symptom onset, showed evidence of DWI (with restricted apparent diffusion coefficient) or PWI changes, and had optimal scan quality. Patients were chosen to reflect a variety of diffusion and perfusion lesion sizes, location of infarct, and clinical characteristics. The scans included examples of large, medium, and small mismatch (PWI>DWI), as well as examples of match (PWI=DWI) and reverse mismatch (PWI<DWI).
Imaging Protocol
MR images of the brain were obtained with a 3-T scanner (Signa; GE Medical Systems) equipped with high-performance gradients (40 mT/m, 184-µs rise time). All imaging was performed with the use of a standard quadrature head coil. The acute stroke imaging protocol included standard anatomic imaging (T2-weighted, fluid-attenuated inversion recovery [FLAIR], and MR angiography), DWI, and PWI.10 Only the latter 2 imaging sequences were evaluated in this study. DWI was performed with a single-shot, spin-echo, echo-planar imaging technique with a diffusion sensitivity of b=1000 s/mm2, 7000 ms/96 ms (repetition time [TR]/echo time [TE]), 19 5-mm slices with 2-mm gap, 32x19-cm field of view, and a 192x192 acquisition matrix reconstructed to a 256x256 matrix. PWI used a single-shot, gradient-echo, echo-planar sequence with 2200 ms/25 ms. Ten 6-mm sections with a 3-mm gap, 32x19-cm field of view, and a 192x192 acquisition matrix were reconstructed to a 256x256 matrix. Five hundred ten images were collected over 112 seconds during the intravenous administration of a 20-mL bolus of gadopentetate dimeglumine (Magnevist; Berlex) injected at 5 mL/s. PWI acquisitions on our scanner were limited to collection of a maximum of 512 images. In this study we chose to collect 51 time points over 112 seconds (which necessitated a TR of 2200 ms) and to collect 10 slices at each time point. This long TR was chosen because in acute stroke the blood flow is often slow, and we need to acquire perfusion maps over the 90- to 120-second range. To increase the PWI coverage, the slice thickness and gap were increased compared with the DWI acquisition.
Image Analysis
The MR scans were independently assessed twice by 6 raters (2 neuroradiologists, 2 stroke neurologists, and 2 stroke fellows), blinded to all clinical information. All raters were experienced in assessing the acute changes of stroke on MRI. The DWI images were examined for evidence of restricted diffusion on a clinical workstation (Advantage Windows, General Electric Medical Systems). Relative mean transit time (rMTT) maps were created with the use of manufacturer-supplied software (Functool 2000, General Electric Medical Systems). Each rater was trained and used the same method. A PWI abnormality was considered present if there was any visible abnormality on the rMTT map when viewed on a gray scale. The DWI and PWI images were viewed on the MR workstation simultaneously. Windowing was adjusted by the rater, with no fixed levels designated. If DWI>PWI, the scan was rated as having reverse mismatch. If PWI=DWI, a matched deficit was considered present. If there was no evidence of a DWI lesion but there was a PWI lesion, then mismatch=100%. If PWI>DWI, the percentage of mismatch between DWI and the rMTT map was estimated {[(rMTT volume-DWI volume)/rMTT volume]x100%} to the nearest 10%. This led to an estimate of the 3-dimensional mismatch in lesion volumes. The raters repeated the aforementioned procedures after a period of at least 2 weeks to determine intrarater reliability.
Four raters manually drew lesion outlines around the DWI and rMTT lesion areas on all images twice, separated by a period of at least 1 week between each rating. We allowed displayed image contrast (window and level settings) to be varied to optimize visualization of the lesions. Lesion volumes were calculated for each image, and the percentage of mismatch was calculated, where volume=[areaxslice numberx(slice thickness+interslice gap)].
Estimates of interrater and intrarater reliability and SEM were calculated simultaneously from a 2-way random-effects ANOVA.11 One-sided, lower-limit 95% CIs were calculated about the estimates. Further analysis of reliability with the use of dichotomized cutoff points at 10% and 20% mismatch was conducted. Suggested benchmarks for reliability are as follows: slight, 0.00 to 0.20; fair, 0.21 to 0.40; moderate, 0.41 to 0.60; substantial, 0.61 to 0.80; and almost perfect, 0.81 to 1.00.12
|
Results |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
The sample of patients included 6 women and 7 men. The median age was 65 years (range, 45 to 82 years), median time from symptom onset to scan was 152 minutes (range, 72 to 420 minutes), and median National Institutes of Health Stroke Scale score was 5 (range, 0 to 18). An example of a MR scan used in the present study is shown in the Figure.
|
For visual assessment representing the state of the art for real-time clinical decision making, the interrater reliability among 6 raters was 0.68 (95% CI, 0.52 to 1.0; SEM=21.6%). The intrarater reliability was 0.80 (95% CI, 0.47 to 1.0; SEM=16.9%). With the use of a dichotomized cutoff point of >10% versus 
10% mismatch, the interrater reliability was 0.71, and intrarater reliability was 0.78. A dichotomized cutoff point of >20% versus 20% mismatch had lower reliability (interrater=0.60, intrarater=0.72)
For PWI-DWI mismatch derived from volume calculations based on lesion tracing, the interrater reliability among 4 raters was 0.66 (95% CI, 0.45 to 1.0; SEM=26.2%). The intrarater reliability was 0.94 (95% CI, 0.81 to 1.0; SEM=10.9%).
|
Discussion |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Results from the present study suggest that quantifying mismatch by the human eye is reproducible but not reliable among observers. The margin of error between raters was large. For example, if one rater were to estimate the percentage mismatch as 10% and another were to estimate it at 40%, then the difference of 30% would still be within the error of the measurement (±21.6%).
The prognostic significance of mismatch in which the PWI lesion is defined by the rMTT remains unclear, but this method has made its way into clinical decision making and trial design.9,13 For example, the Desmoteplase in Acute Stroke (DIAS) trial13 tests the hypothesis that patients with 
20% DWI-PWI mismatch will benefit from use of intravenous desmoteplase in a 3- to 9-hour time window. We conclude from the results of the present study that real-time determination of percent mismatch may lead to inaccurate clinical characterization for the individual patient. Additionally, the failure to account for this error may lead to underpowered clinical trials. However, the level of intrarater reliability may be sufficient for central review of images by a single individual.
This study was performed at 3.0 T. Imaging at this field strength had a number of advantages and disadvantages compared with 1.5 T. Within the context of this study, 3.0-T imaging would intrinsically have had a higher signal-to-noise ratio, which was used to acquire higher than typical resolution DWI and PWI data. DWI contrast is independent of field strength, whereas PWI contrast in T2*-weighted imaging increases with field strength. In this study the TE at 3 T (25 ms) was reduced to compensate for the additional T2*-weighting.
Advances leading to rapid creation of quantified perfusion maps and computer-assisted volume measurements14 may obviate the need for visual estimates of mismatch in the future. In the interim, we would caution the use of visual inspection of mismatch as the deciding factor in treatment or trial enrollment.
|
Acknowledgments |
|---|
Dr Barber was supported by Heart and Stroke Foundation of Canada, Canadian Institutes for Health Research, and Alberta Heritage Foundation Fellowship. Dr Buchan was supported by the Heart and Stroke Foundation of Canada, Canadian Institutes for Health Research, and the Canadian Stroke Network. Dr Coutts was supported by Heart and Stroke Foundation of Canada fellowship and Alberta Heritage Foundation for Medical Research Fellowship. Dr Demchuk was supported by the Alberta Heritage Foundation for Medical Research and Canadian Institutes for Health Research. Dr Eliasziw was supported by the Natural Sciences and Engineering Research Council of Canada. Dr Frayne was supported by the Alberta Heritage Foundation for Medical Research and the Heart and Stroke Foundation of Canada. Dr Mitchell was supported by the Alberta Heritage Foundation for Medical Research and the Multiple Sclerosis Society of Canada. Dr Simon was supported by a Canadian Stroke Network Fellowship and Alberta Heritage Foundation for Medical Research Fellowship. Dr Tomanek was supported by an Alberta Heritage Foundation for Medical Research Fellowship. The 3.0-T MR scanner used in this study was funded in part by the Canada Foundation for Innovation. The acute stroke imaging was paid for by the Alberta Foundation for Health Research. We would like to acknowledge Dr Michael D. Hill for his help in editing the manuscript.
Received November 18, 2002; revision received January 24, 2003; accepted March 5, 2003.
|
References |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
1. The National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med. 1995; 333: 1581–1587.
2. Barber PA, Zhang J, Demchuk AM, Hill MD, Buchan AM. Why are stroke patients excluded from TPA therapy? An analysis of patient eligibility. Neurology. . 2001; 56: 1015–1020.
3. Albers GW. Expanding the window for thrombolytic therapy in acute stroke: the potential role of acute MRI for patient selection. Stroke. 1999; 30; 2230–2237.[Medline] [Order article via Infotrieve]
4. Hakim AM. Ischemic penumbra: the therapeutic window. Neurology. 1998; 51 (suppl 3): S44–S46.
5. Warach S. Thrombolysis in stroke beyond three hours: targeting patients with diffusion and perfusion MRI. Ann Neurol. 2002; 51: 11–13.[CrossRef][Medline] [Order article via Infotrieve]
6. Neumann-Haefelin T, Wittsack HJ, Wenesrski F, Siebler M, Seitz RJ, Modder U, Freund HJ. Diffusion- and perfusion-weighted MRI: the DWI/PWI mismatch region in acute stroke. Stroke. 1999; 30: 1591–1597.
7. Grandin CB, Duprez TP, Smith AM, Mataigne F, Peeters A, Oppenheim C, Cosnard G. Usefulness of magnetic resonance-derived quantitative measurements of cerebral blood flow and volume in prediction of infarct growth in hyperacute stroke. Stroke. 2001; 32: 1147–1153.
8. Jansen O, Schellinger P, Fiebach J, Hacke W, Sartor K. Early recanalisation in acute ischaemic stroke saves tissue at risk defined by MRI. Lancet. 1999; 353: 2036–2037.[CrossRef][Medline] [Order article via Infotrieve]
9. Fiehler J, Foth M, Kucinski T, Knab R, von Bezold M, Weiller C, Zeumer H, Röther J. Severe ADC decreases do not predict irreversible tissue damage in humans. Stroke. 2002; 33: 79–86.
10. Frayne R, Sevick RJ, Demchuk AM, et al. Clinical stroke imaging at 3 T. In: Proceedings of the Eighth Meeting of the International Society for Magnetic Resonance in Medicine 2000; April 1–7, 2000, Berkeley, Calif. Abstract 1253.
11. Eliasziw M, Young SL, Woodbury MG, Fryday-Field K. Statistical methodology for the concurrent assessment of interrater and intrarater reliability: using goniometric measurements as an example. Phys Ther. 1994; 74: 777–788.
12. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 159–174.[CrossRef][Medline] [Order article via Infotrieve]
13. Desmoteplase in Acute ischemic Stroke (DIAS) Trial. Available at: www.paion.de. Accessed September 25, 2002.
14. Mitchell R, Tomanek A, Simon JE, Ryder R, Frayne R, Sevick R, Barber PA, Demchuk AM, Buchan AM. Reliable computer assisted measurement of stroke lesion volumes. Stroke. 2002; 33: 363.Abstract.
This article has been cited by other articles:
 |
|
 |
|
  R. E. Latchaw, M. J. Alberts, M. H. Lev, J. J. Connors, R. E. Harbaugh, R. T. Higashida, R. Hobson, C. S. Kidwell, W. J. Koroshetz, V. Mathews, et al. Recommendations for Imaging of Acute Ischemic Stroke: A Scientific Statement From the American Heart Association Stroke, November 1, 2009; 40(11): 3646 - 3678. [Full Text] [PDF]
|
|
|
|
 |
|
 S. Siemonsen, T. Fitting, G. Thomalla, P. Horn, J. Finsterbusch, P. Summers, C. Saager, T. Kucinski, and J. Fiehler T2' Imaging Predicts Infarct Growth beyond the Acute Diffusion-weighted Imaging Lesion in Acute Stroke Radiology, September 1, 2008; 248(3): 979 - 986. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. Ay, E. M. Arsava, M. Vangel, B. Oner, M. Zhu, O. Wu, A. Singhal, W. J. Koroshetz, and A. G. Sorensen Interexaminer Difference in Infarct Volume Measurements on MRI: A Source of Variance in Stroke Research Stroke, April 1, 2008; 39(4): 1171 - 1176. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 F CHEN, Q LIU, H WANG, Y SUZUKI, N NAGAI, J YU, G MARCHAL, and Y NI Comparing two methods for assessment of perfusion-diffusion mismatch in a rodent model of ischaemic stroke: a pilot study Br. J. Radiol., March 1, 2008; 81(963): 192 - 198. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Butcher, M. Parsons, L. Allport, S. B. Lee, P. A. Barber, B. Tress, G. A. Donnan, S. M. Davis, and for the EPITHET Investigators Rapid Assessment of Perfusion-Diffusion Mismatch Stroke, January 1, 2008; 39(1): 75 - 81. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Luby and S. Warach Reliability of MR Perfusion-Weighted and Diffusion-Weighted Imaging Mismatch Measurement Methods AJNR Am. J. Neuroradiol., October 1, 2007; 28(9): 1674 - 1678. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 O. Wu, S. Christensen, N. Hjort, R. M. Dijkhuizen, T. Kucinski, J. Fiehler, G. Thomalla, J. Rother, and L. Ostergaard Characterizing physiological heterogeneity of infarction risk in acute human ischaemic stroke using MRI Brain, September 1, 2006; 129(9): 2384 - 2393. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. J. Seitz, S. Meisel, P. Weller, U. Junghans, H.-J. Wittsack, and M. Siebler Initial Ischemic Event: Perfusion-weighted MR Imaging and Apparent Diffusion Coefficient for Stroke Evolution Radiology, December 1, 2005; 237(3): 1020 - 1028. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Prosser, K. Butcher, L. Allport, M. Parsons, L. MacGregor, P. Desmond, B. Tress, and S. Davis Clinical-Diffusion Mismatch Predicts the Putative Penumbra With High Specificity Stroke, August 1, 2005; 36(8): 1700 - 1704. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K.S. Butcher, M. Parsons, L. MacGregor, P.A. Barber, J. Chalk, C. Bladin, C. Levi, T. Kimber, D. Schultz, J. Fink, et al. Refining the Perfusion-Diffusion Mismatch Hypothesis Stroke, June 1, 2005; 36(6): 1153 - 1159. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
N. Hjort, K. Butcher, S.M. Davis, C.S. Kidwell, on behalf of the UCLA Thrombolysis Investigators, W.J. Koroshetz, J. Rother, P.D. Schellinger, S. Warach, and L. Ostergaard Magnetic Resonance Imaging Criteria for Thrombolysis in Acute Cerebral Infarct Stroke, February 1, 2005; 36(2): 388 - 397. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Sobesky, O. Z. Weber, F.-G. Lehnhardt, V. Hesselmann, A. Thiel, C. Dohmen, A. Jacobs, M. Neveling, and W.-D. Heiss Which Time-to-Peak Threshold Best Identifies Penumbral Flow?: A Comparison of Perfusion-Weighted Magnetic Resonance Imaging and Positron Emission Tomography in Acute Ischemic Stroke Stroke, December 1, 2004; 35(12): 2843 - 2847. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. E. Rose, A. L. Janke, M. Griffin, S. Finnigan, and J. B. Chalk Improved Prediction of Final Infarct Volume Using Bolus Delay-Corrected Perfusion-Weighted MRI: Implications for the Ischemic Penumbra Stroke, November 1, 2004; 35(11): 2466 - 2471. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. S. Butcher, M. W. Parsons, S. Davis, G. Donnan, S. B. Coutts, J. E. Simon, A. M. Demchuk, R. Frayne, and J. R. Mitchell PWI/DWI Mismatch: Better Definition Required * Response Stroke, November 1, 2003; 34 (11): e215 - e216. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||