| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2004;35:2083.)
© 2004 American Heart Association, Inc.
Original Contributions |
From the University Department of Radiology, Addenbrookes Hospital, Cambridge, United Kingdom.
Correspondence to Dr. Jonathan H Gillard, University Department of Radiology, Addenbrookes Hospital, Cambridge, CB2 2QQ UK. E-mail jhg21{at}cam.ac.uk
| Abstract |
|---|
|
|
|---|
Methods 167 symptomatic patients scheduled for DSA, after screening Doppler ultrasound, were prospectively recruited to undergo CEMRA. Severity of stenosis was measured according to the North American Symptomatic Trial Collaborators (NASCET), European Carotid Surgery Trial (ECST), and the common carotid (CC) methods. Measurements for each method were made for 284 vessels (142 included patients) on both CEMRA and DSA in a blinded and randomized manner by 3 independent attending neuroradiologists.
Results Significant differences in prevalence of severe stenosis were seen with the 3 methods on both DSA and CEMRA, with ECST yielding the least and NASCET the most cases of severe stenosis. Overall, all 3 methods performed similarly well in terms of intermodality correlation and agreement. No significant differences in interobserver agreement were found on either modality. With CEMRA, however, we found a significantly lower sensitivity for detection of severe stenosis with ECST (79.8%) compared with NASCET (93.0%), with DSA as reference standard.
Conclusions Uniformity of carotid stenosis measurement methods is desirable because patient management may otherwise differ substantially. All 3 methods are adequate for use with DSA. With CEMRA, however, this study supports use of the NASCET method because of improved sensitivity for detecting severe stenosis.
Key Words: angiography carotid stenosis magnetic resonance imaging
| Introduction |
|---|
|
|
|---|
Despite some initial confusion generated by these different methods, their equivalence as well as differences are now generally well-understood.6 For NASCET, the pooled analysis of individual patient data from NASCET, ECST, and Veteran Affairs Trial has confirmed that surgery is associated with a significant risk reduction in severe (70% to 99%) stenosis and a more marginal risk reduction in moderate stenosis (50% to 69%).3 These cutoff points of 50% and 70% stenosis with NASCET method have been shown to be equivalent to 65% and 82%, respectively, for both ECST and CC methods.3,6 According to these guidelines, centers using ECST or CC methods should therefore recommend surgery for severe stenosis (82% to 99%), with more individual risk assessment required for the moderate stenosis group (65% to 81%).3,6
Concerns regarding the 1% to 2% risk of neurological complications associated with DSA have generated a strong trend toward the use of noninvasive modalities, with contrast-enhanced magnetic resonance angiography (CEMRA) emerging as the modality of choice.7,8 Several well-designed studies have found acceptably low misclassification rates, supporting the safe use of CEMRA compared with DSA.911 All these previous studies have, however, used the NASCET method for quantifying stenosis. The ECST and CC methods have therefore not been previously validated for use with CEMRA. Although DSA is a purely anatomical technique based on intra-arterial injection, CEMRA is a completely different modality, with different spatial resolution, which relies on physiological blood flow and intravenous injection of contrast medium.7 This may have an influence on the readers perception and measurement accuracy made at different locations in the carotid arterial tree. The main objective of this study was, therefore, to directly compare the NASCET, ECST, and CC measurement methods on CEMRA, with DSA as the reference standard.
| Materials and Methods |
|---|
|
|
|---|
Diagnostic Tests
Full details of the imaging protocol have been described elsewhere.9 Briefly, DSA was performed by an attending neuroradiologist, with bilateral selective CCA catheterization, with images of each bifurcation acquired on 4 projections (antero-posterior, lateral, and bilateral 45° obliques) with a resolution of 0.32x0.32 mm.
CEMRA was performed on a 1.5-T machine (Signa CV/I; GE Medical Systems). A bolus-timed 3-dimensional breath-hold fast spoiled gradient echo acquisition with an elliptic-centric phase-encode ordering scheme was performed. Zero-filling interpolation was used, resulting in a final interpolated voxel size of 0.45x0.45x0.40 mm. Each carotid bifurcation was recorded onto film as 4 maximum intensity projections identical to the ones used for DSA.
Image Analysis
Three experienced neuroradiologists, blinded to clinical history and other diagnostic tests, independently reviewed each MRA and conventional angiogram. Patients identifiers on films were masked. The DSA images were arranged in a randomized order and reported over a period of 4 months. The CEMRA images were then subsequently reported over the following 4 months, after re-randomization.
For each study, the projection that demonstrated the tightest narrowing was selected. Four measurements were then made: (1) luminal diameter of the normal distal ICA beyond the bulb where the artery wall becomes parallel; (2) luminal diameter at the site of maximal narrowing; (3) luminal diameter of the estimated original width of the artery at the site of maximal narrowing; and, finally, (4) luminal diameter of the proximal disease-free CCA where the artery wall becomes parallel. The calculations of percentage stenosis according to NASCET, ECST, and CC methods are illustrated in Figure 1.
|
If no stenosis was present, the degree of stenosis was arbitrarily assigned as 0% for all 3 measurement methods. When an ICA demonstrated a near occlusion with collapse of the distal lumen, degree of stenosis was assigned as 95% for all methods.3 For CEMRA, the presence of a signal void was assumed to represent a severe stenosis for each measurement method.12
Statistical Analysis
For the NASCET method, stenosis was classified as mild (0% to 49%), moderate (50% to 69%), severe (70% to 99%), or complete occlusion. Analysis of 1001 angiograms by Rothwell et al also showed an almost equivalent relationship between the ECST and CC methods.6 Therefore, for both ECST and CC methods, stenosis was classified as mild (0% to 64%), moderate (65% to 81%), severe (82% to 99%), or complete occlusion.
All data were entered in SPSS for Windows (V10.1; SPSS Inc) for analysis. For each measurement method, DSA was used as the reference standard. The final reading used was defined as the mean of the 3 independent readers for all 3 methods. Correlation between CEMRA and DSA was calculated using Pearson correlation coefficient and intraclass coefficient. Agreement between the 2 modalities was assessed using the BlandAltman method and Cohen Kappa statistics. Sensitivity, specificity, and predictive values were calculated to characterize the ability of CEMRA to detect DSA-defined severe stenosis. Finally, interobserver agreement was determined for each measurement method using Kappa statistics.
| Results |
|---|
|
|
|---|
Distribution of Disease
Table 1 shows agreement between CEMRA and DSA according to different stenosis categories for the NASCET, ECST, and CC methods, respectively. With DSA, use of NASCET, ECST, and CC methods would have resulted in 71 cases (NASCET), 57 cases (ECST), and 65 cases (CC) of severe stenosis respectively. With CEMRA, use of NASCET, ECST, and CC methods would have resulted in 91 cases (NASCET), 66 cases (ECST), and 81 cases (CC) of severe stenosis, respectively.
|
Correlation
There was excellent and almost identical correlation between CEMRA and DSA irrespective of measurement method (Figure 2). Pearson correlation coefficient was 0.956 (P<0.001) for NASCET, 0.954 (P<0.001) for ECST, and 0.956 (P<0.001) for CC methods. Similarly, intraclass coefficient was 0.978 (95% CI, 0.971, 0.983) for NASCET, 0.976 (95% CI, 0.970, 0.981) for ECST, and 0.977 (95% CI, 0.971, 0.982) for CC methods.
|
Agreement
BlandAltman plots between CEMRA and DSA for each measurement method are shown in Figure 3. With each method, there was a slight tendency for CEMRA to overestimate stenosis compared with DSA, with mean biases ranging from 0.6% to 2.4%. The limits of agreement, within which 95% of the differences between the 2 modalities are expected to lie, were very similar for all 3 methods, ranging from ±20.0% to ±20.3% around the mean biases. Categorical agreement for NASCET (kappa=0.73; 95% CI, 0.67, 0.79), CC (kappa=0.72; 95% CI, 0.65, 0.78), and ECST (kappa=0.70; 95% CI, 0.63, 0.77) methods were also similar in magnitude.
|
Diagnostic Performance
Diagnostic performance of CEMRA for identifying DSA-defined severe stenosis for each measurement method is summarized in Table 2. For specificity, positive predictive values, and negative predictive values, there were no significant differences between the 3 methods. However, the use of ECST method on CEMRA was associated with a much lower sensitivity (78.9%) for detection of severe stenosis compared with NACET (93.0%), a difference that was statistically significant (P=0.015, Fischer exact test). The CC method had an intermediate sensitivity of 87.7%.
|
Interobserver Agreement
For both DSA and CEMRA, there was no significant difference among observers for the different measurement methods. For DSA, mean kappa values for different pairs of readers was 0.84 (0.83, 0.84, and 0.86) for NASCET, 0.83 (0.81, 0.84, and 0.85) for ECST, and 0.84 (0.83, 0.84, and 0.85) for CC methods, respectively. For CEMRA, mean kappa values for different pairs of readers was 0.84 (0.83, 0.83, and 0.85) for NASCET, 0.80 (0.78, 0.79, and 0.82) for ECST, and 0.79 (0.77, 0.79, and 0.82) for CC methods, respectively. These differences for CEMRA were not statistically significant (
2 test, NASCET versus ECST, P=0.298, NASCET versus CC, P=0.198).
| Discussion |
|---|
|
|
|---|
2 test). Our study therefore highlights that while mathematical conversions are useful as an approximate guide, in practice, significant errors of categorization are likely, especially taking into account inherent interobserver variability in measuring carotid stenosis. Rothwell et al reported a rate of misclassification of up to 15% in their original series.6 Unfortunately, in the absence of a gold standard method, it is practically impossible to determine which measurement method is most anatomically correct. Although planimetric measurements of stenosis on excised surgical specimens might theoretically represent a reference standard against which to validate angiographic measurement methods, problems such as surgical damage or specimen shrinkage may limit accuracy.13
The reproducibility of the 3 methods on DSA has been previously studied by several authors. Rothwell et al found a consistently greater reproducibility for the CC method compared with both NASCET and ECST.4 This has been the basis for some centers choosing to use the CC method.5 Vanninen et al found a marginally higher interobserver agreement for the ECST method compared with NASCET.14 In our study, on both DSA and CEMRA, there were no differences between the 3 methods which all had excellent reproducibility, a finding consistent with other studies.14,15
With regards to intermodality correlation and agreement, all 3 methods overall performed similarly. Moreover, using the BlandAltman method we found similar limits of agreement of
±20% between CEMRA and DSA. This suggests that, irrespective of measurement method, sizable disagreements may occur between CEMRA and DSA, findings consistent with previous literature.16,17 However, a substantial part of this disagreement is actually because of the fact that the measurement methods themselves are prone to significant variability, even with experienced readers.9 CEMRA tended to overestimate severity of stenosis; this was marginally more evident with NASCET, as evidenced by the mean biases shown in Figure 3. As illustrated by the spread of the disagreements (Figure 3), differences between CEMRA and DSA were more pronounced with milder degrees of stenosis but this observation was consistent across all 3 measurement methods.
This is the first study to our knowledge to explore whether diagnostic performance of CEMRA varies according to the measurement method used. All 3 methods also had very similar performance characteristics with regard to specificity and predictive values. However, a finding of clinical and statistical significance was that ECST was associated with a much lower sensitivity (78.9%) compared with both NASCET (93.0%) and CC (87.7%). This may be related to the difficulty of visually estimating the bulbous nature of the unseen normal proximal ICA on CEMRA, with its intrinsically lower spatial resolution. Despite the fact that highly impressive submillimeter resolution can now be achieved with CEMRA, the true in-plane resolution (0.80x0.90 mm) is still 2- to 3-times less than that of DSA (0.32x0.32 mm).
A limitation of our study is that our readers, despite being experienced neuroradiologists working in a tertiary referral center with a large annual volume of carotid imaging studies, primarily use the NASCET method in routine clinical practice. However, we carefully researched and minimized potential sources of bias before making our measurements, and this is reflected by our high measures of reproducibility for all 3 methods. Results from other groups are, however, needed to confirm the generalizability of our results. Moreover, we used the artery rather than the patient as observational unit. The assumption of independence of each unit is, however, statistically valid because each artery was reported independently from the contralateral side and that stenosis severity between arteries from the same patient is not correlated.
Rothwell et al, in their pooled analysis of individual patient data, recently called for the NASCET method to be adopted as the standard method because of the potential for confusion.3 Our data show that uniformity of measurement methods is indeed desirable because the use of different methods also potentially results in significant differences in the prevalence of surgically relevant severe stenosis, even allowing for different thresholds for the definition of severe disease. In regard to which method should be adopted, we have shown that overall, no method was clearly superior to any other and all 3 methods are acceptably valid with DSA. However, because the prognostic value of the CC method has been shown indirectly through regression analysis rather than based on primary trial data, we agree with Eliasziw et al and question the value of using this method for decision-making.18 The CC method in our study did not offer any advantages in terms of reproducibility for DSA and CEMRA. We believe that given the strong trend toward noninvasive techniques, the lower sensitivity for detection of severe stenosis with the ECST method with CEMRA provides small but significant support for the NASCET method to be adopted as gold standard, especially for centers that use CEMRA. This is the most logical alternative given that the most compelling body of evidence for guiding clinical practice for carotid disease, the pooled analysis of individual patient data from the major carotid endarterectomy trials, has been analyzed according to the NASCET method.3 However, individual centers that routinely use CC or ECST need to be familiar with the NASCET method and associated potential sources of error, and to validate these measurement methods locally in terms of accuracy and interobserver variability before introducing any changes. Similarly, our study highlights that other noninvasive modalities such as computed tomography angiography need to be formally validated according to the measurement method used and that the assumption that all measurement methods are equal if different thresholds are adopted may not necessarily hold true across different imaging modalities.
Received January 16, 2004; revision received April 21, 2004; accepted June 1, 2004.
| References |
|---|
|
|
|---|
2. Barnett HJ, Taylor DW, Eliasziw M, Fox AJ, Ferguson GG, Haynes RB, Rankin RN, Clagett GP, Hachinski VC, Sacket DL, Thorpe KE, Meldrum HE, Spence JD. Benefit of carotid endarterectomy in patients with symptomatic moderate or severe stenosis. North American Symptomatic Carotid Endarterectomy Trial Collaborators. N Engl J Med. 1998; 339: 14151425.
3. Rothwell PM, Eliasziw M, Gutnikov SA, Fox AJ, Taylor DW, Mayberg MR, Warlow CP, Barnett HJ, Carotid Endarterectomy Trialist Collaboration. Analysis of pooled data from the randomised controlled trials of endarterectomy for symptomatic carotid stenosis. Lancet. 2003; 361: 107116.[CrossRef][Medline] [Order article via Infotrieve]
4. Rothwell PM, Gibson RJ, Slattery J, Warlow CP. Prognostic value and reproducibility of measurements of carotid stenosis. A comparison of three methods on 1001 angiograms. European Carotid Surgery Trialists Collaborative Group. Stroke. 1994; 25: 24402444.[Abstract]
5. Wardlaw JM, Lewis SC, Humphrey P, Young G, Collie D, Warlow CP. How does the degree of carotid stenosis affect the accuracy and interobserver variability of magnetic resonance angiography? J Neurol Neurosurg Psychiatry. 2001; 71: 155160.
6. Rothwell PM, Gibson RJ, Slattery J, Sellar RJ, Warlow CP. Equivalence of measurements of carotid stenosis. A comparison of three methods on 1001 angiograms. European Carotid Surgery Trialists Collaborative Group. Stroke. 1994; 25: 24352439.[Abstract]
7. Berry E, Kelly S, Westwood ME, Davies LM, Gough MJ, Bamford JM, Meaney JF, Airey CM, Cullingworth J, Barbieri M, Jackson A, Smith MA. The cost-effectiveness of magnetic resonance angiography for carotid artery stenosis and peripheral vascular disease: a systematic review. Health Technol Assess. 2002; 6: 1155.[Medline] [Order article via Infotrieve]
8. Willinsky RA, Taylor SM, TerBrugge K, Farb RI, Tomlinson G, Montanera W. Neurologic complications of cerebral angiography: prospective analysis of 2899 procedures and review of the literature. Radiology. 2003; 227: 522528.
9. U-King-Im JM, Trivedi RA, Graves MJ, Higgins NH, Cross JJ, Tom BD, Hollingworth W, Eales H, Warburton EA, Kirkpatrick PJ, Antoun NM, Gillard JH. Contrast-enhanced MR angiography for carotid disease: diagnostic accuracy and potential therapeutic impact. Neurology. 2004; 62: 12821290.
10. Cosottini M, Pingitore A, Puglioli M, Michelassi MC, Lupi G, Abbruzzese A, Calabrese R, Lombardi M, Parenti G, Bartolozzi C. Contrast-enhanced three-dimensional magnetic resonance angiography of atherosclerotic internal carotid stenosis as the noninvasive imaging modality in revascularization decision making. Stroke. 2003; 34: 660664.
11. Remonda L, Senn P, Barth A, Arnold M, Lovblad KO, Schroth G. Contrast-enhanced 3D MR angiography of the carotid artery: comparison with conventional digital subtraction angiography. AJNR Am J Neuroradiol. 2002; 23: 213219.
12. Nederkoorn PJ, van der Graaf Y, Eikelboom BC, van der Lugt A, Bartels LW, Mali WP. Time-of-flight MR angiography of carotid artery stenosis: does a flow void represent severe stenosis? AJNR Am J Neuroradiol. 2002; 23: 17791784.
13. Wijeyaratne SM, Abbott CR, Gough MJ. A modification to the standard technique for carotid endarterectomy allowing removal of intact endarterectomy specimens: implications for research and quality control of preoperative imaging. Eur J Vasc Endovasc Surg. 2002; 23: 370371.[CrossRef][Medline] [Order article via Infotrieve]
14. Vanninen R, Manninen H, Koivisto K, Tulla H, Partanen K, Puranen M. Carotid stenosis by digital subtraction angiography: reproducibility of the European Carotid Surgery Trial and the North American Symptomatic Carotid Endarterectomy Trial measurement methods and visual interpretation. AJNR Am J Neuroradiol. 1994; 15: 16351641.[Abstract]
15. Young GR, Humphrey PR, Nixon TE, Smith ET. Variability in measurement of extracranial internal carotid artery stenosis as displayed by both digital subtraction and magnetic resonance angiography: an assessment of three caliper techniques and visual impression of stenosis. Stroke. 1996; 27: 467473.
16. Young GR, Humphrey PR, Shaw MD, Nixon TE, Smith ET. Comparison of magnetic resonance angiography, duplex ultrasound, and digital subtraction angiography in assessment of extracranial internal carotid artery stenosis. J Neurol Neurosurg Psychiatry. 1994; 57: 14661478.
17. Young GR, Sandercock PA, Slattery J, Humphrey PR, Smith ET, Brock L. Observer variation in the interpretation of intra-arterial angiograms and the risk of inappropriate decisions about carotid endarterectomy. J Neurol Neurosurg Psychiatry. 1996; 60: 152157.
18. Eliasziw M, Smith RF, Singh N, Holdsworth DW, Fox AJ, Barnett HJ. Further comments on the measurement of carotid stenosis from angiograms. North American Symptomatic Carotid Endarterectomy Trial (NASCET) Group. Stroke. 1994; 25: 24452449.[Abstract]
This article has been cited by other articles:
![]() |
M. Anzidei, A. Napoli, B. C. Marincola, I. Nofroni, D. Geiger, F. Zaccagna, C. Catalano, and R. Passariello Gadofosveset-enhanced MR Angiography of Carotid Arteries: Does Steady-State Imaging Improve Accuracy of First-Pass Imaging? Comparison with Selective Digital Subtraction Angiography Radiology, May 1, 2009; 251(2): 457 - 466. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. M. Chappell, J. M. Wardlaw, G. R. Young, J. H. Gillard, G. H. Roditi, B. Yip, J. P. Pell, P. M. Rothwell, M. M. Brown, M. J. Gough, et al. Carotid Artery Stenosis: Accuracy of Noninvasive Tests--Individual Patient Data Meta-Analysis Radiology, May 1, 2009; 251(2): 493 - 502. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. R Jaff, G. V Goldmakher, M. H Lev, and J. M Romero Imaging of the carotid arteries: the role of duplex ultrasonography, magnetic resonance arteriography, and computerized tomographic arteriography Vascular Medicine, November 1, 2008; 13(4): 281 - 292. [Abstract] [PDF] |
||||
![]() |
S Goode, N Altaf, R A Dineen, S Krishnan, and D Auer Intraplaque haemorrhage mimicking carotid pseudoaneurysm on magnetic resonance angiography Br. J. Radiol., November 1, 2007; 80(959): e271 - e274. [Abstract] [Full Text] [PDF] |
||||
![]() |
E. R. Bates, C. J. D. Babb, D. E. Casey, C. U. Cates, G. R. Duckwiler, T. E. Feldman, W. A. Gray, K. Ouriel, E. D. Peterson, K. Rosenfield, et al. ACCF/SCAI/SVMB/SIR/ASITN 2007 Clinical Expert Consensus Document on Carotid Stenting: A Report of the American College of Cardiology Foundation Task Force on Clinical Expert Consensus Documents (ACCF/SCAI/SVMB/SIR/ASITN Clinical Expert Consensus Document Committee on Carotid Stenting) Vascular Medicine, February 1, 2007; 12(1): 35 - 83. [PDF] |
||||
![]() |
American Society of Interventional & Therapeutic N, Society for Cardiovascular Angiography and Interve, Society for Vascular Medicine and Biology, Society of Interventional Radiology, E. R. Bates, J. D. Babb, D. E. Casey Jr, C. U. Cates, G. R. Duckwiler, T. E. Feldman, et al. ACCF/SCAI/SVMB/SIR/ASITN 2007 Clinical Expert Consensus Document on Carotid Stenting: A Report of the American College of Cardiology Foundation Task Force on Clinical Expert Consensus Documents (ACCF/SCAI/SVMB/SIR/ASITN Clinical Expert Consensus Document Committee on Carotid Stenting) J. Am. Coll. Cardiol., January 2, 2007; 49(1): 126 - 170. [Full Text] [PDF] |
||||
![]() |
A.S. Turk, K.M. Johnson, D. Lum, D. Niemann, B. Aagaard-Kienitz, D. Consigny, J. Grinde, P. Turski, V. Haughton, and C. Mistretta Physiologic and Anatomic Assessment of a Canine Carotid Artery Stenosis Model Utilizing Phase Contrast with Vastly Undersampled Isotropic Projection Imaging AJNR Am. J. Neuroradiol., January 1, 2007; 28(1): 111 - 115. [Abstract] [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2004 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |