Determination of Duplex Doppler Ultrasound Criteria Appropriate to the North American Symptomatic Carotid Endarterectomy Trial
Background and Purpose The North American Symptomatic Carotid Endarterectomy Trial (NASCET) demonstrated the benefit of carotid endarterectomy for symptomatic patients with ≥70% carotid stenosis. Screening for detection of significant carotid occlusive disease has relied on duplex Doppler imaging. However, traditional duplex categories (50% to 79%, 80% to 99%) are not directly applicable to NASCET. We sought to evaluate duplex criteria for determination of ≥70% carotid stenosis.
Methods Duplex scans and arteriograms of 110 patients (210 carotids), performed within 1 month of each other, were reviewed by blinded readers. Arteriographic stenosis was determined by the NASCET method. Duplex measurements of peak systolic and end-diastolic velocity (PSV, EDV) were recorded, and ratios of velocities in the internal and common carotid arteries (ICA, CCA) were calculated. Receiver-operator characteristic (ROC) curves of sensitivity, specificity, positive and negative predictive values (PPV, NPV), and accuracy were determined.
Results Interobserver agreement for measurement of arteriographic stenosis was “almost perfect” (κ=0.86). The criteria chosen for detection of ≥70% stenosis were PSVICA >210 cm/s (sensitivity, 94%; specificity, 77%; PPV, 68%; NPV, 96%; accuracy, 83%), EDVICA >70 cm/s (sensitivity, 92%; specificity, 60%; PPV, 73%; NPV, 86%; accuracy 77%), PSVICA/PSVCCA >3.0 (sensitivity, 91%; specificity, 78%; PPV, 70%; NPV, 94%; accuracy, 83%), and EDVICA/EDVCCA >3.3 (sensitivity, 100%; specificity, 65%; PPV, 65%; NPV, 100%; accuracy, 79%).
Conclusions We conclude that ≥70% carotid stenosis can be reliably determined by duplex Doppler ultrasound. Individual vascular laboratories must validate their own results.
The NASCET,1 in the context of a randomized controlled clinical trial, demonstrated the benefit of carotid endarterectomy in symptomatic patients with ≥70% carotid diameter reduction. Currently carotid endarterectomy is recommended for symptomatic patients with ≥70% carotid stenosis.
The diagnosis of carotid artery stenosis relies on screening by noninvasive techniques, usually duplex Doppler ultrasound. The determination of degree of carotid stenosis by duplex scanning depends on measurement of Doppler-determined velocity and spectral analysis. Most vascular laboratories use traditional duplex criteria that characterize the carotid bifurcation as normal, 1% to 15% stenosis, 16% to 49% stenosis, 50% to 79% stenosis, 80% to 99% stenosis, and complete occlusion.2 These traditional categories were not designed for determination of ≥70% carotid stenosis. Furthermore, these categories were developed based on an arteriographic definition of stenosis different from that used in the NASCET study. We sought to develop duplex Doppler ultrasound criteria for determination of ≥70% carotid stenosis by comparison with arteriography, using the NASCET method of determination of carotid stenosis.
Subjects and Methods
Between January 1992 and January 1994, 110 patients were identified who had undergone both duplex scanning and carotid angiography at the Hospital of the University of Pennsylvania within 1 month of each other (210 carotid arteries). These patients were being evaluated for surgical treatment of carotid artery atherosclerosis and represent all patients during this interval who had complete arteriographic examinations and duplex scanning data available for review.
Percutaneous catheter arteriograms were obtained in all patients with at least two-view or, in the majority of cases, four-view biplane selective common carotid arteriograms. Carotid arteriograms were performed with either standard cut-film techniques or the use of high-resolution digital subtraction imaging (1024×1024 matrix).
The percent stenosis, determined by the arteriogram, was calculated from direct measurements of the maximum stenosis (minimal residual lumen diameter [MRL]) in the carotid bifurcation region (distal CCA and proximal ICA) made with the use of a hand-held magnifier marked in 1-mm increments. This was compared with the diameter of the normal-appearing ICA distal to the bifurcation (DL) with the technique described for the NASCET study. We calculated diameter stenosis using the MRL and DL in the equation [1−(MRL/DL)]×100.
Observers were blinded to both the results of the duplex study and the other observers’ readings. The first 70 vessels were evaluated by three blinded readers, and an interim calculation of interobserver agreement was made. As a result of the “near perfect” agreement of the three observers (see “Results”), a single observer completed the remaining 140 carotid arteries, providing a total of 210 carotid arteries for evaluation with complete duplex and arteriographic data.
Duplex Doppler Ultrasound
Duplex Doppler ultrasound studies were performed on a Hewlett-Packard Sonos 1000 Color Duplex System with the use of a 7.5-MHz linear array transducer with 5.6-MHz Doppler frequency. The cervical ICA, CCA, and external carotid arteries were examined. Velocity waveforms were obtained routinely from the CCA at the base of the neck, just proximal to the carotid bifurcation; the proximal, mid, and distal ICA; and the external carotid artery at a 60° incident angle. In addition, velocity waveforms were obtained from any location at which stenosis was suspected by either B-mode appearance or color flow mapping. The highest PSV and EDV were recorded from each location.
Maximal PSV and EDV in the carotid bifurcation region (distal CCA or ICA [PSVICA, EDVICA]) were used for comparison with maximal angiographic stenosis. The maximal carotid bifurcation PSVICA and EDVICA were compared with the maximal PSV or EDV in the proximal CCA low in the neck (PSVCCA, EDVCCA) and their ratios (PSVICA/PSVCCA, EDVICA/EDVCCA) calculated.
Sensitivity was defined as the number of true-positive studies divided by the sum of true-positive and false-negative studies; specificity was defined as the number of true-negative studies divided by the sum of false-positive and true-negative studies; PPV was defined as the number of true-positive studies divided by the sum of true-positive and false-positive studies; NPV was defined as the number of true-negative studies divided by the sum of true-negative and false-negative studies; and accuracy was defined as the sum of true-positive and true-negative studies defined by the total number of studies performed.
ROC curves were generated to predict a ≥70% angiographic stenosis. These curves describe sensitivity, specificity, PPV, NPV, and accuracy of each criterion (PSVICA, EDVICA, PSVICA/PSVCCA, EDVICA/EDVCCA).
Interobserver variability for interpretation of arteriographic stenosis was assessed with the κ statistic, in which the degree of agreement between different readers was defined by the scale of Landis and Koch3 : <0.00, poor; 0.00 to 0.20, slight; 0.21 to 0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, substantial; and 0.81 to 1.0, almost perfect.
A total of 210 carotid arteries were evaluable. A ≥70% arteriographic stenosis was present in 69 cases (33%), not including 17 occluded ICAs (8%). Interobserver variability for the first 70 carotid arteries evaluated by three observers was almost perfect (κ=0.86).
Plots of degree of arteriographic stenosis versus the cardinal duplex parameters (PSV, EDV, PSVICA/ PSVCCA, EDVICA/EDVCCA) are shown in Figs 1 through 4⇓⇓⇓⇓. From these scatterplots the sensitivity, specificity, PPV, NPV, and accuracy for various values and combinations of the PSVICA, EDVICA, PSVICA/PSVCCA, and EDVICA/EDVCCA were determined and are shown in Figs 5 through 8⇓⇓⇓⇓ and Table 1⇓. Suggested criteria for determination of ≥70% carotid stenosis are shown in Table 2⇓.
Combined Criteria (Table 1⇑)
When all four criteria for ≥70% stenosis are met (PSVICA >210 cm/s, EDVICA >70 cm/s, PSVICA/PSVCCA >3.0, EDVICA/EDVCCA >3.3), a sensitivity and NPV of 100% are achieved. The specificity (75%) and PPV (72%) are somewhat lower, with an overall accuracy of 85%. Of 69 arteries with ≥70% stenosis, only 13 (19%) met all four criteria. The five false-positive cases identified (there were no false-negative cases) all had stenoses between 60% and 67%, with a mean stenosis of 65%.
Patients with symptomatic carotid artery occlusive disease rarely undergo arteriography as the first diagnostic test. The most widely used screening modality for noninvasive diagnosis of carotid occlusive disease is duplex Doppler ultrasound. The NASCET investigators found a beneficial effect of carotid endarterectomy for symptomatic patients with ≥70% carotid stenosis, which necessitates the development of duplex criteria appropriate to the results of the trial.
Traditional duplex categories of stenosis (50% to 79%, 80% to 99%) are inadequate to properly identify patients who meet the NASCET criteria. Furthermore, the traditional duplex categories, while based on correlation of duplex and arteriogram data, used an alternative method of arteriographic stenosis measurement to that of NASCET. These former validation studies calculated stenosis using the ratio of the residual lumen of the carotid artery compared with the estimated normal bulb diameter, the method of the European Carotid Surgery Trial.4 The NASCET investigators used a comparison of the minimal residual lumen to that of the normal distal ICA measured in its cervical portion. The importance of the method of measurement of angiographic carotid stenosis has been emphasized by several authors.5 6 7 8 Clearly it is imperative that the same arteriographic method be used for validation of duplex studies as was used in the randomized trial if adequate correlation is to be achieved. We used the NASCET method of determination of arteriographic stenosis in this study.
Quantification of stenosis by duplex scanning includes a combination of the information obtained from B-mode ultrasound imaging and Doppler velocity analysis. Both components of the duplex scan are subject to the artifacts and pitfalls inherent to ultrasound technology. Calcific plaque in the carotid artery can produce acoustic shadowing of the B-mode image of the artery and can make the Doppler signal unobtainable in the region of the calcium. Often obtaining alternative views of the same portion of the artery will remedy the situation and allow the artery to be interrogated by Doppler spectral analysis. Characteristics of the Doppler signal proximal and distal to the area that is acoustically opaque because of calcifications also allow for significant information to be obtained from the artery. It is rare that no signal can be obtained as a result of calcification, and we encountered no such examinations in the course of this study. In addition, the presence of thrombus, particularly acute thrombus, within the carotid artery can often be difficult to visualize by B-mode imaging since it is weakly echogenic. This can cause difficulty with differentiation between patent and occluded arteries. The sonographer relies on not only the B-mode image but the absence of a Doppler signal and the presence of a highly obstructed waveform proximal to the suspected occlusion to confirm the diagnosis of carotid occlusion. The B-mode image rather than the Doppler velocity is relied on primarily for determination of lesser degrees of carotid stenosis that are not hemodynamically significant. For higher degrees of carotid stenosis that cause hemodynamic disturbance, the cardinal Doppler parameters described in this study are relied on chiefly to determine the degree of carotid stenosis.
Analysis of Doppler velocity criteria by ROC curves allows the criteria chosen to be tailored to the individual patient and institutional needs. Once the sensitivity, specificity, PPV, NPV, and accuracy of duplex scanning parameters at an individual institution are established, cut points can be chosen that are appropriate for various situations. For example, if one wishes to apply duplex to a population as a screening test for detection of a ≥70% carotid stenosis, one would choose criteria with high sensitivity and NPV so as not to miss any patients who have appropriate lesions. On the other hand, if one desires to perform surgery based on the results of a duplex examination alone without confirmatory arteriography,9 10 11 12 13 high specificity and PPV would be desirable so as not to operate on patients who do not truly meet the criteria of the randomized trial. Once the ROC curve has been generated, optimal criteria may be selected for either indication. It is apparent from examination of the ROC curves that as the test becomes more sensitive, it becomes less specific; as the PPV rises, the NPV declines. The arbitrary criteria suggested in Table 2⇑ were chosen to provide the highest possible accuracy of the test while maintaining high sensitivity and NPV and are best suited to use as a screening test. Alternative criteria could be chosen to afford high specificity and PPV. The duplex criteria chosen should be tailored to the specific institutional and individual situations. This is greatly facilitated by the use of ROC curves.
Our choice of PSVICA >210 cm/s as a criterion for determination of ≥70% carotid stenosis is based on its high accuracy, sensitivity, and NPV in the ROC analysis. This value is intermediate between values recently suggested by other authors (130 cm/s14 and 270 cm/s6 ) and affords similar sensitivity, specificity, predictive values, and accuracy. The wide variability in suggested PSVICA criteria points out the importance of establishing ROC curves for individual institutions. The somewhat low PPV of the PSVICA criterion (68%) is due to 19 false-positive cases. However, the average stenosis in this group of 19 false-positive cases was 66%, with a range of 60% to 68%. Thus, even though the PPV for ≥70% stenosis prediction by this criterion is somewhat low, it is not due to false-positives that stray greatly from the desired threshold value of ≥70%. A higher PSV may be chosen to increase the PPV if duplex is to be used as the sole preoperative imaging modality.
The EDV has been a useful duplex criterion for high-grade stenoses when aliasing is problematic for measurements of PSV. We chose EDVICA >70 cm/s because of its high sensitivity (92%) and NPV (86%), with overall accuracy of 77%. This value is in a range comparable to previously suggested cutoff values for determination of ≥70% carotid stenosis by use of the EDVICA (100 cm/s,14 110 cm/s,6 and 130 cm/s5 ).
Velocity ratios are useful for overcoming variability in interval measurements of PSV and EDV from examination to examination. These ratios remain constant despite the hemodynamic effects of ipsilateral tandem lesions or contralateral stenoses and changes in blood pressure. We found a PSVICA/PSVCCA ratio >3.0 to provide high sensitivity (91%), NPV (94%), and accuracy (83%). This value is in a range compatible with previously reported ratios (≥3.3,14 >4.05 ).
The EDV ratio (EDVICA/EDVCCA >3.3) provides perfect sensitivity and NPV (100%) but an overall accuracy of 79%. This is due to eight false-positive studies with a mean stenosis of 62%.
The range of values for each suggested criterion indicates the importance of individual validation of duplex scanning results against the gold standard of arteriography. This is greatly facilitated by the use of ROC analysis. Once the ROC curve is established, criteria appropriate for use as either a broad screening test (with high sensitivity and NPV) or as a sole preoperative imaging modality before surgery (high PPV) may be selected. The actual criteria chosen for each application must be tailored to the individual situation.
Selected Abbreviations and Acronyms
|CCA||=||common carotid artery|
|ICA||=||internal carotid artery|
|NASCET||=||North American Symptomatic Carotid Endarterectomy Trial|
|NPV||=||negative predictive value|
|PPV||=||positive predictive value|
|PSV||=||peak systolic velocity|
Reprint requests to Jeffrey P. Carpenter, MD, Hospital of the University of Pennsylvania, 4 Silverstein/3400 Spruce St, Philadelphia, PA 19104.
- Received August 24, 1995.
- Revision received November 6, 1995.
- Accepted December 28, 1995.
- Copyright © 1996 by American Heart Association
Roederer GO, Langlois YE, Chan ATW, Primozich J, Lawrence RJ, Chikos PM, Strandness DE. Ultrasonic duplex scanning of the extracranial carotid arteries: improved accuracy using new features from the common carotid artery. J Cardiovasc Ultrasonography. 1982;1:373-380.
Moneta GL, Edwards JM, Chitwood RW, Taylor LM Jr, Lee RW, Cummings CA, Porter JM. Correlation of North American Symptomatic Carotid Endarterectomy Trial (NASCET) angiographic definition of 70% to 99% internal carotid artery stenosis with duplex scanning. J Vasc Surg. 1993;17:152-159.
Neale ML, Chambers JL, Kelly AT, Connrard S, Lawton MA, Roche J, Appleberg M. Reappraisal of duplex criteria to assess significant carotid stenosis with special reference to reports from the North American Symptomatic Carotid Endarterectomy Trial and the European Carotid Surgery Trial. J Vasc Surg. 1994;20:642-649.
Alexandrov AV, Bladin CF, Maggisano R, Norris JW. Measuring carotid stenosis: time for a reappraisal. Stroke. 1993;24:1292-1296.
Wagner WH, Treiman RL, Cossman DV, Foran RF, Levin PM, Cohen JL. The diminishing role of diagnostic arteriography in carotid artery disease: duplex scanning as definitive preoperative study. Ann Vasc Surg. 1991;5:105-110.
Mittl RL, Broderick M, Carpenter JP, Goldberg HI, Listerud J, Mishkin MM, Berkowitz HD, Atlas SW. Blinded-reader comparison of magnetic resonance angiography and duplex ultrasonography for carotid artery bifurcation stenosis. Stroke. 1994;25:4-10.
Faught WE, Mattos MA, Bemmelen PS, Hodgson KJ, Barkmeier LD, Ramsey DE, Sumner DS. Color-flow duplex scanning of carotid arteries: new velocity criteria based on receiver operator characteristic analysis for threshold stenoses used in the symptomatic and asymptomatic carotid trials. J Vasc Surg. 1994;19:818-828.