This Article Abstract Alert me when this article is cited Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Eliasziw, M. Articles by Barnett, H.J.M. Search for Related Content PubMed PubMed Citation Articles by Eliasziw, M. Articles by Barnett, H.J.M.

(Stroke. 1995;26:1747-1752.)
© 1995 American Heart Association, Inc.

Articles

Accuracy and Prognostic Consequences of Ultrasonography in Identifying Severe Carotid Artery Stenosis

M. Eliasziw, PhD; R.N. Rankin, MB, ChB; A.J. Fox, MD; R.B. Haynes, MD; H.J.M. Barnett, MD for the North American Symptomatic Carotid Endarterectomy Trial (NASCET) Group

From the Departments of Epidemiology and Biostatistics (M.E.), Clinical Neurological Sciences (M.E., A.J.F., H.J.M.B.), and Diagnostic Radiology (R.N.R., A.J.F.), University of Western Ontario; The John P. Robarts Research Institute (M.E., H.J.M.B.), London, Ontario; and the Departments of Clinical Epidemiology and Biostatistics and of Medicine, McMaster University, Hamilton, Ontario (R.B.H.), Canada.

Correspondence to H.J.M. Barnett, The John P. Robarts Research Institute, 100 Perth Dr, London, Ontario, Canada, N6A 5K8.

	Abstract

Top Abstract Introduction Subjects and Methods Results Discussion References

 
Background and Purpose The accuracy of routine ultrasonography in detecting severe carotid artery stenosis was evaluated in comparison with cerebral angiography. The precision of ultrasonographic criteria in predicting the risk of stroke was also assessed.

Methods A total of 1011 symptomatic carotid bifurcations were studied in patients from the North American Symptomatic Carotid Endarterectomy Trial (NASCET). Given that all patients were considered for entry into the trial, the chance of a verification bias affecting the analyses was minimized. The ultrasonographic data consisted of peak systolic velocities and frequency changes from both the internal and common carotid arteries. Angiographic stenosis was calculated as in NASCET. Receiver operating characteristic (ROC) curves were constructed from the ultrasonographic data for the detection of 70% or greater stenosis on the basis of an angiographic assessment. Kaplan-Meier stroke-free survival curves were used to predict the risk of stroke.

Results The areas under the ROC curves ranged from 0.74 to 0.75 (95% confidence interval [CI], 0.69 to 0.79). The sensitivities and specificities ranged from 0.65 to 0.71. The risk of stroke at 18 months declined sharply as the degree of angiographically defined stenosis declined from 99% to 70%. No pattern of decline was apparent on the basis of the ultrasonographic data.

Conclusions The results indicate that the accuracy of ultrasonography is moderate when flow parameters are used to assess the degree of stenosis. Ultrasonography should be used as a screening tool to exclude patients with no carotid artery disease from further testing. Conventional angiography remains an essential investigation before assigning the risk of stroke and deciding appropriate treatment for extracranial carotid artery disease.

Key Words: angiography • carotid arteries • carotid endarterectomy • ultrasonics

	Introduction

Top Abstract Introduction Subjects and Methods Results Discussion References

 
Carotid ultrasonography has become widely available and is currently the principal noninvasive tool for evaluating carotid artery disease. Screening for this disease has become particularly important in light of the published reports from the North American Symptomatic Carotid Endarterectomy Trial¹ and the European Carotid Surgery Trial (ECST)² in 1991. These clinical trials showed that carotid endarterectomy plus best medical therapy significantly reduces the risk of stroke in symptomatic patients with 70% to 99% angiographically defined stenosis of the ICA in comparison to patients who receive best medical therapy alone. For symptomatic patients with lesser degrees of stenosis, the benefit of endarterectomy is unknown, as randomization of such patients into NASCET and follow-up of patients in ECST still continues.

The announcement of significant surgical benefits has yielded a flurry of publications aimed at determining ultrasonographic criteria for identifying an angiographic carotid artery stenosis of 70% and above.^{3 4 5 6 7} Although there is clearly a need to safely identify those symptomatic patients with 70% stenosis and offer them surgical therapy, the lack of uniformity among the reported criteria casts doubt over the ability of ultrasonography to accurately pinpoint a 70% stenosis.

Further uncertainties about ultrasonography have been perpetuated by the use of unsuitable study designs in assessing accuracy. All of the reported studies prescreened patients with ultrasonography. Those who were suspected of having severe stenosis on the basis of ultrasonographic findings alone proceeded to angiography as a prelude to carotid endarterectomy. Patients considered to have lesser degrees of stenosis did not have their disease status verified by angiographic investigations. Given that the process of selecting patients for angiographic verification depended on the results of the ultrasonographic findings, the resulting estimates of accuracy are seriously biased. Specifically, the reported sensitivities (true positive rates) and specificities (true negative rates) may be erroneously inflated.⁸ Ransohoff and Feinstein⁹ have described this phenomenon as workup bias (or verification bias). Also, the sample sizes used in the studies were small, and the distribution of angiographic stenoses in the samples was rarely reported.

Data collected from the first 3 years of NASCET (1988 through 1991) provide an ideal opportunity to examine the accuracy of ultrasonography in detecting a 70% stenosis of the ICA. Before the termination of the first part of NASCET in February 1991, all symptomatic patients were considered for entry into the trial. In accordance with study protocol, patients were entered into the trial only if the participating center determined the stenosis to be 30% to 99% by angiography. Adherence to the protocol has assured a broad distribution of stenoses in the sample of patients and has minimized the chance of a verification bias from entering into the analyses. The clinical outcome data from the trial also allow the assessment of accuracy in a unique manner. One is able to determine how precisely the ultrasonographic criteria are able to predict the patients' future risk of stroke and the beneficial effects of endarterectomy compared with angiography.

	Subjects and Methods

Top Abstract Introduction Subjects and Methods Results Discussion References

 
Subjects
NASCET is a prospective multicenter study designed to determine the role of carotid endarterectomy for symptomatic patients with moderate (30% to 69%) and severe (70% to 99%) angiographically defined stenosis at the carotid bifurcation. Patients with ischemic symptoms (transient ischemic attack or nondisabling stroke) were recruited if their latest appropriate symptom (related to their carotid lesion) had occurred within 120 days before randomization and if the ischemic event was not attributable to a cardiac source of embolism. Other exclusion criteria included age over 80 years, presence of significant intracranial vascular disease, and life-threatening or other disabling conditions that could interfere with the interpretation of outcome events. Full details of the study protocol have been published.^{1 10} From January 1988 through February 1991, 1360 patients were recruited from 50 academic centers across North America. At entry into the trial, patients were randomly assigned to receive either medical care alone or medical care plus carotid endarterectomy. Follow-up consisted of clinical examinations every 3 months during the first year and every 4 months thereafter, at which time occurrences of stroke were noted.

Neuroimaging
Biplane (anterior-posterior, lateral, and/or oblique) selective carotid angiography was used for the assessment of the degree of stenosis in all patients at entry. Ultrasonography was performed concurrently to the angiogram but was not used in the decision-making process for entering patients into the study. Its purpose was to establish a baseline study for comparison with subsequent follow-up studies. Ultrasonographic examinations included both longitudinal and transverse views of the carotid bifurcation and the relevant stenosis. Also included were Doppler readings from the site of maximal stenosis, the normal ICA, the CCA, and the external carotid artery. The vast majority of the transducers used were in the 5-MHz range, and almost all the recordings were made at a reasonably standard angle of 60°. Hard copies of all angiograms and ultrasonographic scans were sent to the central office. Videotapes of the ultrasonographic scans were not used because of the different formats available and the length of time required to review large numbers of examinations. Angiograms were reviewed blindly by the principal neuroradiologist (A.J.F.), and the degree of stenosis was measured by the strict criteria advocated in NASCET.¹¹ The degrees of interobserver and intraobserver reliability for measuring percentage of stenosis were high (ICC: 0.93 and 0.95, respectively).¹² These figures provide reassurance for considering angiography to be the "comparative gold standard" in the present study. The ultrasonographic scans were read independently of the angiograms by the principal neurosonographer (R.N.R.) and without knowledge of the angiographic results. Data recorded from the ultrasonographic hard copies consisted of the ICPSV or ICPFC and the CCPSV or CCPFC. A subset of 54 scans reporting peak systolic velocity and 48 scans reporting peak frequency changes were selected to estimate the degrees of interobserver and intraobserver reliability. The results indicate high reliability (ICPSV, ICC: 0.97 and 0.98, respectively; CCPSV, ICC: 0.92 and 0.88, respectively; ICPFC, ICC: 0.98 and 0.97, respectively; CCPFC, ICC: 0.79 and 0.76, respectively).

Statistical Analysis
ROC curves were constructed to assess the accuracy of ultrasonographic criteria for detecting a 70% angiographically defined stenosis. A ROC curve is a graph that displays the relationship between the sensitivity and specificity of a diagnostic tool across a spectrum of cutpoints that could be used to classify patients as diseased or nondiseased. The overall accuracy of a diagnostic tool is expressed in terms of the area under the ROC curve, ranging from 0.5 (poor) to 1.0 (perfect). The LABROC4 computer program developed by Metz¹³ was used to generate the fitted curves and the areas under the curves. Corresponding two-sided 95% CIs for the areas were also calculated. Kaplan-Meier survival curves were used to estimate the risk of ipsilateral stroke at 18 months for each decile category of stenosis, as defined by angiography and ultrasonography.

The ultrasonographic decile categories of stenosis were defined on the basis of the ICPSV, ICPFC, and ICA/CCA ratio (Table 1). The ultrasonographic cutpoints appearing in Table 1 were selected to be representative of the ones reported in the literature. For example, an ICPSV cutpoint of 250 cm/s for a 70% angiographic stenosis is considered to be representative of the reported ICPSV values ranging from 175 to 325 cm/s,^{3 4 5 6 7 14 15} whereas an ICA/CCA ratio cutpoint of 3.0 is representative of values ranging from 3.0 to 4.0.^{4 5 16} Cutpoints for higher degrees of stenosis were interpolated from published scattergrams that showed the relationship between ultrasonographic velocities, ratios, and the degree of angiographically defined carotid stenosis.^{3 7 14 16} Although peak systolic velocities and frequency changes are more commonly used, the peak systolic ICA/CCA ratio has also been included, since it has appeared in the literature on several occasions.^{3 4 14 16 17}

View this table: [in this window] [in a new window]   Table 1. Ultrasonographic Criteria Defining Decile Ranges of Stenosis

	Results

Top Abstract Introduction Subjects and Methods Results Discussion References

 
A total of 1011 patients had complete ultrasonographic data sent to the central office. For the other 349 patients, ultrasonographic data were either incomplete or unavailable. Among the 1011, 633 patients (62.6%) had ultrasonographic scans in participating centers using peak systolic velocity, and the remaining patients were in centers using peak frequency changes. The raw data are shown as scattergrams in Fig 1. The sample of patients is representative of a broad range of carotid disease severity, regardless of the diagnostic tool used. There is an apparent overall positive relationship between the ultrasonographic and angiographic measurements. However, the amount of variability increases as the severity of stenosis increases.

View larger version (0K): [in this window] [in a new window]   Figure 1. Scattergrams of internal carotid peak systolic velocity (top), internal carotid peak frequency change (middle), and ratio of internal carotid peak systolic velocity (or frequency change) to common carotid peak systolic velocity (or frequency change) versus angiographically defined carotid stenosis (bottom).

ROC Curves
The ROC curves, illustrating the accuracy with which ultrasonographic criteria are able to detect an angiographic stenosis of 70% or greater, are shown in Fig 2. The four curves are virtually identical. The areas under the ROC curve range from 0.74 to 0.75 (Table 2). The sensitivity and specificity at the curves' point of inflection are also given in Table 2. Further analyses were carried out to determine if the size of the participating center or the year in which the imaging studies were carried out affected the results. Size was determined on the basis of the number of patients that a participating center entered into the study: small (<20 patients), medium (21 to 35 patients), and large (>35 patients). Three calendar years were considered: 1988, 1989, and 1990 to 1991. There were no apparent differences among the areas under the ROC curve calculated for centers of different sizes or across the three consecutive time periods. Overall, the results from the ROC curve analyses indicate that the accuracy of ultrasonography is moderate when peak systolic velocities and frequency changes from both the ICA and CCA are used to assess the degree of stenosis.

View larger version (0K): [in this window] [in a new window]   Figure 2. Receiver operating characteristic (ROC) curves illustrating the accuracy with which ultrasonography criteria are able to detect an angiographic stenosis of 70% or greater: top, peak systolic velocity (solid line indicates ICPSV; dotted line, ICPSV/CCPSV ratio); bottom, peak systolic frequency change (solid line, ICPFC; dotted line, ICPFC/CCPFC).

View this table: [in this window] [in a new window]   Table 2. Areas Under the Receiver Operating Characteristic Curves

Prognostic Predictions
Of the 1011 patients, 484 had angiographically defined stenosis of 70% at baseline. Since NASCET is an ongoing trial that is still entering patients, the protocol allows only disclosure of the follow-up data on severe (70% stenosis) patients. Therefore, this subset forms the sample for examining the patients' prognoses by decile category of stenosis. The risks of ipsilateral stroke at 18 months are given in Table 3. Ipsilateral strokes are any strokes that occur on the same side as the artery for which the patient was randomized into the trial. It is observed that the risk of stroke for medically treated patients increases considerably as the degree of angiographically defined stenosis increases by decile category. Given the relatively constant risk for the surgically treated patients, the net result is a declining benefit of surgery that corresponds to a decline in the percentage of stenosis. These relationships are not observed for the ultrasonographic criteria based on absolute measures of ICPSV and ICPFC. Paradoxically, the greatest risk of stroke occurs among patients who have been diagnosed by ultrasonography as having <70% stenosis (Table 3). Of the 184 patients in this category, 44 (24%) were deemed to have 90% stenosis when measured by angiography. This misclassification is reflected in the elevated value for the mean angiographic stenosis for this decile category. A similar result is observed in the <70% decile category when ultrasonographic ratios were used to define the severity of stenosis (Table 3). Despite this, the risks of stroke and benefits of surgery derived using ultrasonographic ratios have a similar decline as was observed for angiographically defined stenosis (Table 3).

View this table: [in this window] [in a new window]   Table 3. Risk of Ipsilateral Stroke at 18 Months According to Stenosis Defined by Angiography and Ultrasonographic Criteria

	Discussion

Top Abstract Introduction Subjects and Methods Results Discussion References

 
In an attempt to reduce the morbidity, mortality, and cost associated with angiography, ultrasonography is increasingly being used as the primary method for assessing the severity of disease in the carotid arteries and for making decisions with regard to additional angiographic investigations and/or carotid endarterectomy. However, its accuracy in detecting operable lesions is still a subject of debate.¹⁸ In contrast to previous optimistic reports of high levels of accuracy (ranging from 0.81 to 0.98^{3 4 5 6 7 19 20 21 22} ), the present study indicates that the level of accuracy may only be moderate (areas under the ROC curve ranging from 0.74 to 0.75).

An important, but often overlooked, reason for this divergence in results is the manner by which patients were selected for angiographic verification of carotid stenosis,^{9 23} usually as a prelude to carotid endarterectomy. One study reported that of the 1873 patients who had carotid ultrasonography studies, only 96 (5.1%) were subjected to subsequent angiographic verification.³ Since only the angiographically verified cases were used in the ROC analyses, the authors reported that the estimates of accuracy were expectedly biased upward. After correcting the analyses for verification bias, the areas under the ROC curves decreased substantially.³ Specifically, the area under the ROC curve corresponding to ICPSV dropped from 0.94 to 0.78. Similarly, the ROC area corresponding to ICPSV/CCPSV dropped from 0.91 to 0.66. The corrected figures reported in this previous study are consistent with our results.

The use of ultrasonography as the sole diagnostic test for the selection of patients for carotid endarterectomy has been widely discussed.^{24 25 26 27 28 29 30} The results of our study indicate that critical errors can occur. The number of false negatives for 70% stenosis is disconcerting. Ultrasonography will fail to identify some near occlusions, mistaking them for mild to moderate lesions because of the lack of turbulence resulting from the severely limited flow, or alternatively misread them as occlusions. A near occlusion mistakenly interpreted as an occlusion may deny the opportunity for endarterectomy to a patient for whom it is most urgently needed. The use of color-flow Doppler to increase the accuracy of placing the Doppler cursor for duplex ultrasonography appears to slightly increase the accuracy of stenosis determination^{31 32} but not by an amount that is likely to significantly alter the findings of this study. Color-flow Doppler is useful, however, in differentiating near occlusions from occlusions, since a string of color may often be seen in a nearly occluded artery.^{33 34}

As shown by the results of the present study, there is a sharp decline in the risk of stroke and benefit from endarterectomy as the degree of angiographically defined stenosis declines from 99% to 70% (Table 3). This declining benefit underscores the importance of identifying precisely the degree of stenosis. Careful measurement is becoming more universal as a prelude to decision making about endarterectomy than ever before.

Even if ultrasonography becomes more refined and better correlated with the angiographic measurements used in NASCET, there still remain arterial features other than degree of stenosis that escape identification but have prognostic importance. Ultrasonography may not identify the presence of intraluminal thrombus, it is inadequate for the identification of ulceration, and it will not reveal intracranial carotid or middle cerebral artery stenosis or occlusion of the middle cerebral trunk. It fails to visualize intracranial vascular anomalies and intracranial aneurysms.^{35 36} Although none of these individual conditions occur in large numbers, the aggregate is sufficient to deter most neurologists and surgeons from accepting ultrasonography alone as a prelude to carotid endarterectomy.

Possible errors in identifying carotid lesions using Doppler sonography can be corrected by reviewing MR angiography images of the carotid bifurcations.^{21 28 37 38} Although MR angiography has major advantages in being able to display the carotid bifurcation and intracranial arteries in a format similar to that of conventional angiography, it remains imperfect in giving precise measurements of stenosis. At present, the degree of stenosis is overestimated, and the problem of signal void related to turbulence has not been resolved.^{39 40}

Some have argued that angiography carries too much risk.^{28 41} This would be an acceptable deterrent for its use if endarterectomy was free of risk and if perfection in estimating stenosis was achievable by a combination of ultrasonography and MR angiography. Recent randomized trials of carotid endarterectomy in symptomatic patients reported perioperative rates of stroke (of any severity) and death between 5.8% and 7.5%.^{1 2 42} In comparison, a literature review reporting on eight early prospective studies (spanning the years 1973 through 1984) has estimated that the overall risk of any stroke due to angiography was 1.0% (95% CI, 0.6 to 1.5) and the overall rate of mortality was 0.06% (95% CI, 0.0 to 0.18).⁴³ Among the 2320 patients currently enrolled in NASCET, 15 patients (0.65%) had postangiographic strokes (H.J.M.B., E.M., unpublished data, 1995). Given that NASCET excludes patients with major disabling strokes from entry into the trial, and because the published prospective studies of stroke complications from angiography consistently reported a 5-to-1 ratio of nondisabling ischemic events to disabling strokes, an additional three events (representing disabling strokes) may be added to the angiographic complication rate of potential NASCET patients. This increases the number of strokes from 15 to 18, thus yielding a stroke rate from angiography of 0.78%.

These figures may be higher than can be presently achieved. Over the past 10 years, selective cerebral angiography has undergone profound changes. There have been important advances in catheter design, nonionic contrast agents are routinely used, and high-resolution digital systems have been introduced that not only reduce contrast medium load but also shorten examination time. All of these improvements aim to reduce the risk of angiographic complications. Two recently published studies using digital subtraction angiography estimated the risk of any stroke to be between 0.09% and 0.3%.^{44 45} While angiography still carries a minimal but definite risk of complication, at the same time ultrasonography cannot claim to be free of risk. In the past 25 years, four neurological and two cardiac adverse events have been reported to be associated with the performance of carotid ultrasonography.^{46 47 48}

On the basis of our results from analyzing data from 50 academic centers across North America, we recommend that ultrasonography be used as a screening tool to exclude patients with no carotid artery disease from further testing. Patients suspected of having disease, and who are suitable for endarterectomy, should have confirmatory angiography before a treatment decision is made. Furthermore, those patients who have 30% to 69% diameter reduction by the method of measurement used in NASCET, and who show progression on follow-up ultrasonographic examination, should have angiography if they appear to have progressed to 70% or beyond before deciding on further treatment. The immediate elimination of conventional angiography in favor of combined Doppler and MR angiography is premature.^{18 22 49} Ultrasonography (using peak systolic velocities and frequency changes from both the ICA and CCA) is less capable than angiography of identifying with comparable precision a stenosis that angiography recognizes to be at or above 70%. Conventional angiography remains an essential investigation before carotid endarterectomy and while everyone awaits the final perfection and widespread availability of both color-flow Doppler and MR angiography.

	Selected Abbreviations and Acronyms

 

CCA = common carotid artery CCPFC = common carotid peak frequency change CCPSV = common carotid peak systolic velocity CI = confidence interval ICA = internal carotid artery ICC = intraclass correlation coefficient ICPFC = internal carotid peak frequency change ICPSV = internal carotid peak systolic velocity NASCET = North American Symptomatic Carotid Endarterectomy Trial ROC = receiver operating characteristic

	Acknowledgments

 
This study was supported by grant R01-NS-24456 from the National Institute of Neurological Disorders and Stroke. The authors acknowledge the support of all participants in NASCET and thank SmithKline Beecham for providing Ecotrin for all NASCET patients. The authors thank Dr Charles Metz for providing the computer software used to generate the ROC curves and Dr William Dawson for reading the ultrasonographic scans in the reliability study.

Received February 7, 1995; revision received May 4, 1995; accepted May 25, 1995.

	References

Top Abstract Introduction Subjects and Methods Results Discussion References

 

1. North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis. N Engl J Med. 1991;325:445-453. [Abstract]
2. European Carotid Surgery Trialists' Collaborative Group. MRC European Carotid Surgery Trial: interim results for symptomatic patients with severe (70-99%) or with mild (0-29%) carotid stenosis. Lancet. 1991;337:1235-1243. [Medline] [Order article via Infotrieve]
3. Hunink MGM, Polak JF, Barlan MM, O'Leary DH. Detection and quantification of carotid artery stenosis: efficacy of various Doppler velocity parameters. AJR. 1993;160:619-625. [Abstract/Free Full Text]
4. Moneta GL, Edwards JM, Chitwood RW, Taylor LM, Lee RW, Cummings CA, Porter JM. Correlation of North American Symptomatic Carotid Endarterectomy Trial (NASCET) angiographic definition of 70% to 99% internal carotid artery stenosis with duplex scanning. J Vasc Surg. 1993;17:152-159. [Medline] [Order article via Infotrieve]
5. Faught WE, Mattos MA, van Bemmelen PS, Hodgson KJ, Barkmeier LD, Ramsey DE, Sumner DS. Color-flow duplex scanning of carotid arteries: new velocity criteria based on receiver operator characteristic analysis for threshold stenoses used in the symptomatic and asymptomatic carotid trials. J Vasc Surg. 1994;19:818-828. [Medline] [Order article via Infotrieve]
6. Neale ML, Chambers JL, Kelly AT, Connard S, Lawton MA, Roche J, Appleberg M. Reappraisal of duplex criteria to assess significant carotid stenosis with special reference to reports from the North American Symptomatic Carotid Endarterectomy Trial and the European Carotid Surgery Trial. J Vasc Surg. 1994;20:642-649. [Medline] [Order article via Infotrieve]
7. Browman MW, Cooperberg PL, Harrison PB, Marsh JI, Mallek N. Duplex ultrasound criteria for internal carotid stenosis >70% diameter: angiographic correlation and ROC curve analysis. Can Assoc Radiol J. 1995. In press.
8. Begg CB, Greenes RA. Assessment of diagnostic tests when disease verification is subject to selection bias. Biometrics. 1983;39:207-215. [Medline] [Order article via Infotrieve]
9. Ransohoff DF, Feinstein AR. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. N Engl J Med. 1978;299:926-930. [Abstract]
10. North American Symptomatic Carotid Endarterectomy Trial (NASCET) Steering Committee. North American Symptomatic Carotid Endarterectomy Trial: methods, patient characteristics, and progress. Stroke. 1991;22:711-720. [Abstract/Free Full Text]
11. Fox AJ. How to measure carotid stenosis. Radiology. 1993;186:316-318. [Free Full Text]
12. Eliasziw M, Smith RF, Singh N, Holdsworth DW, Fox AJ, Barnett HJM. Further comments on the measurement of carotid stenosis from angiograms. Stroke. 1994;25:2445-2449. [Abstract]
13. Metz CE. Some practical issues of experimental design and data analysis in radiological ROC studies. Invest Radiol. 1989;24:234-245. [Medline] [Order article via Infotrieve]
14. Ranke C, Creutzig A, Alexander K. Duplex scanning of the peripheral arteries: correlation of the peak velocity ratio with angiographic diameter reduction. Ultrasound Med Biol. 1992;18:433-440. [Medline] [Order article via Infotrieve]
15. Robinson ML, Sacks D, Perlmutter GS, Marinelli DL. Diagnostic criteria for carotid duplex sonography. AJR. 1988;151:1045-1049. [Abstract/Free Full Text]
16. Blackshear WM, Phillips DJ, Chikos PM, Harley JD, Thiele BL, Strandness DE. Carotid artery velocity patterns in normal and stenotic vessels. Stroke. 1980;11:67-71. [Abstract/Free Full Text]
17. Carroll BA. Carotid sonography. Radiology. 1991;178:303-313. [Free Full Text]
18. Ackerman RH, Candia MR. Identifying clinically relevant carotid disease. Stroke. 1994;25:1-3. [Medline] [Order article via Infotrieve]
19. Poindexter JM, Shah PM, Clauss RH, Babu SC. The clinical utility of carotid duplex scanning. J Cardiovasc Surg. 1991;32:64-68. [Medline] [Order article via Infotrieve]
20. Mattle HP, Kent KC, Edelman RR, Atkinson DJ, Skillman JJ. Evaluation of the extracranial carotid arteries: correlation of magnetic resonance angiography, duplex ultrasonography, and conventional angiography. J Vasc Surg. 1991;13:838-844. [Medline] [Order article via Infotrieve]
21. Turnipseed WD, Kennell TW, Turski PA, Acher CW, Hoch JR. Magnetic resonance angiography and duplex imaging: noninvasive tests for selecting symptomatic carotid endarterectomy candidates. Surgery. 1993;114:643-648. [Medline] [Order article via Infotrieve]
22. Mittl RL, Broderick M, Carpenter JP, Goldberg HI, Listerud J, Mishkin MM, Berkowitz HD, Atlas SW. Blinded-reader comparison of magnetic resonance angiography and duplex ultrasonography for carotid artery bifurcation stenosis. Stroke. 1994;25:4-10. [Abstract]
23. Gray R, Begg CB, Greenes RA. Construction of receiver operating characteristic curves when disease verification is subject to selection bias. Med Decis Making. 1984;4:151-164.
24. Ricotta JJ, Holen J, Schenk E, Plassche W, Green RM, Gramiak R, DeWeese JA. Is routine angiography necessary prior to carotid endarterectomy? J Vasc Surg. 1984;1:96-102. [Medline] [Order article via Infotrieve]
25. Goodson SF, Flanigan P, Bishara RA, Schuler JJ, Kikta MJ, Meyer JP. Can carotid duplex scanning supplant arteriography in patients with focal carotid territory symptoms? J Vasc Surg. 1987;5:551-557. [Medline] [Order article via Infotrieve]
26. Geuder JW, Lamparello PJ, Riles TS, Giangola G, Imparato AM. Is duplex scanning sufficient evaluation before carotid endarterectomy. J Vasc Surg. 1989;9:193-201. [Medline] [Order article via Infotrieve]
27. Gertler JP, Cambria RP, Kistler JP, Geller SC, MacDonald NR, Brewster DC, Abbott WM. Carotid surgery without arteriography: noninvasive selection of patients. Ann Vasc Surg. 1991;5:253-256. [Medline] [Order article via Infotrieve]
28. Polak JF, Kalina P, Donaldson MC, O'Leary DH, Whittemore AD, Mannick JA. Carotid endarterectomy: preoperative evaluation of candidates with combined Doppler sonography and MR angiography: work in progress. Radiology. 1993;186:333-338. [Abstract/Free Full Text]
29. McKittrick JE, Cisek PL, Pojunas KW, Blum GM, Ortgiesen P, Lim RA. Are both color-flow duplex scanning and cerebral arteriography required prior to carotid endarterectomy? Ann Vasc Surg. 1993;7:311-316. [Medline] [Order article via Infotrieve]
30. Dawson DL, Zierler RE, Strandness DE, Clowes AW, Kohler TR. The role of duplex scanning and arteriography before carotid endarterectomy: a prospective study. J Vasc Surg. 1993;18:673-680. [Medline] [Order article via Infotrieve]
31. Steinke W, Kloetzsch C, Hennerici M. Carotid artery disease assessed by color doppler flow imaging: correlation with standard doppler sonography and angiography. AJNR. 1990;11:259-266. [Abstract]
32. Londrey GL, Spadone DP, Hodgson KJ, Ramsey DE, Barkmeier LD, Sumner DS. Does color-flow imaging improve the accuracy of duplex carotid evaluation? J Vasc Surg. 1991;13:659-663. [Medline] [Order article via Infotrieve]
33. Hallam MJ, Reid JM, Cooperberg PL. Color-flow doppler and conventional duplex scanning of the carotid bifurcation: prospective, double-blind, correlative study. AJR. 1989;152:1101-1105. [Abstract/Free Full Text]
34. Erickson SJ, Mewissen MW, Foley WD, Lawson TL, Middleton WD, Quiroz FA, Macrander SJ, Lipchik EO. Stenosis of the internal carotid artery: assessment using color doppler imaging compared with angiography. AJR. 1989;152:1299-1305. [Abstract/Free Full Text]
35. Young N, Soo YS, Fischer P. Comparison of duplex ultrasound with angiography in assessment of carotid bifurcation disease. Australas Radiol. 1992;36:54-58. [Medline] [Order article via Infotrieve]
36. McCrory DC, Goldstein LB, Samsa GP, Oddone EZ, Landsman PB, Moore WS, Matchar DB. Predicting complications of carotid endarterectomy. Stroke. 1993;24:1285-1291. [Abstract/Free Full Text]
37. Huston J, Lewis BD, Wiebers DO, Meyer FB, Riederer SJ, Weaver AL. Carotid artery: prospective blinded comparison of two-dimensional time-of-flight MR angiography with conventional angiography and duplex US. Radiology. 1993;186:339-344. [Abstract/Free Full Text]
38. Lustgarten JH, Solomon RA, Quest DO, Khanjdi AG, Mohr JP. Carotid endarterectomy after noninvasive evaluation by duplex ultrasonography and magnetic resonance angiography. Neurosurgery. 1994;34:612-618. [Medline] [Order article via Infotrieve]
39. Riles TS, Eidelman EM, Litt AW, Pinto RS, Oldford F, Schwartzenberg GWST. Comparison of magnetic resonance angiography, conventional angiography, and duplex scanning. Stroke. 1992;23:341-346. [Abstract/Free Full Text]
40. Patel MR, Klufas RA, Kim D, Edelman RR, Kent KC. MR angiography of the carotid bifurcation: artifacts and limitations. AJR. 1994;162:1431-1437. [Abstract/Free Full Text]
41. Davies KN, Humphrey PR. Complications of cerebral angiography in patients with symptomatic carotid territory ischaemia screened by carotid ultrasound. J Neurol Neurosurg Psychiatry. 1993;56:967-972. [Abstract]
42. Mayberg MR, Wilson E, Yatsu F, Weiss DG, Messina L, Hershey LA, Colling C, Eskridge J, Deykin D, Winn R. Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis. JAMA. 1991;266:3289-3294. [Abstract]
43. Hankey GJ, Warlow CP, Sellar RJ. Cerebral angiographic risk in mild cerebrovascular disease. Stroke. 1990;21:209-222. [Abstract/Free Full Text]
44. Grzyska U, Freitag J, Zeumer H. Selective cerebral intraarterial DSA: complication rate and control of risk factors. Neuroradiology. 1990;32:296-299. [Medline] [Order article via Infotrieve]
45. Waugh JR, Sacharias N. Arteriographic complications in the DSA era. Radiology. 1992;182:243-246. [Abstract/Free Full Text]
46. Dick MT, Cape CA. Complications of duplex carotid imaging. Angiology. 1994;45:235-238.
47. Friedman SG. Transient ischemic attacks resulting from carotid duplex imaging. Surgery. 1990;107:153-155. [Medline] [Order article via Infotrieve]
48. Rosario JA, Hachinski VC, Lee DH, Fox AJ. Adverse reactions to duplex scanning. Lancet. 1987;2:1023. Letter.
49. Masaryk TJ, Obuchowski NA. Noninvasive carotid imaging: caveat emptor. Radiology. 1993;186:325-331.[Free Full Text]

This article has been cited by other articles:

				E. S. Roach, M. R. Golomb, R. Adams, J. Biller, S. Daniels, G. deVeber, D. Ferriero, B. V. Jones, F. J. Kirkham, R. M. Scott, et al. Management of Stroke in Infants and Children: A Scientific Statement From a Special Writing Group of the American Heart Association Stroke Council and the Council on Cardiovascular Disease in the Young Stroke, September 1, 2008; 39(9): 2644 - 2691. [Abstract] [Full Text] [PDF]

				L. Saba and G. Mallarini MDCTA of Carotid Plaque Degree of Stenosis: Evaluation of Interobserver Agreement Am. J. Roentgenol., January 1, 2008; 190(1): W41 - W46. [Abstract] [Full Text] [PDF]

				L. Saba, G. Caddeo, R. Sanfilippo, R. Montisci, and G. Mallarini CT and Ultrasound in the Study of Ulcerated Carotid Plaque Compared with Surgical Results: Potentialities and Advantages of Multidetector Row CT Angiography AJNR Am. J. Neuroradiol., June 1, 2007; 28(6): 1061 - 1066. [Abstract] [Full Text] [PDF]

				J. R. Davies, J. H. Rudd, and P. L. Weissberg Molecular and Metabolic Imaging of Atherosclerosis J. Nucl. Med., November 1, 2004; 45(11): 1898 - 1907. [Abstract] [Full Text] [PDF]

				J Ellul, P Talelli, G Terzis, A Chrysanthopoulou, G Gioldasis, and T Papapetropoulos Is the common carotid artery intima-media thickness associated with functional outcome after acute ischaemic stroke? J. Neurol. Neurosurg. Psychiatry, August 1, 2004; 75(8): 1197 - 1199. [Abstract] [Full Text] [PDF]

				J. M. U-King-Im, R. A. Trivedi, M. J. Graves, N. J. Higgins, J. J. Cross, B. D. Tom, W. Hollingworth, H. Eales, E. A. Warburton, P. J. Kirkpatrick, et al. Contrast-enhanced MR angiography for carotid disease: Diagnostic and potential clinical impact Neurology, April 27, 2004; 62(8): 1282 - 1290. [Abstract] [Full Text] [PDF]

				I. Borisch, M. Horn, B. Butz, N. Zorger, B. Draganski, T. Hoelscher, U. Bogdahn, and J. Link Preoperative Evaluation of Carotid Artery Stenosis: Comparison of Contrast-Enhanced MR Angiography and Duplex Sonography with Digital Subtraction Angiography AJNR Am. J. Neuroradiol., June 1, 2003; 24(6): 1117 - 1122. [Abstract] [Full Text] [PDF]

				P. J. Nederkoorn, Y. van der Graaf, and M.G. M. Hunink Duplex Ultrasound and Magnetic Resonance Angiography Compared With Digital Subtraction Angiography in Carotid Artery Stenosis: A Systematic Review Stroke, May 1, 2003; 34(5): 1324 - 1331. [Abstract] [Full Text] [PDF]

				F. Bilora, V. Boccioletti, F. Petrobelli, and A. Girolami Atherosclerosis and Secondary Deep Vein Thrombosis: A Difficult Correlation Clinical and Applied Thrombosis/Hemostasis, April 1, 2003; 9(2): 121 - 124. [Abstract] [PDF]

				G. Schulte-Altedorneburg, D. W. Droste, S. Felszeghy, L. Csiba, V. Popa, K. Hegedus, J. Kollar, L. Modis, and E. B. Ringelstein Detection of Carotid Artery Stenosis by In Vivo Duplex Ultrasound: Correlation With Planimetric Measurements of the Corresponding Postmortem Specimens Stroke, October 1, 2002; 33(10): 2402 - 2407. [Abstract] [Full Text] [PDF]

				P. J. Nederkoorn, W. P.Th.M. Mali, B. C. Eikelboom, O. E.H. Elgersma, E. Buskens, M.G. M. Hunink, L. J. Kappelle, P. C. Buijs, A. F.J. Wust, A. van der Lugt, et al. Preoperative Diagnosis of Carotid Artery Stenosis: Accuracy of Noninvasive Testing Stroke, August 1, 2002; 33(8): 2003 - 2008. [Abstract] [Full Text] [PDF]

				H. J.M. Barnett, H. E. Meldrum, and M. Eliasziw The appropriate use of carotid endarterectomy Can. Med. Assoc. J., April 1, 2002; 166(9): 1169 - 1179. [Abstract] [Full Text] [PDF]

				K. Logason, S. Karacagil, H.-G. Hardemark, A. Bostrom, A. Hellberg, and C. Ljungman Carotid Artery Endarterectomy Solely Based on Duplex Scan Findings Vascular and Endovascular Surgery, January 1, 2002; 36(1): 9 - 15. [Abstract] [PDF]

				T. J. Tegos, K. J. Kalomiris, M. M. Sabetai, E. Kalodiki, and A. N. Nicolaides Significance of Sonographic Tissue and Surface Characteristics of Carotid Plaques AJNR Am. J. Neuroradiol., September 1, 2001; 22(8): 1605 - 1612. [Abstract] [Full Text] [PDF]

				M. Vergara, E. G. Grant, S. M. El Saden, and G. M. Hathout Duplex US for the Estimation of Internal Carotid Stenosis Drs Grant and colleagues respond: Radiology, May 1, 2001; 219(2): 575 - 577. [Full Text]

				G. Manca, G. Parenti, R. Bellina, G. Boni, M. Grosso, W. Bernini, C. Palombo, M. Paterni, G. Pelosi, M. Lanza, et al. 111In Platelet Scintigraphy for the Noninvasive Detection of Carotid Plaque Thrombosis Stroke, March 1, 2001; 32(3): 719 - 727. [Abstract] [Full Text] [PDF]

				M. Koga, K. Kimura, K. Minematsu, and T. Yamaguchi Diagnosis of Internal Carotid Artery Stenosis Greater than 70% with Power Doppler Duplex Sonography AJNR Am. J. Neuroradiol., February 1, 2001; 22(2): 413 - 417. [Abstract] [Full Text] [PDF]

				G. J. Jager, I. S. Kritiansen, F. A. McAlister, S. E. Straus, G. H. Guyatt, and R. B. Haynes Helping Patients Integrate Research Evidence JAMA, November 22, 2000; 284(20): 2594 - 2595. [Full Text] [PDF]

				J. M. Serfaty, P. Chirossel, J. M. Chevallier, R. Ecochard, J. C. Froment, and P. C. Douek Accuracy of Three-Dimensional Gadolinium-Enhanced MR Angiography in the Assessment of Extracranial Carotid Artery Disease Am. J. Roentgenol., August 1, 2000; 175(2): 455 - 463. [Abstract] [Full Text] [PDF]

				F. A. McAlister, S. E. Straus, G. H. Guyatt, R. B. Haynes, and for the Evidence-Based Medicine Working Group Users' Guides to the Medical Literature: XX. Integrating Research Evidence With the Care of the Individual Patient JAMA, June 7, 2000; 283(21): 2829 - 2836. [Abstract] [Full Text] [PDF]

				J. Dix and J. Skrocki Evaluation of Carotid Stenosis by Angiography: Potential Bias toward Overestimated Measurements Introduced by Prior Interpretation of Doppler Sonograms AJNR Am. J. Neuroradiol., April 1, 2000; 21(4): 639 - 642. [Abstract] [Full Text]

M. M. Horrow, J. Stassi, A. Shurman, J. D. Brody, C. L. Kirby, and H. K. Rosenberg The Limitations of Carotid Sonography: Interpretive and Technology-Related Errors Am. J. Roentgenol., January 1, 2000; 174(1): 189 - 194. [Abstract] [Full Text] [PDF]