(Stroke. 1995;26:1747-1752.)
© 1995 American Heart Association, Inc.
Articles |
From the Departments of Epidemiology and Biostatistics (M.E.), Clinical Neurological Sciences (M.E., A.J.F., H.J.M.B.), and Diagnostic Radiology (R.N.R., A.J.F.), University of Western Ontario; The John P. Robarts Research Institute (M.E., H.J.M.B.), London, Ontario; and the Departments of Clinical Epidemiology and Biostatistics and of Medicine, McMaster University, Hamilton, Ontario (R.B.H.), Canada.
Correspondence to H.J.M. Barnett, The John P. Robarts Research Institute, 100 Perth Dr, London, Ontario, Canada, N6A 5K8.
| Abstract |
|---|
|
|
|---|
Methods A total of 1011 symptomatic carotid bifurcations were studied in patients from the North American Symptomatic Carotid Endarterectomy Trial (NASCET). Given that all patients were considered for entry into the trial, the chance of a verification bias affecting the analyses was minimized. The ultrasonographic data consisted of peak systolic velocities and frequency changes from both the internal and common carotid arteries. Angiographic stenosis was calculated as in NASCET. Receiver operating characteristic (ROC) curves were constructed from the ultrasonographic data for the detection of 70% or greater stenosis on the basis of an angiographic assessment. Kaplan-Meier stroke-free survival curves were used to predict the risk of stroke.
Results The areas under the ROC curves ranged from 0.74 to 0.75 (95% confidence interval [CI], 0.69 to 0.79). The sensitivities and specificities ranged from 0.65 to 0.71. The risk of stroke at 18 months declined sharply as the degree of angiographically defined stenosis declined from 99% to 70%. No pattern of decline was apparent on the basis of the ultrasonographic data.
Conclusions The results indicate that the accuracy of ultrasonography is moderate when flow parameters are used to assess the degree of stenosis. Ultrasonography should be used as a screening tool to exclude patients with no carotid artery disease from further testing. Conventional angiography remains an essential investigation before assigning the risk of stroke and deciding appropriate treatment for extracranial carotid artery disease.
Key Words: angiography carotid arteries carotid endarterectomy ultrasonics
| Introduction |
|---|
|
|
|---|
The announcement of significant surgical benefits has yielded a
flurry of publications aimed at determining ultrasonographic criteria
for identifying an angiographic carotid artery stenosis of 70%
and above.3 4 5 6 7 Although there is clearly a need to safely
identify those symptomatic patients with
70%
stenosis and offer them surgical therapy, the lack of
uniformity among the reported criteria casts doubt over the ability of
ultrasonography to accurately pinpoint a 70% stenosis.
Further uncertainties about ultrasonography have been perpetuated by the use of unsuitable study designs in assessing accuracy. All of the reported studies prescreened patients with ultrasonography. Those who were suspected of having severe stenosis on the basis of ultrasonographic findings alone proceeded to angiography as a prelude to carotid endarterectomy. Patients considered to have lesser degrees of stenosis did not have their disease status verified by angiographic investigations. Given that the process of selecting patients for angiographic verification depended on the results of the ultrasonographic findings, the resulting estimates of accuracy are seriously biased. Specifically, the reported sensitivities (true positive rates) and specificities (true negative rates) may be erroneously inflated.8 Ransohoff and Feinstein9 have described this phenomenon as workup bias (or verification bias). Also, the sample sizes used in the studies were small, and the distribution of angiographic stenoses in the samples was rarely reported.
Data collected from the first 3 years of NASCET (1988 through 1991)
provide an ideal opportunity to examine the accuracy of ultrasonography
in detecting a
70% stenosis of the ICA. Before the
termination of the first part of NASCET in February 1991, all
symptomatic patients were considered for entry into the
trial. In accordance with study protocol, patients were entered into
the trial only if the participating center determined the
stenosis to be 30% to 99% by angiography. Adherence to the
protocol has assured a broad distribution of stenoses in the
sample of patients and has minimized the chance of a verification bias
from entering into the analyses. The clinical outcome data from
the trial also allow the assessment of accuracy in a unique manner. One
is able to determine how precisely the ultrasonographic criteria are
able to predict the patients' future risk of stroke and the beneficial
effects of endarterectomy compared with
angiography.
| Subjects and Methods |
|---|
|
|
|---|
Neuroimaging
Biplane (anterior-posterior, lateral, and/or oblique) selective
carotid angiography was used for the assessment of the degree of
stenosis in all patients at entry. Ultrasonography was
performed concurrently to the angiogram but was not used in the
decision-making process for entering patients into the study. Its
purpose was to establish a baseline study for comparison with
subsequent follow-up studies. Ultrasonographic examinations included
both longitudinal and transverse views of the carotid bifurcation and
the relevant stenosis. Also included were Doppler readings
from the site of maximal stenosis, the normal ICA, the CCA, and
the external carotid artery. The vast majority of the transducers used
were in the 5-MHz range, and almost all the recordings were
made at a reasonably standard angle of 60°. Hard copies of all
angiograms and ultrasonographic scans were sent to the central office.
Videotapes of the ultrasonographic scans were not used because of the
different formats available and the length of time required to review
large numbers of examinations. Angiograms were reviewed blindly by the
principal neuroradiologist (A.J.F.), and the degree of stenosis
was measured by the strict criteria advocated in NASCET.11
The degrees of interobserver and intraobserver reliability for
measuring percentage of stenosis were high (ICC: 0.93 and 0.95,
respectively).12 These figures provide reassurance for
considering angiography to be the "comparative gold standard" in
the present study. The ultrasonographic scans were read
independently of the angiograms by the principal neurosonographer
(R.N.R.) and without knowledge of the angiographic results. Data
recorded from the ultrasonographic hard copies consisted of the
ICPSV or ICPFC and the CCPSV or CCPFC. A subset of 54 scans reporting
peak systolic velocity and 48 scans reporting peak frequency changes
were selected to estimate the degrees of interobserver and
intraobserver reliability. The results indicate high reliability
(ICPSV, ICC: 0.97 and 0.98, respectively; CCPSV, ICC: 0.92 and 0.88,
respectively; ICPFC, ICC: 0.98 and 0.97, respectively; CCPFC, ICC: 0.79
and 0.76, respectively).
Statistical Analysis
ROC curves were constructed to assess the accuracy of
ultrasonographic criteria for detecting a
70% angiographically
defined stenosis. A ROC curve is a graph that displays the
relationship between the sensitivity and specificity of a
diagnostic tool across a spectrum of cutpoints that could
be used to classify patients as diseased or nondiseased. The overall
accuracy of a diagnostic tool is expressed in terms of the
area under the ROC curve, ranging from 0.5 (poor) to 1.0 (perfect). The
LABROC4 computer program developed by Metz13
was used to generate the fitted curves and the areas under the curves.
Corresponding two-sided 95% CIs for the areas were also calculated.
Kaplan-Meier survival curves were used to estimate the risk of
ipsilateral stroke at 18 months for each decile category of
stenosis, as defined by angiography and ultrasonography.
The ultrasonographic decile categories of stenosis were defined
on the basis of the ICPSV, ICPFC, and ICA/CCA ratio (Table 1
). The ultrasonographic cutpoints appearing in Table 1
were selected to be representative of the ones reported
in the literature. For example, an ICPSV cutpoint of 250 cm/s for a
70% angiographic stenosis is considered to be
representative of the reported ICPSV values ranging
from 175 to 325 cm/s,3 4 5 6 7 14 15 whereas an ICA/CCA ratio
cutpoint of 3.0 is representative of values ranging
from 3.0 to 4.0.4 5 16 Cutpoints for higher degrees of
stenosis were interpolated from published scattergrams that
showed the relationship between ultrasonographic velocities, ratios,
and the degree of angiographically defined carotid
stenosis.3 7 14 16 Although peak systolic
velocities and frequency changes are more commonly used, the peak
systolic ICA/CCA ratio has also been included, since it has appeared in
the literature on several occasions.3 4 14 16 17
|
| Results |
|---|
|
|
|---|
|
ROC Curves
The ROC curves, illustrating the accuracy with which
ultrasonographic criteria are able to detect an angiographic
stenosis of 70% or greater, are shown in Fig 2
.
The four curves are virtually identical. The areas under the ROC curve
range from 0.74 to 0.75 (Table 2
). The
sensitivity and specificity at the curves' point of inflection are
also given in Table 2
. Further analyses were carried out to
determine if the size of the participating center or the year in which
the imaging studies were carried out affected the results. Size was
determined on the basis of the number of patients that a participating
center entered into the study: small (<20 patients), medium (21 to 35
patients), and large (>35 patients). Three calendar years were
considered: 1988, 1989, and 1990 to 1991. There were no apparent
differences among the areas under the ROC curve calculated for centers
of different sizes or across the three consecutive time periods.
Overall, the results from the ROC curve analyses indicate that
the accuracy of ultrasonography is moderate when peak systolic
velocities and frequency changes from both the ICA and CCA are used to
assess the degree of stenosis.
|
|
Prognostic Predictions
Of the 1011 patients, 484 had angiographically defined
stenosis of
70% at baseline. Since NASCET is an ongoing
trial that is still entering patients, the protocol allows only
disclosure of the follow-up data on severe (
70% stenosis)
patients. Therefore, this subset forms the sample for examining the
patients' prognoses by decile category of stenosis. The risks
of ipsilateral stroke at 18 months are given in Table 3
.
Ipsilateral strokes are any strokes that occur on the same side as the
artery for which the patient was randomized into the trial. It is
observed that the risk of stroke for medically treated patients
increases considerably as the degree of angiographically defined
stenosis increases by decile category. Given the relatively
constant risk for the surgically treated patients, the net result is a
declining benefit of surgery that corresponds to a decline in the
percentage of stenosis. These relationships are not observed
for the ultrasonographic criteria based on absolute measures of ICPSV
and ICPFC. Paradoxically, the greatest risk of stroke occurs among
patients who have been diagnosed by ultrasonography as having <70%
stenosis (Table 3
). Of the 184 patients in this category, 44
(24%) were deemed to have
90% stenosis when measured by
angiography. This misclassification is reflected in the elevated value
for the mean angiographic stenosis for this decile category. A
similar result is observed in the <70% decile category when
ultrasonographic ratios were used to define the severity of
stenosis (Table 3
). Despite this, the risks of stroke and
benefits of surgery derived using ultrasonographic ratios have a
similar decline as was observed for angiographically defined
stenosis (Table 3
).
|
| Discussion |
|---|
|
|
|---|
An important, but often overlooked, reason for this divergence in results is the manner by which patients were selected for angiographic verification of carotid stenosis,9 23 usually as a prelude to carotid endarterectomy. One study reported that of the 1873 patients who had carotid ultrasonography studies, only 96 (5.1%) were subjected to subsequent angiographic verification.3 Since only the angiographically verified cases were used in the ROC analyses, the authors reported that the estimates of accuracy were expectedly biased upward. After correcting the analyses for verification bias, the areas under the ROC curves decreased substantially.3 Specifically, the area under the ROC curve corresponding to ICPSV dropped from 0.94 to 0.78. Similarly, the ROC area corresponding to ICPSV/CCPSV dropped from 0.91 to 0.66. The corrected figures reported in this previous study are consistent with our results.
The use of ultrasonography as the sole diagnostic test for
the selection of patients for carotid
endarterectomy has been widely
discussed.24 25 26 27 28 29 30 The results of our study indicate that
critical errors can occur. The number of false negatives for
70%
stenosis is disconcerting. Ultrasonography will fail to
identify some near occlusions, mistaking them for mild to moderate
lesions because of the lack of turbulence resulting from the severely
limited flow, or alternatively misread them as occlusions. A near
occlusion mistakenly interpreted as an occlusion may deny the
opportunity for endarterectomy to a patient for
whom it is most urgently needed. The use of color-flow Doppler to
increase the accuracy of placing the Doppler cursor for duplex
ultrasonography appears to slightly increase the accuracy of
stenosis determination31 32 but not by an amount
that is likely to significantly alter the findings of this study.
Color-flow Doppler is useful, however, in differentiating near
occlusions from occlusions, since a string of color may often be seen
in a nearly occluded artery.33 34
As shown by the results of the present study, there is a sharp
decline in the risk of stroke and benefit from
endarterectomy as the degree of angiographically
defined stenosis declines from 99% to 70% (Table 3
). This
declining benefit underscores the importance of identifying precisely
the degree of stenosis. Careful measurement is becoming more
universal as a prelude to decision making about
endarterectomy than ever before.
Even if ultrasonography becomes more refined and better correlated with the angiographic measurements used in NASCET, there still remain arterial features other than degree of stenosis that escape identification but have prognostic importance. Ultrasonography may not identify the presence of intraluminal thrombus, it is inadequate for the identification of ulceration, and it will not reveal intracranial carotid or middle cerebral artery stenosis or occlusion of the middle cerebral trunk. It fails to visualize intracranial vascular anomalies and intracranial aneurysms.35 36 Although none of these individual conditions occur in large numbers, the aggregate is sufficient to deter most neurologists and surgeons from accepting ultrasonography alone as a prelude to carotid endarterectomy.
Possible errors in identifying carotid lesions using Doppler sonography can be corrected by reviewing MR angiography images of the carotid bifurcations.21 28 37 38 Although MR angiography has major advantages in being able to display the carotid bifurcation and intracranial arteries in a format similar to that of conventional angiography, it remains imperfect in giving precise measurements of stenosis. At present, the degree of stenosis is overestimated, and the problem of signal void related to turbulence has not been resolved.39 40
Some have argued that angiography carries too much risk.28 41 This would be an acceptable deterrent for its use if endarterectomy was free of risk and if perfection in estimating stenosis was achievable by a combination of ultrasonography and MR angiography. Recent randomized trials of carotid endarterectomy in symptomatic patients reported perioperative rates of stroke (of any severity) and death between 5.8% and 7.5%.1 2 42 In comparison, a literature review reporting on eight early prospective studies (spanning the years 1973 through 1984) has estimated that the overall risk of any stroke due to angiography was 1.0% (95% CI, 0.6 to 1.5) and the overall rate of mortality was 0.06% (95% CI, 0.0 to 0.18).43 Among the 2320 patients currently enrolled in NASCET, 15 patients (0.65%) had postangiographic strokes (H.J.M.B., E.M., unpublished data, 1995). Given that NASCET excludes patients with major disabling strokes from entry into the trial, and because the published prospective studies of stroke complications from angiography consistently reported a 5-to-1 ratio of nondisabling ischemic events to disabling strokes, an additional three events (representing disabling strokes) may be added to the angiographic complication rate of potential NASCET patients. This increases the number of strokes from 15 to 18, thus yielding a stroke rate from angiography of 0.78%.
These figures may be higher than can be presently achieved. Over the past 10 years, selective cerebral angiography has undergone profound changes. There have been important advances in catheter design, nonionic contrast agents are routinely used, and high-resolution digital systems have been introduced that not only reduce contrast medium load but also shorten examination time. All of these improvements aim to reduce the risk of angiographic complications. Two recently published studies using digital subtraction angiography estimated the risk of any stroke to be between 0.09% and 0.3%.44 45 While angiography still carries a minimal but definite risk of complication, at the same time ultrasonography cannot claim to be free of risk. In the past 25 years, four neurological and two cardiac adverse events have been reported to be associated with the performance of carotid ultrasonography.46 47 48
On the basis of our results from analyzing data from 50 academic centers across North America, we recommend that ultrasonography be used as a screening tool to exclude patients with no carotid artery disease from further testing. Patients suspected of having disease, and who are suitable for endarterectomy, should have confirmatory angiography before a treatment decision is made. Furthermore, those patients who have 30% to 69% diameter reduction by the method of measurement used in NASCET, and who show progression on follow-up ultrasonographic examination, should have angiography if they appear to have progressed to 70% or beyond before deciding on further treatment. The immediate elimination of conventional angiography in favor of combined Doppler and MR angiography is premature.18 22 49 Ultrasonography (using peak systolic velocities and frequency changes from both the ICA and CCA) is less capable than angiography of identifying with comparable precision a stenosis that angiography recognizes to be at or above 70%. Conventional angiography remains an essential investigation before carotid endarterectomy and while everyone awaits the final perfection and widespread availability of both color-flow Doppler and MR angiography.
| Selected Abbreviations and Acronyms |
|---|
|
| Acknowledgments |
|---|
Received February 7, 1995; revision received May 4, 1995; accepted May 25, 1995.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
E. S. Roach, M. R. Golomb, R. Adams, J. Biller, S. Daniels, G. deVeber, D. Ferriero, B. V. Jones, F. J. Kirkham, R. M. Scott, et al. Management of Stroke in Infants and Children: A Scientific Statement From a Special Writing Group of the American Heart Association Stroke Council and the Council on Cardiovascular Disease in the Young Stroke, September 1, 2008; 39(9): 2644 - 2691. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Saba and G. Mallarini MDCTA of Carotid Plaque Degree of Stenosis: Evaluation of Interobserver Agreement Am. J. Roentgenol., January 1, 2008; 190(1): W41 - W46. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. Saba, G. Caddeo, R. Sanfilippo, R. Montisci, and G. Mallarini CT and Ultrasound in the Study of Ulcerated Carotid Plaque Compared with Surgical Results: Potentialities and Advantages of Multidetector Row CT Angiography AJNR Am. J. Neuroradiol., June 1, 2007; 28(6): 1061 - 1066. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. R. Davies, J. H. Rudd, and P. L. Weissberg Molecular and Metabolic Imaging of Atherosclerosis J. Nucl. Med., November 1, 2004; 45(11): 1898 - 1907. [Abstract] [Full Text] [PDF] |
||||
![]() |
J Ellul, P Talelli, G Terzis, A Chrysanthopoulou, G Gioldasis, and T Papapetropoulos Is the common carotid artery intima-media thickness associated with functional outcome after acute ischaemic stroke? J. Neurol. Neurosurg. Psychiatry, August 1, 2004; 75(8): 1197 - 1199. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. M. U-King-Im, R. A. Trivedi, M. J. Graves, N. J. Higgins, J. J. Cross, B. D. Tom, W. Hollingworth, H. Eales, E. A. Warburton, P. J. Kirkpatrick, et al. Contrast-enhanced MR angiography for carotid disease: Diagnostic and potential clinical impact Neurology, April 27, 2004; 62(8): 1282 - 1290. [Abstract] [Full Text] [PDF] |
||||
![]() |
I. Borisch, M. Horn, B. Butz, N. Zorger, B. Draganski, T. Hoelscher, U. Bogdahn, and J. Link Preoperative Evaluation of Carotid Artery Stenosis: Comparison of Contrast-Enhanced MR Angiography and Duplex Sonography with Digital Subtraction Angiography AJNR Am. J. Neuroradiol., June 1, 2003; 24(6): 1117 - 1122. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J. Nederkoorn, Y. van der Graaf, and M.G. M. Hunink Duplex Ultrasound and Magnetic Resonance Angiography Compared With Digital Subtraction Angiography in Carotid Artery Stenosis: A Systematic Review Stroke, May 1, 2003; 34(5): 1324 - 1331. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. Bilora, V. Boccioletti, F. Petrobelli, and A. Girolami Atherosclerosis and Secondary Deep Vein Thrombosis: A Difficult Correlation Clinical and Applied Thrombosis/Hemostasis, April 1, 2003; 9(2): 121 - 124. [Abstract] [PDF] |
||||
![]() |
G. Schulte-Altedorneburg, D. W. Droste, S. Felszeghy, L. Csiba, V. Popa, K. Hegedus, J. Kollar, L. Modis, and E. B. Ringelstein Detection of Carotid Artery Stenosis by In Vivo Duplex Ultrasound: Correlation With Planimetric Measurements of the Corresponding Postmortem Specimens Stroke, October 1, 2002; 33(10): 2402 - 2407. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. J. Nederkoorn, W. P.Th.M. Mali, B. C. Eikelboom, O. E.H. Elgersma, E. Buskens, M.G. M. Hunink, L. J. Kappelle, P. C. Buijs, A. F.J. Wust, A. van der Lugt, et al. Preoperative Diagnosis of Carotid Artery Stenosis: Accuracy of Noninvasive Testing Stroke, August 1, 2002; 33(8): 2003 - 2008. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. J.M. Barnett, H. E. Meldrum, and M. Eliasziw The appropriate use of carotid endarterectomy Can. Med. Assoc. J., April 1, 2002; 166(9): 1169 - 1179. [Abstract] [Full Text] [PDF] |
||||
![]() |
K. Logason, S. Karacagil, H.-G. Hardemark, A. Bostrom, A. Hellberg, and C. Ljungman Carotid Artery Endarterectomy Solely Based on Duplex Scan Findings Vascular and Endovascular Surgery, January 1, 2002; 36(1): 9 - 15. [Abstract] [PDF] |
||||
![]() |
T. J. Tegos, K. J. Kalomiris, M. M. Sabetai, E. Kalodiki, and A. N. Nicolaides Significance of Sonographic Tissue and Surface Characteristics of Carotid Plaques AJNR Am. J. Neuroradiol., September 1, 2001; 22(8): 1605 - 1612. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Vergara, E. G. Grant, S. M. El Saden, and G. M. Hathout Duplex US for the Estimation of Internal Carotid Stenosis Drs Grant and colleagues respond: Radiology, May 1, 2001; 219(2): 575 - 577. [Full Text] |
||||
![]() |
G. Manca, G. Parenti, R. Bellina, G. Boni, M. Grosso, W. Bernini, C. Palombo, M. Paterni, G. Pelosi, M. Lanza, et al. 111In Platelet Scintigraphy for the Noninvasive Detection of Carotid Plaque Thrombosis Stroke, March 1, 2001; 32(3): 719 - 727. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. Koga, K. Kimura, K. Minematsu, and T. Yamaguchi Diagnosis of Internal Carotid Artery Stenosis Greater than 70% with Power Doppler Duplex Sonography AJNR Am. J. Neuroradiol., February 1, 2001; 22(2): 413 - 417. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. J. Jager, I. S. Kritiansen, F. A. McAlister, S. E. Straus, G. H. Guyatt, and R. B. Haynes Helping Patients Integrate Research Evidence JAMA, November 22, 2000; 284(20): 2594 - 2595. [Full Text] [PDF] |
||||
![]() |
J. M. Serfaty, P. Chirossel, J. M. Chevallier, R. Ecochard, J. C. Froment, and P. C. Douek Accuracy of Three-Dimensional Gadolinium-Enhanced MR Angiography in the Assessment of Extracranial Carotid Artery Disease Am. J. Roentgenol., August 1, 2000; 175(2): 455 - 463. [Abstract] [Full Text] [PDF] |
||||
![]() |
F. A. McAlister, S. E. Straus, G. H. Guyatt, R. B. Haynes, and for the Evidence-Based Medicine Working Group Users' Guides to the Medical Literature: XX. Integrating Research Evidence With the Care of the Individual Patient JAMA, June 7, 2000; 283(21): 2829 - 2836. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. Dix and J. Skrocki Evaluation of Carotid Stenosis by Angiography: Potential Bias toward Overestimated Measurements Introduced by Prior Interpretation of Doppler Sonograms AJNR Am. J. Neuroradiol., April 1, 2000; 21(4): 639 - 642. [Abstract] [Full Text] |
||||
![]() |
M. M. Horrow, J. Stassi, A. Shurman, J. D. Brody, C. L. Kirby, and H. K. Rosenberg The Limitations of Carotid Sonography: Interpretive and Technology-Related Errors Am. J. Roentgenol., January 1, 2000; 174(1): 189 - 194. [Abstract] [Full Text] [PDF] |