(Stroke. 2000;31:1444.)
© 2000 American Heart Association, Inc.
Comments, Opinions, and Reviews |
Critical Appraisal of the Design and Reporting of Studies of Imaging and Measurement of Carotid Stenosis
From the Department of Clinical Neurology, Radcliffe Infirmary (P.M.R., S.T.P.), Oxford, UK; and the Department of Clinical Neurosciences, Western General Hospital (J.W., C.P.W.), Edinburgh, UK.
Correspondence to Dr P. Rothwell, Department of Clinical Neurology, Radcliffe Infirmary, Woodstock Road, Oxford OX2 6HE, UK. E-mail peter.rothwell{at}clneuro.ox.ac.uk
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background and Purpose—Several hundred studies have been published over the last few years on imaging and measurement of carotid stenosis. Despite all this research, there is still no consensus about how best to image and measure stenosis. One possible explanation for this is that many of the studies have not been large enough or methodologically sound enough to allow useful conclusions to be drawn. We aimed to assess the design and methods of a random sample of published studies of imaging and measurement of carotid stenosis using 9 simple criteria.
Methods—A formal literature search was performed for studies of imaging and measurement of carotid stenosis. Two subsets were randomly selected for detailed assessment: 20 studies published before 1991 and 20 published between 1993 and 1997 (some years after the initial publication of the ECST and NASCET trials). The criteria used to assess the selected studies were as follows: prospective rather than retrospective study design; patient selection based on a consecutive series or a random sample; adequate detail of study population; adequate detail of imaging techniques; inclusion of all investigations, ie, patients with poor-quality imaging were not excluded; blinded assessment of images; adequate detail of derivation of measurement of stenosis from images or data; adequate data on the reproducibility of measurements of stenosis; and study powered according to a sample-size calculation.
Results—There were many basic methodological deficiencies in both
subsets of studies, with relatively little evidence of improvement with
time. For example, only 33% of studies were prospective, only 45%
studied a consecutive or random selection of patients, and only 38%
reported any data on the reproducibility of measurements. More than
half of the studies satisfied 
4 of the 9 quality criteria. However,
there was considerable variation between studies, with 7 studies
satisfying 
7 criteria and 10 studies satisfying 2. No study was
based on a sample-size calculation. The number of patients studied was
often small, particularly in the more recent studies: median sample
size was 100 in the 1970–1990 studies and 58 in the 1993–1997 studies
(P<0.0001).
Conclusions—The design and reporting of published studies of imaging and measurement of carotid stenosis are poor and have not improved much in recent years. The majority of published studies are not of a sufficient standard to enable the results to be used to inform clinical practice. The utility of future studies could be improved considerably by better adherence to 9 simple methodological guidelines.
Key Words: diagnostic imaging • measurement • carotid stenosis
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The number of carotid endarterectomies being performed in the United States and Europe increased dramatically in the 1970s and early 1980s.1 2 There was a parallel increase in the frequency of carotid angiography1 and an increasing interest in noninvasive imaging. A literature search for studies of noninvasive methods of carotid imaging published between 1977 and 1993 found 568 relevant articles.3 Interest intensified after publication of the first results of the European Carotid Surgery Trial [ECST] and the North American Symptomatic Carotid Endarterectomy Trial [NASCET] in 1991,4 5 which showed that the degree of symptomatic carotid stenosis was a major predictor of risk of ipsilateral ischemic stroke for patients undergoing medical treatment and thus also an indicator of the benefit that could be derived from carotid endarterectomy. This led to a flurry of editorials and reviews on the different methods of measurement of stenosis6 7 8 9 10 11 12 13 14 and many additional studies comparing different imaging techniques. A manual search of Stroke from 1993 to 1998 (P.M.R.) revealed 42 such studies (excluding articles on vessel dissection or plaque morphology).
Carotid endarterectomy rates are rising again,15 16 and many clinicians are keen to use noninvasive methods, such as duplex ultrasound scanning, spiral CT scanning, and magnetic resonance angiography, to select patients for surgery. Conventional arterial angiography has significant morbidity and mortality,17 18 and it is now recognized that early retrospective studies probably underestimated these risks.19 20 A systematic review of the prospective studies of angiography in patients with cerebrovascular disease reported a risk of permanent neurological sequelae of 1% and an overall mortality rate of 0.1%,17 and significantly higher risks have been reported.18 Although the risks of arterial angiography may be lower in some very experienced centers, the procedure is still costly and time consuming, often requiring admission to hospital.21 For these reasons, it has become clear that we need, if possible, to progress to the routine use of noninvasive methods of carotid imaging in the selection of patients for endarterectomy.
The results of the clinical trials that demonstrated that endarterectomy was beneficial for symptomatic carotid stenosis were stratified by use of measurements of stenosis on arterial angiograms. There are no direct data to show that noninvasive methods of imaging can be used in the same way to differentiate between patients who should benefit from endarterectomy and those who may not. It is necessary, therefore, to validate noninvasive methods against conventional angiography. However, although many such studies have been published, the majority are undermined by poor design, inadequate sample sizes, and inappropriate analysis and presentation of data. It is at least partly as a consequence of this that there is still no consensus about how best to image the carotid artery. If noninvasive methods of imaging are to be properly validated and the findings of different studies compared, then a consistent and methodologically sound approach to study design and analysis must be adopted. This is particularly important given the large sample sizes required to accurately define the measurement characteristics of different imaging techniques and the consequent need for systematic reviews of studies of carotid imaging.3 22 23 24
We assessed the design and reporting of a random sample of published studies of imaging and measurement of carotid stenosis using 9 simple criteria. Statistical analysis and presentation of results will be dealt with in a future paper. To assess whether the quality of studies has improved since the publication of the initial results of ECST and NASCET and the consequent realization of the importance of accurate measurement of the degree of carotid stenosis, articles published in 2 specific periods were studied: 1970–1990 (before ECST and NASCET) and 1993–1997 (some years after the publication of the initial results of ECST and NASCET).
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Inclusion Criteria
We aimed to review articles that reported original data derived from imaging the carotid circulation in human subjects, the primary aim of which was either to determine the properties of the measurement of carotid stenosis using 1 imaging modality or to compare measurements of stenosis using 2 or more different imaging modalities. Studies could be confined to conventional angiography or a noninvasive method of imaging or could be a comparison of both. The review was confined to studies published in English-language journals.
Search Strategy
Articles were identified by the following means:
- Articles already known to the authors from previous
studies of the literature.
- A single observer (P.M.R.) identified potential articles using
CD ROM (Cambridge Medline 1970–1997). The search strategy was based on
combinations of the following terms: carotid; stenosis;
angiography; duplex; Doppler; ultrasound; ultrasonography; magnetic
resonance imaging; magnetic resonance angiography; CT angiography.
- The reference lists of all articles known to the authors and of
articles identified from the literature were searched.
Selection of Studies for Review
Given the very large number of published studies, it was not
possible to review all those identified in the literature search.
Therefore, of the studies that fulfilled the inclusion criteria, 20
were randomly selected from each of the 2 publication periods. All of
the articles published in each of the 2 periods were numbered
sequentially (by S.T.P.). Another reviewer (P.M.R.) then used the
random number–generating function in SPSS (version 7.0) to select 20
numbers at random from the range of numbers available for each period.
Forty articles were thus selected for detailed study.
Methodological Criteria
Each article was assessed by 2 reviewers (P.M.R. and S.T.P.).
The number of patients studied was recorded, and 9 methodological
criteria were used to assess the quality of the articles (see below).
Each of the criteria are considered in detail in the Discussion. If the
2 reviewers disagreed in their assessment of an article, they
reassessed the article together until a consensus was reached. The
criteria used to assess the selected studies were as follows:
- Prospective rather than retrospective study design.
- Patient selection based on a consecutive series or a random
sample.
- Adequate detail of study population (age, sex, clinical
presentation, and indications for investigation).
- Adequate detail of imaging techniques (sufficient for the study
to be repeated).
- Inclusion of all investigations, ie, patients with poor-quality
imaging were not excluded.
- Blinded assessment of images.
- Adequate detail of derivation of measurement of
stenosis from images or data (sufficient for the study to be
repeated).
- Adequate data on the reproducibility of measurements of
stenosis (data on either intraobserver agreement or
interobserver agreement were accepted).
- Study powered according to a sample-size calculation.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The electronic literature search identified 486 publications that concerned carotid imaging. Exclusion of studies that did not fit our inclusion criteria left 132 studies. A further 13 studies that were already known to the authors and that fulfilled the inclusion criteria were not identified by the search. A search of the reference lists of these 145 studies revealed an additional 17 studies that fulfilled the inclusion criteria. Of the resulting sample of 162 studies, 93 were published before 1991 and 69 were published between 1993 and 1997. Twenty studies from each period were randomly selected for detailed review (1970–199125 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 and 1993–199745 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 ). No studies by any of the authors were selected, and thus there was no conflict of interest.
The methodological assessments of the studies are shown in Table 1
. There were many methodological
deficiencies in both cohorts, with relatively little evidence of
improvement with time. For example, only 33% of studies were
prospective, only 45% studied a consecutive or random selection of
patients, and only 38% reported any data on the reproducibility of
measurements. More than half of the studies satisfied 4 of the 9
quality criteria. However, there was considerable variation between
studies, with 7 studies satisfying 7 criteria and 10 satisfying 2
(Table 2). The only major improvements in
the 1993–1997 cohort were in the description of the study population
and the description of the method of measurement of stenosis
used for the techniques studied.
|
|
No study was based on a sample-size calculation. Two studies looked at much large numbers of patients than the rest (500 patients40 and 400 patients63 ). However, the sample sizes in the remaining studies were often very small. This was particularly true of the more recent studies; median sample size was 100 in the 1970–1990 studies and 58 in the 1993–1997 studies (P<0.0001). Ten of the 1970–1990 studies had sample sizes of 100 or more compared with only 2 of the more recent studies.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
It is clearly important that effective investigations should be encouraged and ineffective investigations abandoned. The only way in which this will be achieved is by the performance, publication, and dissemination of high-quality and adequately powered clinical studies and systematic reviews. However, the design and reporting of the majority of studies reviewed here did not reach a sufficient standard. Although there is no gold standard with which to assess the quality of a study, and the choice of assessment criteria could be criticized, the 9 criteria could not be said to be unreasonable. The fact that some studies satisfied nearly all the criteria suggests that they were not unrealistic. Each of the methodological criteria that were used in this study is justified below.
Study Design (Criterion 1) and Selection of Patients
(Criterion 2)
Studies should concentrate on patient populations that are
comparable to patients seen in ordinary clinical practice. The answer
to an important question is of little value if asked of the "wrong"
patients. For example, a study of the reproducibility of measurement of
stenosis on angiograms from a group of patients in which the
majority had a carotid stenosis of <30% by the ECST method
(<0 by the NASCET method)26 will tell us very little
about measurement in patients with moderate and severe
stenosis, the group in which variability might affect clinical
decision making. Similarly, a study of intravenous digital
subtraction angiography in predominantly young, fit patients will give
much better results than an identical study in an older, more relevant
population with widespread vascular disease; the causes of inadequate
visualization of the stenosis with intravenous
angiography, such as movement artifact, cardiac failure, and poor
respiratory function, increase in frequency with age and concurrent
disease. The population studied is of particular importance in studies
of the complications of imaging procedures. Studies of the
complications of conventional selective arterial
angiography that concentrated on an elderly population with
symptomatic cerebrovascular disease found much higher
morbidity and mortality than studies in less-selected
populations.18 19
Patients studied should be consecutive or random samples. Selection bias can undermine the generalizability of study results. For example, several of the 1970–1990 studies compared arterial angiography with intravenous digital subtraction angiography. However, most of these were retrospective and confined to a small proportion of patients who, presumably for specific but undeclared reasons, had been imaged by both techniques.25 30 33 36 37 40 41 Perhaps in units that routinely perform intravenous digital angiography, arterial angiography is more likely to be performed in patients in whom intravenous angiography has not been completely adequate. Results from studies of such highly selected patients are unlikely to be generalizable. Only 4 of the 1970–1990 studies were prospective, and only 4 of the retrospective studies selected patients in a consecutive or random fashion. This was somewhat improved in the later cohort, but there were still several retrospective studies with potentially biased selection of cases.
Criterion 3: Was the Study Population Adequately
Described?
It is essential that articles describe the study population. A
study is of little value, no matter how well it is performed, if the
published article does not give sufficient information to allow other
investigators to replicate the work or other clinicians to apply the
results to their own clinical practice. However, few of the 1970–1990
studies reported any clinical data on the patients whose carotid
arteries were imaged. The age, sex, clinical presentation,
and indications for investigation in the patients studied should be the
minimum information reported. This was the one area in which the
1993–1997 studies were much improved.
Criterion 4: Were Sufficient Details of the Imaging Technique
Provided?
Clearly, any study of an imaging technique should give sufficient
technical detail for others to be able to repeat it. This is, in fact,
the one area in which virtually all studies excelled. Indeed, there was
often a striking contrast between the amount of technical detail and
the lack of any other methodological information. Overall, in the 40
studies reviewed, the median number of lines of text detailing the
imaging technique was 20 (range 5 to 60) compared with a median of only
3 lines (range 1 to 20) describing the selection of cases and their
clinical details.
Criterion 5: Were all Investigations Included?
In general, poor-quality investigations should not be excluded
from studies evaluating imaging or measurement of stenosis. If
a patient has been put through an investigation and the result has been
used to inform a clinical decision, it should be good enough to include
in a study. Many of the 1970–1990 studies that compared the accuracy
and reproducibility of measurement of stenosis using
intravenous angiography with that using
arterial angiography excluded intravenous
investigations that were considered
inadequate.30 34 40 41 44 Consequently,
intravenous angiography compared well with conventional
angiography. The results of these studies contrast with 2 studies that
included all investigations, in which adequate views of both carotid
bifurcations were seen in only 26%35 and
42%36 of patients imaged with intravenous
angiography. Similarly, some studies of the reproducibility of
measurement of stenosis on angiograms have been confined to
angiograms selected on the basis of quality.25 26 29
Exclusion of poor-quality investigations undermines the relevance of
these studies to clinical practice.
Criterion 6: Was the Assessment of Images Blinded to Other
Information?
The need for blinding of observers to any information that might
bias their measurements is self-evident. For example, in studies of the
interobserver reproducibility of measurement of the degree of carotid
stenosis with a particular technique, the observers should be
independent and blinded to the findings of the others. Similarly, in
studies comparing measurements made by different techniques, the
observer should be blinded to any information that might lead to
recognition of the fact that 2 investigations are from the same
patient. Despite the importance of blinding, in nearly half the studies
reviewed, it was neither implicit nor stated that observers were
blinded.
Criterion 7: Was the Method of Measurement of Stenosis
Described?
Although it is only in recent years that the disparities between
measurements made by the different methods have been realized, it is
still very surprising that 8 of the 1970–1990 studies did not define
how stenosis had been measured on the
angiograms.30 31 32 35 36 41 42 44 Of those studies that did
give details, there was an even split between the ECST
method25 28 29 34 40 and the NASCET
method.27 33 37 39 43 An exact definition of how the
degree of stenosis is derived is particularly important in
studies of noninvasive methods of imaging. Perhaps not surprisingly
given the increased awareness of the disparities between the different
methods of measurement, only 2 of the 1993–1997 studies failed to
provide an adequate description of the method used.53 61
Of the remaining 1993–1997 studies, 10 used only the NASCET
method,46 49 50 51 54 55 56 58 59 63 4 used only the ECST
method,45 47 48 64 and 4 used both
methods.52 57 60 62
Criterion 8: Were Data on the Reproducibility of Measurements
Reported?
Interobserver agreement in the interpretation of radiological
investigations may be little greater than that expected by chance
alone.65 66 67 For example, reproducibility of measurements
of stenosis on angiograms of the coronary or
peripheral arterial circulations can be very
poor.68 69 There is no reason to assume that measurements
of carotid stenosis by any technique will be less prone to
observer variability. Any study that involves measurement of
stenosis should provide some information about the
reproducibility of the measurements made. Measurements are only likely
to have clinical utility if they are reproducible. However, fewer than
half of the studies reported any reproducibility data.
Criterion 9: Sample Size
Clinical trials often require very large sample sizes to measure
the effectiveness of treatments with sufficient precision to influence
clinical practice.70 The same principles should be applied
to the validation of new methods of imaging. However, it is difficult
to perform randomized controlled trials comparing different imaging
methods. Trials would have to be vast to produce a reliable estimate of
the effect of a method on eventual patient outcome.71 In
practice, it is reasonable for imaging studies to compare a new
technique with an established gold standard in the same group of
patients and then extrapolate the results to estimate the likely
outcome if the old technique were to be replaced by the new technique.
However, although such studies will need much smaller sample sizes than
randomized controlled trials that examine patient outcome, they still
need to have the power to define any differences between the different
techniques with clinically useful precision.
The sample size required will depend on the exact nature of the question being assessed. The simplest question, and the one that generally requires the smallest sample size, is assessment of the sensitivity and specificity of one test to detect a threshold defined by a gold standard, eg, 70% stenosis or complete occlusion as defined on conventional angiography. However, even this requires a large sample size to have clinically useful precision. For example, if we suppose that a population of 600 patients has a 20% prevalence of 70% to 99% carotid stenosis on conventional angiography (ie, 120 cases), and that carotid ultrasound correctly identifies 108 of these, the sensitivity of carotid ultrasound in the detection of severe stenosis would be 90%, but the lower 95% CI of this estimate is only 75%. In other words, even though the sample size was larger than any of the 40 studies reviewed here and was 10 times larger than many of them, the study still does not have the power to exclude clinically unacceptable false-negative rates.
None of the 40 studies reviewed were powered according to prespecified sample-size calculation. However, some of the small sample sizes in the 1970–1990 studies are understandable. Many of these studies were performed very early in the evolution of noninvasive methods, and the results were not intended to be applied directly to clinical practice. Moreover, the importance of accurate measurement of stenosis had not been demonstrated clearly. The situation was quite different, however, at the time of the 1993–1997 studies. Most of the imaging techniques studied were already used routinely and now required proper validation. This was implicit in many of the studies, several of which made clinical recommendations on the basis of their results. However, the sample sizes were completely inadequate in virtually all of the 1993–1997 studies. Indeed, on average, the later studies were much smaller than the earlier studies. A continuing profusion of small studies, many of which have inadequate methods, is likely to confuse rather than inform clinical practice.
Meta-Analyses of Imaging Studies
One way in which to extract some useful information from a group
of small studies is to combine the results to increase
precision.72 The techniques are now in place to allow
useful meta-analysis of diagnostic
studies.3 22 23 24 However, this is only possible if the
design of studies and the analysis and presentation
of data are of a sufficient and reasonably uniform standard. Our
results suggest that this is not currently the case for published
studies of imaging and measurement of carotid stenosis.
Conclusions
If the results of clinical trials of carotid
endarterectomy are to be applied to clinical
practice with noninvasive imaging, then new techniques must be properly
validated against angiography. Review of previous research in this area
shows that study methods and reporting of results are often poor. It
will only be possible to apply the results of studies to clinical
practice if the design, analysis, and reporting are of good
quality. The quality standards set out in this article for study design
provide a reasonable basis on which to proceed.
|
Received September 16, 1999; revision received March 16, 2000; accepted March 17, 2000.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1. Gillum RF. Epidemiology of carotid endarterectomy and cerebral arteriography in the United States. Stroke. 1995;26:1724–1728.
2. Murie JA, John TG, Morris PJ. Carotid endarterectomy in Great Britain and Ireland: practice between 1984 and 1992. Br J Surg. 1994;81:827–831.[Medline] [Order article via Infotrieve]
3.
Blakeley DD, Oddone EZ, Hasselblad V, Simel DL,
Matchar DB. Noninvasive carotid artery testing: a meta-analytic review.
Ann Intern Med. 1995;122:360–367.
4. European Carotid Surgery Trialists’ Collaborative Group. MRC European Carotid Surgery Trial: interim results for symptomatic patients with severe (70–99%) or with mild (0–29%) carotid stenosis. Lancet. 1991;337:1235–1243.[Medline] [Order article via Infotrieve]
5. North American Symptomatic Carotid Endarterectomy Trial Collaborators. Beneficial effect of carotid endarterectomy in symptomatic patients with high-grade stenosis. N Engl J Med. 1991;325:445–453.[Abstract]
6. Ackerman RH, Candia MR. Identifying clinically relevant carotid disease. Stroke. 1994;25:1–3.[Medline] [Order article via Infotrieve]
7.
Alexandrov AV, Bladin CF, Maggisano R, Norris
JW. Measuring carotid stenosis: time for a reappraisal.
Stroke. 1993;24:1292–1296.
8.
Fox AJ. How to measure carotid stenosis.
Radiology. 1993;186:316–318.
9.
Barnett HJM, Warlow CP. Carotid
endarterectomy and the measurement of
stenosis. Stroke. 1993;24:1281–1284.
10. Toole JF, Castaldo JE. Accurate measurement of carotid stenosis: chaos in methodology. J Neuroimaging. 1994;4:222–230.[Medline] [Order article via Infotrieve]
11. Hennerici MG, Daffertshofer M. Recent diagnostic improvement for the non-invasive assessment of cervical and cerebral arteriopathy. Cerebrovasc Dis. 1994;4(suppl 4):2–7.
12. Eliasziw M, Smith RF, Singh N, Holdsworth DW, Fox AJ, Barnett HJM, for the NASCET Group. Further comments on the measurement of carotid stenosis from angiograms. Stroke. 1994;25:2445–2449.[Abstract]
13.
Bladin CF, Alexandrov AV, Murphy J, Maggisano R,
Norris JW. Carotid Stenosis Index: a new method of measuring
internal carotid artery stenosis. Stroke. 1995;26:230–234.
14. Rothwell PM, Warlow CP. Making sense of measurement of carotid stenosis. Cerebrovasc Dis. 1996;6:54–58.
15.
Tu JV, Hannan EL, Anderson GM, Wu K, Vranizan K,
Popp AJ, Grumbach K. The fall and rise of carotid
endarterectomy in the United States and Canada.
N Engl J Med. 1998;339:1441–1447.
16.
Hsia DC, Moscoe LM, Krushat WM.
Epidemiology of carotid
endarterectomy among Medicare beneficiaries:
1985–1996 update. Stroke. 1998;29:346–350.
17.
Hankey GJ, Warlow CP, Sellar RJ. Cerebral
angiographic risk in patients with mild cerebrovascular disease.
Stroke. 1990;21:209–222.
18.
Davies KN, Humphrey PR. Complications of cerebral
angiography in patients with symptomatic carotid territory
ischaemia screened by carotid ultrasound. J Neurol Neurosurg
Psychiatry. 1993;56:967–972.
19. Muller R, Greitz I, Lilieqvist B. Aortocervical angiography on occlusive cerebrovascular disease. Neurology. 1964;14:136–146.
20. Kerber CW, Cromwell LD, Drayer BP, Bank WO. Cerebral ischemia, I: current angiographic techniques, complications and safety. Am J Roentgenol. 1978;130:1097–1103.[Abstract]
21. Hankey GJ, Warlow CP. Symptomatic carotid ischaemic events: safest and most cost effective way of selecting patients for angiography, before carotid endarterectomy. BMJ. 1990;300:1485–1491.
22. Irwig L, Macaskill P, Glasziou P, Fahey M. Meta-analytic methods for diagnostic test accuracy. J Clin Epidemiol. 1995;48:119–130.[Medline] [Order article via Infotrieve]
23.
Irwig L, Tosteson ANA, Gatsonis C, Lau J, Colditz
G, Chalmers TC, Mosteller F. Guidelines for meta-analyses
evaluating diagnostic tests. Ann Intern Med. 1994;120:667–676.
24. Sackett DL. Meta-analytic methods for diagnostic test accuracy: discussion. J Clin Epidemiol. 1995;48:131–132.
25. Brown PM, Johnston KW. The difficulty of quantifying the severity of carotid stenosis. Surgery. 1982;92:468–473.[Medline] [Order article via Infotrieve]
26.
Chikos PM, Fisher LD, Hirsch JH, Harley JD,
Thiele BL, Strandness DE Jr. Observer variability in evaluating
extracranial carotid artery stenosis. Stroke. 1983;14:885–892.
27. Murie JA, McKay AJ. Radiographic assessment of the extracranial internal carotid artery. J Cardiovasc Surg. 1986;27:154–157.[Medline] [Order article via Infotrieve]
28.
Deweese JA, May AG, Lipchik EO, Rob CG. Anatomic
and hemodynamic correlations in carotid artery
stenosis. Stroke. 1970;1:149–157.
29.
O’Leary DH, Bryan FA, Goodison MW, Rifkin MD,
Gramiak R, Ball M, Bond MG, Dunn RA, Goldberg BB, Toole JF, Wheeler HG,
Gustafson NF, Ekholm S, Raines JK. Measurement variability of carotid
atherosclerosis: real-time (B-mode) ultrasonography and
angiography. Stroke. 1987;18:1011–1017.
30. D’Alotto C, Pelz DM, Rankin RN. A comparison of angiography, intravenous digital subtraction angiography and duplex ultrasound in the diagnosis of carotid artery atherosclerosis. J Can Assoc Radiol. 1985;36:200–208.[Medline] [Order article via Infotrieve]
31.
Fisher M, Ahmadi J, Zee C-S, Terry R, Weiner JM.
Arteriography of the carotid bifurcation: oblique projections.
Neurology. 1985;35:1201–1204.
32.
O’Donnell TF, Erdoes L, Mackey WC, McCullough J,
Shepard A, Heggerick P, Isner J, Callow AD. Correlation of B-mode
ultrasound imaging and arteriography with pathologic findings at
carotid endarterectomy. Arch Surg. 1985;120:443–449.
33. Russell D, Bakke SJ, Nakstad P, Wiberg J, Nyberg-Hansen R. A comparison of pulsed doppler spectral analysis and intravenous digital subtraction angiography in the detection of carotid occlusive disease. Acta Neurol Scand. 1985;72:571–576.[Medline] [Order article via Infotrieve]
34.
Zwiewbal WJ, Strother CM, Austin CW, Sackett JF.
Comparison of ultrasound and IV-DSA for carotid evaluation.
Stroke. 1985;16:633–643.
35.
Hoffman MG, Gomes AS, Pais SO. Limitations in the
interpretation of intravenous carotid digital subtraction
angiography. AJR Am J Roentgenol. 1984;142:261–264.
36. Earnest F, Houser W, Forbes GS, Kispert DB, Sundt TM. The accuracy and limitations of intravenous digital subtraction angiography in the evaluation of atherosclerotic cerebrovascular disease: angiographic and surgical correlation. Mayo Clin Proc. 1983;58:735–746.[Medline] [Order article via Infotrieve]
37. Eikelboom BC, Ackerstaff RGA, Moll FL, de Vries AR. Accuracy and usefulness of combined duplex scanning and digital subtraction angiography. In: Greenhalgh RM, Rose FC, eds. Progress in Stroke Research 2. London, UK: Pitman Press; 1983:178–184.
38.
Eikelboom BC, Riles TR, Mintzer R, Baumann FG,
DeFillip G, Lin J, Imparato AM. Inaccuracy of angiography in the
diagnosis of carotid ulceration. Stroke. 1983;14:882–884.
39. Reily LM, Lusby RJ, Ehrenfeld WK. Carotid plaque pathology: the role of digital subtraction angiography and ultrasound imaging in identifying significant disease. In: Greenhalgh RM, Rose FC, eds. Progress in Stroke Research 2. London, UK: Pitman Press; 1983:162–177.
40.
Wood GW, Lukin RR, Tomsick TA, Chambers AA.
Digital subtraction angiography with intravenous injection:
assessment of 1000 carotid bifurcations. AJR Am J
Roentgenol. 1983;140:855–859.
41.
Chilcote WA, Modic MT, Pavlicek WA, Little JR,
Furlan AJ, Duchesneau PM, Weinstein MA. Digital subtraction angiography
of the carotid arteries: a comparative study in 100 patients.
Radiology. 1981;139:287–295.
42. Edwards JH, Kricheff II, Riles T, Imparato A. Angiographically undetected ulceration of the carotid bifurcation as a cause of embolic stroke. Radiology. 1979;132:369–373.[Abstract]
43. Kempczinski RF, Wood GW, Berlatzky Y, Pearce WH. Digital subtraction angiography versus non-invasive cerebrovascular testing in the detection of carotid bifurcation disease. In: Greenhalgh RM, Rose FC, eds. Progress in Stroke Research 2. London, UK: Pitman Press; 1983:185–196.
44.
Sheldon JJ, Janowitz W, Leborgne JM, Sivina M,
Rojo N. Intravenous DSA of extracranial carotid lesions: comparison
with other tech specimens. AJR Am J Roentgenol.. 1984;143:1289–94.
45.
Steinke W, Meairs S, Ries S, Hennerici M.
Sonographic assessment of carotid artery stenosis: comparison
of power Doppler imaging and color Doppler flow imaging.
Stroke. 1996;27:91–94.
46. Mittl RL Jr, Broderick M, Carpenter JP, Goldberg HI, Listerud J, Mishkin MM, Berkowitz HD, Atlas SW. Blinded-reader comparison of magnetic resonance angiography and duplex ultrasonography for carotid artery bifurcation stenosis. Stroke. 1994;25:4–10.[Abstract]
47.
Kagawa R, Moritake K, Shima T, Okada Y. Validity
of B-mode ultrasonographic findings in patients undergoing carotid
endarterectomy in comparison with angiographic and
clinicopathologic features. Stroke. 1996;27:700–705.
48.
Griewing B, Morgenstern C, Driesner F, Kallwellis
G, Walker ML, Kessler C. Cerebrovascular disease assessed by color-flow
and power Doppler ultrasonography: comparison with digital
subtraction angiography in internal carotid artery stenosis.
Stroke. 1996;27:95–100.
49.
Leclerc X, Godefroy O, Pruvo JP, Leys D. Computed
tomographic angiography for the evaluation of carotid artery
stenosis. Stroke. 1995;26:1577–1581.
50.
Sitzer M, Furst G, Fischer H, Siebler M, Fehlings
T, Kleinschmidt A, Kahn T, Steinmetz H. Between-method correlation in
quantifying internal carotid stenosis. Stroke. 1993;24:1513–1518.
51. Turnipseed WD, Kennell TW, Turski PA, Acher CW, Hoch JR. Magnetic resonance angiography and duplex imaging: noninvasive tests for selecting symptomatic carotid endarterectomy candidates. Surgery. 1993;114:643–648.[Medline] [Order article via Infotrieve]
52. Vanninen R, Manninen H, Koivisto K, Tulla H, Partanen K, Puranen M. Carotid stenosis by digital subtraction angiography: reproducibility of the European Carotid Surgery Trial and the North American Symptomatic Carotid Endarterectomy Trial measurement methods and visual interpretation. Am J Neuroradiol. 1994;15:1635–1641.[Abstract]
53. Spartera C, Morettini G, Marino G, Marsili L, Di Cesare E, La Barbera G, Petrassi C, Ventura M. Detection of internal carotid artery stenosis: comparison of 2D-MR angiography, duplex scanning, and arteriography. J Cardiovasc Surg. 1993;34:209–213.[Medline] [Order article via Infotrieve]
54.
Huston J, Lewis BD, Wiebers DO, Meyer FB,
Riederer SJ, Weaver AL. Carotid artery: prospective blinded comparison
of two-dimensional time-of-flight MR angiography with conventional
angiography and duplex US. Radiology. 1993;186:339–344.
55. Srinivasan J, Mayberg MR, Weiss DG, Eskridge J. Duplex accuracy compared with angiography in the Veterans Affairs Cooperative Studies Trial for Symptomatic Carotid Stenosis. Neurosurgery. 1995;36:648–653.[Medline] [Order article via Infotrieve]
56.
Patel MR, Kuntz KM, Klufas RA, Kim D, Kramer J,
Polak JF, Skillman JJ, Whittemore AD, Edelman RR, Kent C. Preoperative
assessment of the carotid bifurcation: can magnetic resonance
angiography and duplex ultrasonography replace contrast arteriography?
Stroke. 1995;26:1753–1758.
57. Frey JL, Heiserman J, Fram EK, Strava D. Utility of magnetic resonance angiography and carotid ultrasound in the evaluation of carotid stenosis. J Stroke Cerebrovasc Dis. 1995;5:83–87.
58. White JE, Russell WL, Greer MS, Whittle MT. Efficacy of screening MR angiography and doppler ultrasonography in the evaluation of carotid artery stenosis. Am Surg. 1994;60:340–348.[Medline] [Order article via Infotrieve]
59. Anson JA, Heiserman JE, Drayer BP, Spetzler RF. Surgical decisions on the basis of magnetic resonance angiography of the carotid arteries. Neurosurgery. 1993;32:335–343.[Medline] [Order article via Infotrieve]
60.
Young GR, Humphrey PRD, Nixon TE, Smith ETS.
Variability in measurement of extracranial internal carotid artery
stenosis as displayed by both digital subtraction and magnetic
resonance angiography. Stroke. 1996;27:467–473.
61. Lustgarten JH, Soloman RA, Quest DO, Khanjdi AG, Mohr JP. Carotid endarterectomy after noninvasive evaluation by duplex ultrasonography and magnetic resonance angiography. Neurosurgery. 1994;34:612–619.[Medline] [Order article via Infotrieve]
62. Neale M, Chambers JL, Kelly AT, Connard S, Lawton MA, Roche J, Appleberg M. Reappraisal of duplex criteria to assess significant carotid stenosis with special reference to reports from the North American Symptomatic Carotid Endarterectomy Trial and the European Carotid Surgery Trial. J Vasc Surg. 1994;20:642–649.[Medline] [Order article via Infotrieve]
63. Ballard JL, Fleig K, De Lange M, Killeen JD. The diagnostic accuracy of duplex ultrasonography for evaluating carotid bifurcation. Am J Surg. 1994;168:123–126.[Medline] [Order article via Infotrieve]
64. Buijs PC, Klop RBJ, Eikelboom BC, Mali W, Bakker CJG, Beek FJA, van Gils APG, Dillon EH, Ramos LMP. Carotid bifurcation imaging: magnetic resonance angiography compared to conventional angiography and Doppler ultrasound. Eur J Vasc Surg. 1993;7:245–251.[Medline] [Order article via Infotrieve]
65. Eisenberg RL, Hedgcock MW, Gooding GAW, DeMartini WJ, Akin JR, Ovenfors CO. Compensation examination of the cervical and lumbar spines: critical disagreement in radiographic interpretation. Am J Radiol. 1980;134:519–522.[Abstract]
66.
Herman PG, Gerson DE, Hessel SJ. Disagreements in
chest roentgen interpretation. Chest. 1975;68:278–282.
67.
Norden C, Philipps E, Levy P, Kass E. Variation
in interpretation of intravenous pyelograms. Am
J Epidemiol. 1970;91:155–160.
68. Slot HB, Strijbosch, Greep JM. Interobserver variability in single plane aortography. Surgery. 1981;90:497–503.[Medline] [Order article via Infotrieve]
69.
Zir LM, Miller SW, Dinsmore RE, Gilbert JP,
Harthorne JW. Interobserver variability in coronary
angiography. Circulation. 1976;53:627–632.
70. Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Statistics Med. 1984;3:409–420.
71. Dixon AK. Evidence-based diagnostic radiology. Lancet. 1997;350:509–512.[Medline] [Order article via Infotrieve]
72. Peto R. Why do we need systematic overviews of randomized trials? Statistics Med. 1987;6:233–240.
This article has been cited by other articles:
 |
|
 |
|
  F. M. Chappell, J. M. Wardlaw, G. R. Young, J. H. Gillard, G. H. Roditi, B. Yip, J. P. Pell, P. M. Rothwell, M. M. Brown, M. J. Gough, et al. Carotid Artery Stenosis: Accuracy of Noninvasive Tests--Individual Patient Data Meta-Analysis Radiology, May 1, 2009; 251(2): 493 - 502. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 F. Runck, R.P. Steiner, W.A. Bautz, and M.M. Lell MR Imaging: Influence of Imaging Technique and Postprocessing on Measurement of Internal Carotid Artery Stenosis AJNR Am. J. Neuroradiol., October 1, 2008; 29(9): 1736 - 1742. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S Khan, G C Cloud, S Kerry, and H S Markus Imaging of vertebral artery stenosis: a systematic review J. Neurol. Neurosurg. Psychiatry, November 1, 2007; 78(11): 1218 - 1225. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H.M. Silvennoinen, S. Ikonen, L. Soinne, M. Railo, and L. Valanne CT Angiographic Analysis of Carotid Artery Stenosis: Comparison of Manual Assessment, Semiautomatic Vessel Analysis, and Digital Subtraction Angiography AJNR Am. J. Neuroradiol., January 1, 2007; 28(1): 97 - 103. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Lell, C. Fellner, U. Baum, T. Hothorn, R. Steiner, W. Lang, W. Bautz, and F.A. Fellner Evaluation of Carotid Artery Stenosis with Multisection CT and MR Imaging: Influence of Imaging Modality and Postprocessing AJNR Am. J. Neuroradiol., January 1, 2007; 28(1): 104 - 110. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R.L. Cuffe and P.M. Rothwell Effect of Nonoptimal Imaging on the Relationship Between the Measured Degree of Symptomatic Carotid Stenosis and Risk of Ischemic Stroke Stroke, July 1, 2006; 37(7): 1785 - 1791. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. R. Sprouse II, G. H. Meier, F. N. Parent, R. J. DeMasi, C. J. LeSar, C. Nelms, K. Carter, M. J. Marcinczyk, R. G. Gayle, and B. Mendoza Are We Undertreating Carotid Stenoses Diagnosed by Ultrasound Alone? Vascular and Endovascular Surgery, March 1, 2005; 39(2): 143 - 151. [Abstract] [PDF]
|
|
|
|
|
|
D. E. Hyde, A. J. Fox, I. Gulka, P. Kalapos, D. H. Lee, D. M. Pelz, and D. W. Holdsworth Internal Carotid Artery Stenosis Measurement: Comparison of 3D Computed Rotational Angiography and Conventional Digital Subtraction Angiography Stroke, December 1, 2004; 35(12): 2776 - 2781. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. J.W. Koelemay, P. J. Nederkoorn, J. B. Reitsma, and C. B. Majoie Systematic Review of Computed Tomographic Angiography for Assessment of Carotid Artery Disease Stroke, October 1, 2004; 35(10): 2306 - 2312. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. E. Feasby and J. M. Findlay CT angiography for the assessment of carotid stenosis Neurology, August 10, 2004; 63(3): 412 - 413. [Full Text] [PDF]
|
|
|
|
 |
|
 D. W. Dodick, I. Meissner, F. B. Meyer, and H. J. Cloft Evaluation and Management of Asymptomatic Carotid Artery Stenosis Mayo Clin. Proc., July 1, 2004; 79(7): 937 - 944. [Abstract] [PDF]
|
|
|
|
|
|
J. M. U-King-Im, R. A. Trivedi, M. J. Graves, N. J. Higgins, J. J. Cross, B. D. Tom, W. Hollingworth, H. Eales, E. A. Warburton, P. J. Kirkpatrick, et al. Contrast-enhanced MR angiography for carotid disease: Diagnostic and potential clinical impact Neurology, April 27, 2004; 62(8): 1282 - 1290. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. Kennedy, H. Quan, W. A. Ghali, and T. E. Feasby Importance of the imaging modality in decision making about carotid endarterectomy Neurology, March 23, 2004; 62(6): 901 - 904. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. M. Rothwell For Severe Carotid Stenosis Found on Ultrasound, Further Arterial Evaluation Prior to Carotid Endarterectomy Is Unnecessary: The Argument Against Stroke, July 1, 2003; 34(7): 1817 - 1819. [Full Text] [PDF]
|
|
|
|
|
|
J. U-King-Im, R. Trivedi, J. H. Gillard, D. C.C. Johnston, J. D. Eastwood, and L. B. Goldstein Contrast-Enhanced Magnetic Resonance Angiography Carotid Arteries * Response Stroke, May 1, 2003; 34 (5): e15 - e15. [Full Text] [PDF]
|
|
|
|
|
|
P. J. Nederkoorn, Y. van der Graaf, and M.G. M. Hunink Duplex Ultrasound and Magnetic Resonance Angiography Compared With Digital Subtraction Angiography in Carotid Artery Stenosis: A Systematic Review Stroke, May 1, 2003; 34(5): 1324 - 1331. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. M. Rothwell, S. A. Gutnikov, and C. P. Warlow Reanalysis of the Final Results of the European Carotid Surgery Trial Stroke, February 1, 2003; 34(2): 514 - 523. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. C.C. Johnston, J. D. Eastwood, T. Nguyen, and L. B. Goldstein Contrast-Enhanced Magnetic Resonance Angiography of Carotid Arteries: Utility in Routine Clinical Practice Stroke, December 1, 2002; 33(12): 2834 - 2838. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. J. Nederkoorn, Y. van der Graaf, B. C. Eikelboom, A. van der Lugt, L. W. Bartels, and W. P.T.M. Mali Time-of-Flight MR Angiography of Carotid Artery Stenosis: Does a Flow Void Represent Severe Stenosis? AJNR Am. J. Neuroradiol., November 1, 2002; 23(10): 1779 - 1784. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Schulte-Altedorneburg, D. W. Droste, S. Felszeghy, L. Csiba, V. Popa, K. Hegedus, J. Kollar, L. Modis, and E. B. Ringelstein Detection of Carotid Artery Stenosis by In Vivo Duplex Ultrasound: Correlation With Planimetric Measurements of the Corresponding Postmortem Specimens Stroke, October 1, 2002; 33(10): 2402 - 2407. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
P. J. Nederkoorn, W. P.Th.M. Mali, B. C. Eikelboom, O. E.H. Elgersma, E. Buskens, M.G. M. Hunink, L. J. Kappelle, P. C. Buijs, A. F.J. Wust, A. van der Lugt, et al. Preoperative Diagnosis of Carotid Artery Stenosis: Accuracy of Noninvasive Testing Stroke, August 1, 2002; 33(8): 2003 - 2008. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S G Patel, D A Collie, J M Wardlaw, S C Lewis, A R Wright, R J Gibson, and R J Sellar Outcome, observer reliability, and patient preferences if CTA, MRA, or Doppler ultrasound were used, individually or together, instead of digital subtraction angiography before carotid endarterectomy J. Neurol. Neurosurg. Psychiatry, July 1, 2002; 73(1): 21 - 28. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
D. C.C. Johnston and L. B. Goldstein Clinical carotid endarterectomy decision making: Noninvasive vascular imaging versus angiography Neurology, April 24, 2001; 56(8): 1009 - 1015. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. S. Jeret, H. J. M. Barnett, and J. P. Broderick Carotid endarterectomy: Another wake-up call Neurology, April 24, 2001; 56(8): 1118 - 1118. [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||