| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2004;35:2306.)
© 2004 American Heart Association, Inc.
Original Contributions |
From the Departments of Surgery (M.J.W.K.), Neurology (P.J.N.), Clinical Epidemiology & Biostatics (J.B.R.), and Radiology (C.B.M.), Academic Medical Center, Amsterdam, The Netherlands.
Correspondence to Dr Paul J. Nederkoorn, Department of Neurology, H2221, Academic Medical Center, PO Box 22700, 1100 DE Amsterdam, The Netherlands. E-mail p.j.nederkoorn{at}amc.uva.nl
| Abstract |
|---|
|
|
|---|
Methods The PubMed, MEDLINE, PREMEDLINE, EMBASE, and CINAHL databases were searched to retrieve studies published between 1990 and July 2003, comparing CTA and intra-arterial digital subtraction angiography in patients with symptomatic carotid artery disease that presented raw data for detection of a <70% or 70% to 99% stenosis or an occlusion. Two observers independently assessed methodological quality and constructed 2x2 tables for sensitivity and specificity of CTA for detection of a 70% to 99% stenosis versus <70% stenosis or an occlusion, and for <99% stenosis versus occlusion. A bivariate random effects model was used to calculate the pooled sensitivity and specificity of CTA for detection of these lesions.
Results Some 864 patients (66% male) with a mean age of 66 years were studied in the 28 studies included in the meta-analysis. In all studies, a single-slice CT-scan was used. Only 8 studies satisfied all methodological quality criteria. The pooled sensitivity and specificity for detection of a 70% to 99% stenosis were 85% (95% CI, 79% to 89%) and 93% (95% CI, 89% to 96%), respectively. For detection of an occlusion, the sensitivity and specificity were 97% (95% CI, 93% to 99%) and 99% (95% CI, 98% to 100%), respectively. Incomplete reporting of demographic characteristics and technical differences in the individual studies obstructed a meaningful subgroup analysis.
Conclusions CTA is an accurate modality for detection of severe carotid artery disease, especially for detection of occlusions. The fair methodological quality of the included studies must be taken into account when interpreting these results.
Key Words: angiography carotid stenosis computed tomography diagnosis meta-analysis review, systematic
| Introduction |
|---|
|
|
|---|
Computed tomographic angiography (CTA) is a potentially attractive noninvasive tool because fast helical CT scanners are widely available. Iodine contrast may limit its application in patients with renal insufficiency or cardiac failure. The axial source images can be used to grade the severity of stenosis and to visualize the arterial lumen and surrounding tissues. In addition, CTA offers several postprocessing techniques to construct 3D images of the artery.3 Definitive grading of stenosis by combining source images and 3D reconstructions is not standardized. Each postprocessing technique has its strengths and limitations. Shaded surface display reconstructions provide a 3D image of the outer vessel wall, but no information about the residual lumen. Volume rendering allows for 3D reconstructions similar to shaded surface display images and can differentiate between the arterial wall and surrounding structures. The accuracy of the frequently used maximum intensity projection (MIP) reconstructions may be limited by bone or calcifications that obscure the depiction of intraluminal contrast and thus the residual lumen. Multiplanar and curved planar reformations facilitate reconstruction of the tortuous cervical arteries in user-defined anatomical planes.
All of these qualities give CTA the potential to replace or complement arteriography, color duplex scanning (CDS), or magnetic resonance angiography (MRA). However, the introduction of new technologies in clinical practice requires a thorough evaluation in studies of high methodological quality. It has been shown that the quality of design and reporting of studies of diagnostic imaging of the carotid arteries leaves room for improvement.4
We conducted a systematic literature review and a meta-analysis to obtain the best available estimates of the diagnostic performance of CTA compared with conventional angiography or IADSA for assessment of the carotid arteries in patients with symptomatic carotid artery disease.
| Methods |
|---|
|
|
|---|
Studies comparing CTA with conventional angiography or IADSA as reference standard in patients with symptomatic carotid artery disease for detection of a 70% to 99% stenosis or an occlusion, and that presented 2x2 contingency tables or data allowing their construction, were included in our analysis. If such data were available for only a subset of patients, this subset was included. From studies reporting repeatedly on the same study population, only the most recent study was included. Duplicate publications were excluded. Authors were contacted to ensure that possibly overlapping study populations were included only once. When authors did not respond, only the most recent or largest study was included. If 2x2 tables could not be calculated from the published studies, authors were requested to provide these data. These studies were excluded if the authors did not respond.
Quality Assessment and Data Extraction
The methodological quality of included studies was graded independently by 2 observers (M.J.W.K. and P.J.N.). The following elements for good study quality were scored: consecutively enrolled patients, prospective study design, clear description of CTA technique (sufficient detail to permit replication), clear definition of cutoff levels (ie, grading lesions as <70% stenosis, 70% to 99% stenosis or occlusion), and independent blind assessment of CTA and arteriography. Some of these design-related aspects have been shown to introduce bias and overestimation of diagnostic accuracy.5 If authors did not explicitly state consecutive enrollment, prospective design, and use of blinding, we assumed that this was not the case. Discrepancies in methodological judgment were resolved by consensus.
The same observers independently extracted raw data from the studies to construct contingency tables. Separate tables were made for discrimination between a 70% to 99% stenosis versus <70% stenosis or occlusion, and between occlusion versus no occlusion. Results for multiple observers in a study were averaged, if applicable. Other elements that were extracted included the year of publication, interval between CTA and arteriography, the patients mean or median age, proportion of male subjects, qualifying symptoms, 3D postprocessing technique, and whether the asymptomatic contralateral carotid artery was included in the analysis. Data were considered missing if they were not mentioned explicitly in the text. Discrepancies were resolved by discussion.
Analysis
We used a bivariate meta-regression model to obtain summary estimates of sensitivity and specificity.6,7 Rather than using a single outcome measure per study, like the diagnostic odds ratio in summary receiver operating characteristic analysis, the bivariate model preserves the 2D nature of diagnostic data by directly analyzing the logarithmically (logit) transformed sensitivity {log[sens/(1-sens)]} and specificity {log[spec/(1-spec)]} of each study in a single model. This model estimates and incorporates the possible correlation between logit sensitivity and specificity within studies due to possible differences in threshold between studies. The bivariate model uses a random effects approach for both sensitivity and specificity, allowing for heterogeneity beyond chance due to clinical or methodological differences between studies. In addition, the model acknowledges the difference in precision by which sensitivity and specificity have been measured in each study. Thus, studies with a larger number of patients with the target condition receive more weight in the calculation of the summary estimate of sensitivity, whereas studies with more patients without the target condition are more influential in the pooling of specificity.
The model requires logit transformation of the sensitivity and specificity. A standard correction of adding 0.5 to all cells of the 2x2 table was applied when either sensitivity or specificity was 100%. The model produces the following results: a random effect estimate of the mean sensitivity and specificity with 95% CIs, the amount of between-study variation for sensitivity and specificity separately, and the strength and shape of the correlation between sensitivity and specificity. Using these results, we calculated a 95% confidence ellipse around the summary estimate of sensitivity and specificity. All results have been transformed back (antilogit) to the original scale and values have been plotted values in receiver operating characteristic space.
Covariates can be introduced into the model to explain variation in sensitivity and specificity between studies. Covariates were selected if a specific methodological or clinical variable showed a positive Spearman correlation with the sensitivity or specificity with a probability value <0.1. The Proc Mixed procedure in SAS version 8.2 for Windows (SAS Institute) was used to fit all bivariate models.
When multiple postprocessing modalities were evaluated in a study, we decided to include this study only once in the analysis, and preferably the results of MIP reconstructions such as these are most frequently used in daily practice.
Assessment of Publication Bias
We applied Beggs method to explore the possibility of publication bias. This method uses the adjusted rank correlation between test accuracy estimates and their variances. A positive Spearman
indicates a trend toward higher test accuracy in studies with a smaller sample size.8
| Results |
|---|
|
|
|---|
Some 28 studies,2249 of which 21 (75%) were in the English language, were included in the analysis. Table 1 lists their methodological and demographic characteristics. Eight studies (29%) satisfied all methodological quality criteria. In all but 3 studies25,40,48 stenosis was graded according to North American Symptomatic Carotid Endarterectomy Trial (NASCET) criteria.
|
Meta-Analysis
Table 2 lists the data from the included studies. The median and mean sample sizes of the studies were 23 and 30 patients, respectively. The meta-analysis comprises a total of 864 patients. The median of their mean age was 66 years and the median proportion of male subjects was 66% (data from 22 and 23 studies, respectively). For detection of a 70% to 99% stenosis, the pooled sensitivity was 85% (95% CI, 79% to 89%) and the pooled specificity was 93% (95% CI, 89% to 96%). The Figure represents the data from the original studies and the pooled sensitivity and specificity with confidence ellipse. Two studies using a cutoff point for stenosis at 80% were also included in this analysis.27,48 Excluding these studies from analysis yielded a pooled sensitivity of 85% (95% CI, 79% to 90%) and a specificity of 94% (95% CI, 90% to 96%).
|
|
In nearly all studies, the sensitivity and specificity were 100% for detection of an occlusion. A correction of adding 0.5 to all cells would have led to substantial downward bias in the summary estimates of sensitivity and specificity in our bivariate model, or in any other meta-analytic approach, because of the low occlusion rate in these studies. As the results were highly homogenous among studies, we did a fixed effect pooling, resulting in a sensitivity of 97% (95% CI, 93% to 99%) and a specificity of 99% (95% CI, 98% to 100%).
Incomplete reporting of patient demographics, symptoms, and interval between CTA and arteriography in 14 of the studies (50%) obstructed analysis of the influence of these covariates on the diagnostic performance. Only the year of publication and design-related characteristics could be included one at a time. The diagnostic accuracy was not influenced by the year of publication. In addition, we found no significant differences for other covariates other than a higher specificity in prospective studies compared with retrospective studies. In a comparison between studies using MIP projections to grade stenoses and studies using other postprocessing techniques, we also found no significant differences. (Data available from the authors.)
Publication Bias
The Spearman
for correlation between test accuracy and sample size of the included studies was 0.49 (P=0.003). This is an indication for publication bias, probably because studies with a small sample size tend to report higher diagnostic accuracy.
| Discussion |
|---|
|
|
|---|
Our estimates of the diagnostic accuracy of CTA are more conservative than in a similar recent meta-analysis that reported a pooled sensitivity of 91.6% and a specificity of 97.4% for detection of a 70% to 99% stenosis.51 This difference may have several explanations. Although Hollingworth et al located more studies, their analysis comprised only 15 studies, whereas we included 28 studies. This indicates that we did not fail to identify relevant studies. In addition, whereas we included studies only once, they included several studies evaluating different postprocessing techniques more than once. The latter may result in smaller estimates of the SE due to the correlation of the results of multiple postprocessing techniques in the same patient. Finally, a random effects method of pooling data is likely to produce more conservative estimates than calculating crude estimates, which it did.
How does the accuracy of CTA relate to CDS and MRA? A recent meta-analysis reported a pooled sensitivity and specificity of 95% and 90%, respectively, for detection of a 70% to 99% stenosis with MRA, and of 86% and 87% with CDS, respectively.52 For detection of occlusions, MRA had a sensitivity and specificity of 98% and 100%, respectively, and CDS of 96% and 100%.52 The current meta-analysis indicates that the accuracy of CTA for grading stenoses lies somewhere in between, whereas its accuracy to detect occlusions is comparable to MRA and CDS.
It would be interesting to know if performing multiple noninvasive tests could improve patient selection for an intervention, as suggested in a study of CDS, MRA, and angiography, but such data are lacking for CTA.50 It is also unclear if CTA has additional clinical value over MRA or CDS. The 4 studies that directly compared CTA, MRA, and arteriography reported no significant differences in diagnostic accuracy of both modalities.4345,49 We could not locate studies comparing CTA, CDS, and arteriography simultaneously.
We tried to explore possible differences between the various postprocessing techniques. However, it was often unclear from the original articles which (combination of axial images and) postprocessing technique was used for definitive grading of the stenosis. Our distinction between MIP and all other postprocessing techniques detected no differences in diagnostic accuracy.
Our review has several limitations. We tried to minimize the influence of publication bias by searching multiple literature databases. We recognize that our search was limited by the fact that we did not hand-search leading journals or conference proceedings. Whereas a failure to include the gray literature may overestimate a treatment effect by 15%,53 a similar effect for studies of diagnostic accuracy has yet to be established. We do not think that the positive Beggs test for publication bias shows that our search failed to identify all relevant studies, but merely indicates that small studies with promising results have a higher likelihood of publication than larger studies with less favorable results. Indeed, the median sample size of the studies was small. The estimates of diagnostic accuracy derived from small studies have a wide confidence interval. Although meta-analysis is a tool to calculate more precise estimates, it would be desirable if original studies included a larger number of patients.
In line with the study by Rothwell et al,4 we found that the methodological quality of the included studies was fair. In addition, the meta-analysis was partly obstructed by incomplete reporting of patient demographics and study design, and unclear presentation of the results. This is not unique to the studies we retrieved for our systematic review but occurs in diagnostic research in many fields.54 The STAndards for Reporting of Diagnostic accuracy (STARD) steering committee has proposed guidelines for the conduct and reporting of diagnostic research to improve the quality of such studies.55 It is desirable that future studies will adhere to this concept if only to facilitate the performance of systematic reviews and meta-analyses, thereby speeding the appreciation of new imaging tools.
In conclusion, the overall methodological quality of studies comparing CTA and angiography for assessment of symptomatic carotid artery disease is fair. In these studies, CTA is an accurate test to detect a 70% to 99% stenosis in the carotid artery, is highly accurate for the detection and exclusion of occlusions, and may be used as an alternative for CDS and MRA. Whether patients will benefit from a work-up with CTA remains unclear, as currently no data are available of decision analyses or clinical trials that compare outcomes and costs of management of symptomatic carotid artery disease based on a noninvasive work-up with CDS, MRA, CTA, or arteriography.
Received March 26, 2004; revision received June 23, 2004; accepted July 5, 2004.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
T. Wolff, J. Guirguis-Blake, T. Miller, M. Gillespie, and R. Harris Screening for Carotid Artery Stenosis: An Update of the Evidence for the U.S. Preventive Services Task Force Ann Intern Med, December 18, 2007; 147(12): 860 - 870. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. M. Kramer, M. J. Budoff, Z. A. Fayad, V. A. Ferrari, C. Goldman, J. R. Lesser, E. T. Martin, S. Rajagopalan, J. P. Reilly, G. P. Rodgers, et al. ACCF/AHA 2007 Clinical Competence Statement on vascular imaging with computed tomography and magnetic resonance Vascular Medicine, November 1, 2007; 12(4): 359 - 378. [PDF] |
||||
![]() |
C. M. Kramer, M. J. Budoff, Z. A. Fayad, V. A. Ferrari, C. Goldman, J. R. Lesser, E. T. Martin, S. Rajagopalan, J. P. Reilly, G. P. Rodgers, et al. ACCF/AHA 2007 Clinical Competence Statement on Vascular Imaging With Computed Tomography and Magnetic Resonance: A Report of the American College of Cardiology Foundation/American Heart Association/American College of Physicians Task Force on Clinical Competence and Training Developed in Collaboration With the Society of Atherosclerosis Imaging and Prevention, the Society for Cardiovascular Angiography and Interventions, the Society of Cardiovascular Computed Tomography, the Society for Cardiovascular Magnetic Resonance, and the Society for Vascular Medicine and Biology J. Am. Coll. Cardiol., September 11, 2007; 50(11): 1097 - 1114. [Full Text] [PDF] |
||||
![]() |
R. L. DeLaPaz and for the Expert Panel on Neurologic Imaging Cerebrovascular Disease AJNR Am. J. Neuroradiol., June 1, 2007; 28(6): 1197 - 1199. [Full Text] [PDF] |
||||
![]() |
E. S. Bartlett, T. D. Walters, S. P. Symons, and A. J. Fox Carotid Stenosis Index Revisited With Direct CT Angiography Measurement of Carotid Arteries to Quantify Carotid Stenosis Stroke, February 1, 2007; 38(2): 286 - 291. [Abstract] [Full Text] [PDF] |
||||
![]() |
American Society of Interventional & Therapeutic N, Society for Cardiovascular Angiography and Interve, Society for Vascular Medicine and Biology, Society of Interventional Radiology, E. R. Bates, J. D. Babb, D. E. Casey Jr, C. U. Cates, G. R. Duckwiler, T. E. Feldman, et al. ACCF/SCAI/SVMB/SIR/ASITN 2007 Clinical Expert Consensus Document on Carotid Stenting: A Report of the American College of Cardiology Foundation Task Force on Clinical Expert Consensus Documents (ACCF/SCAI/SVMB/SIR/ASITN Clinical Expert Consensus Document Committee on Carotid Stenting) J. Am. Coll. Cardiol., January 2, 2007; 49(1): 126 - 170. [Full Text] [PDF] |
||||
![]() |
M. Lell, C. Fellner, U. Baum, T. Hothorn, R. Steiner, W. Lang, W. Bautz, and F.A. Fellner Evaluation of Carotid Artery Stenosis with Multisection CT and MR Imaging: Influence of Imaging Modality and Postprocessing AJNR Am. J. Neuroradiol., January 1, 2007; 28(1): 104 - 110. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. T. de Weert, M. Ouhlous, E. Meijering, P. E. Zondervan, J. M. Hendriks, M. R.H.M. van Sambeek, D. W.J. Dippel, and A. van der Lugt In Vivo Characterization and Quantification of Atherosclerotic Carotid Plaque Components With Multidetector Computed Tomography and Histopathological Correlation Arterioscler. Thromb. Vasc. Biol., October 1, 2006; 26(10): 2366 - 2372. [Abstract] [Full Text] [PDF] |
||||
![]() |
E.S. Bartlett, T.D. Walters, S.P. Symons, and A.J. Fox Diagnosing Carotid Stenosis Near-Occlusion by Using CT Angiography AJNR Am. J. Neuroradiol., March 1, 2006; 27(3): 632 - 637. [Abstract] [Full Text] [PDF] |
||||
![]() |
E.S. Bartlett, S.P. Symons, and A.J. Fox Correlation of Carotid Stenosis Diameter and Cross-Sectional Areas with CT Angiography AJNR Am. J. Neuroradiol., March 1, 2006; 27(3): 638 - 642. [Abstract] [Full Text] [PDF] |
||||
![]() |
E.S. Bartlett, T.D. Walters, S.P. Symons, and A.J. Fox Quantification of Carotid Stenosis on CT Angiography AJNR Am. J. Neuroradiol., January 1, 2006; 27(1): 13 - 19. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Mutze, G. Rademacher, G. Matthes, N. Hosten, and D. Stengel Blunt Cerebrovascular Injury in Patients with Blunt Multiple Trauma: Diagnostic Accuracy of Duplex Doppler US and Early CT Angiography Radiology, December 1, 2005; 237(3): 884 - 892. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. D. Bui, D. Gelfand, S. Whipple, S. E. Wilson, R. M. Fujitani, R. Conroy, H. Pham, and I. L. Gordon Comparison of CT and Catheter Arteriography for Evaluation of Peripheral Arterial Disease Vascular and Endovascular Surgery, November 1, 2005; 39(6): 481 - 490. [Abstract] [PDF] |
||||
![]() |
H. Ota, K. Takase, H. Rikimaru, M. Tsuboi, T. Yamada, A. Sato, S. Higano, T. Ishibashi, and S. Takahashi Quantitative Vascular Measurements in Arterial Occlusive Disease RadioGraphics, September 1, 2005; 25(5): 1141 - 1158. [Abstract] [Full Text] [PDF] |
||||
![]() |
C. P. Derdeyn, E. Buskens, P. J. Nederkoorn, Y. van der Graaf, and M. G. M. Hunink Conventional Angiography Remains an Important Tool for Measurement of Carotid Arterial Stenosis * Dr Buskens and colleagues respond: Radiology, May 1, 2005; 235(2): 711 - 713. [Full Text] [PDF] |
||||
| ||||||||||||||||||||||||||||||||||||||||||||||||