Diagnosing Intracranial Aneurysms With MR Angiography
Systematic Review and Meta-Analysis
Background and Purpose—The aim of this study was to evaluate the sensitivity and specificity of MR angiography (MRA) in the diagnosis of ruptured and unruptured intracranial aneurysms.
Methods—A systematic search was performed on 4 electronic databases on relevant articles that were published from January 1998 to October 2013. Inclusion criteria were met by 12 studies that compared MRA with digital subtraction angiography as reference standard. Two independent reviewers evaluated the methodological quality of the studies. Data from eligible studies were extracted and used to construct 2×2 contingency tables on a per-aneurysm level. Pooled estimates of sensitivity and specificity were calculated for all studies and subgroups of studies. Heterogeneity was tested, and risk for publication bias was assessed.
Results—Included studies were of high methodological quality. Studies with larger sample size tended to have higher diagnostic performance. Most studies used time-of-flight MRA technique. Among the 960 patients assessed, 772 aneurysms were present. Heterogeneity with reference to sensitivity and specificity was moderate to high. Pooled sensitivity of MRA was 95% (95% confidence interval, 89%–98%), and pooled specificity was 89% (95% confidence interval, 80%–95%). False-negative and false-positive aneurysms detected on MRA were mainly located at the skull base and middle cerebral artery. Freehand 3-dimensional reconstructions performed by the radiologist significantly increased diagnostic performance. Studies performed on 3 Tesla showed a trend toward higher performance (P=0.054).
Conclusions—Studies on diagnostic performance of MRA show high sensitivity with large variation in specificity in the detection of intracranial aneurysms.
Intracranial aneurysms are most reliably imaged by selective catheter digital subtraction angiography (DSA),1,2 but DSA is time-consuming and invasive. In recent years, computed tomographic angiography (CTA) has widely been used as a routine primary diagnostic test.3,4 However, both CTA and DSA are associated with radiation exposure and possible allergy to iodinated contrast material. The role of MR angiography (MRA) as an alternative primary noninvasive test in the diagnosis of intracranial aneurysms, therefore, needs to be evaluated.5
In 2000, a systematic review on diagnostic accuracy of MRA in intracranial aneurysms was published by White et al.6 Studies published until December 1998 were included in that review. Since then, no further meta-analysis on diagnostic performance of MRA in the field of intracranial aneurysms has been published.
The aim of this study was to evaluate the sensitivity and specificity of MRA in the diagnosis of ruptured and unruptured intracranial aneurysms by performing a systematic review, including more recently published studies.
A literature search was performed on 4 electronic databases including PubMed/MEDLINE, Embase, and Cochrane database of controlled trials to identify potentially relevant articles that were published between January 1, 1999, and October 12, 2013. The search was performed with the use of the following terms: Magnetic Resonance Angiography (a Medical Subject Heading [MeSH] term) and Aneurysm (MeSH term) or Intracranial Aneurysm (MeSH term) or Subarachnoid Hemorrhage (MeSH term). The search in PubMed/MEDLINE was limited to articles with abstracts; case reports and reviews were excluded. The search in Embase was limited to articles with abstracts and articles on humans. In addition, searching in all databases was performed using free-text terms, including all possible variations of MR Angiography, Intracranial Aneurysm, and Subarachnoid Hemorrhage. No language restrictions were used.
We searched for studies that assessed the diagnostic performance of MRA in patients with ruptured and unruptured intracranial aneurysms. Studies were eligible if (1) all patients underwent DSA as reference standard for presence of an aneurysm, and (2) absolute numbers on per-aneurysm basis were reported and a complete 2×2 table could be constructed. Information about absolute numbers on per-patient basis was not a criterion for inclusion. If sensitivity and specificity were reported but the absolute numbers could not be derived, authors of the studies were contacted for further information.
Studies were excluded if (1) <5 patients were included, (2) the primary aim of the study was technical evaluation or image postprocessing, (3) duplicate report on (parts of) the same study population was present (in that case, the publication with the largest patient group was included), or (4) only a subgroup of patients had undergone the standard of reference.
Data Collection and Quality Assessment
Two reviewers (A.S. and B.W.) independently selected articles on the basis of title and abstract. If an article was considered eligible by ≥1 reviewer, the full article was evaluated by both reviewers. Furthermore, the 2 reviewers independently cross-checked the reference list from relevant review articles for additional studies, and expert recommendations for studies were considered.
Methodological quality of the studies was assessed by the Quality Assessment of Diagnostic Accuracy Studies tool-1 and -2 (QUADAS-1, QUADAS-2).7,8 The QUADAS-1 tool is an evidence-based quality assessment tool for diagnostic accuracy studies consisting of 14 quality items, with a score 14 being the maximum attainable. A revised tool, the QUADAS-2 tool, was developed in 2012. This tool comprises 4 domains: patient selection, index test, reference standard, and flow and timing, with each domain being assessed in terms of risk of bias and the first 3 in terms of applicability concerns. The maximum attainable score of QUADAS-2 is 7.
Among others, the following study data were extracted by means of a standardized data extraction form: study population, number of patients included and analyzed, prevalence of aneurysms, proportion of aneurysms ≤3 mm, MRA technique and field strength, location and size of aneurysms, and absolute numbers of true-positive, false-positive, true-negative, and false-negative MRA results on aneurysm level. We further evaluated whether observers could assess 3-dimensional (3D) reconstructions on a workstation (interactive, freehand) or whether the observers were provided with fixed reconstructions in standard planes (fixed). In case there was >1 observer per study, we extracted the absolute numbers from the best observer. If diagnostic performance of MRA and CTA among the same study population was analyzed, we also extracted the reported data for CTA to construct 2×2 contingency tables on a per-aneurysm basis. Quality scoring and data extraction were independently performed by 2 reviewers (A.S. and B.W). Disagreement was resolved by consulting a third reviewer (W.v.Z.) and by reviewers’ consensus.
The hierarchical summary receiver operating characteristic (ROC) model9 and the bivariate model10 were used for construction of a summary ROC curve and calculation of summary estimates of sensitivity and specificity. The models use a random-effects approach that assumes that variability between study results is because of not only random sampling error but also true differences between studies (such as population and thresholds). The summary ROC curve illustrates the trade-off between sensitivity and specificity at different thresholds for a positive MRA result. The average operating point on this curve indicates where the center of the study results lie. Summary estimates for sensitivity, specificity, and diagnostic odds ratio correspond with this operating point.
Heterogeneity of the study results was evaluated by calculation of the I2 statistic.11 This statistic estimates the percentage of total variation across studies that is attributable to differences between studies rather than chance. I2 values can vary between 0% (no heterogeneity) and 100% (100% variance because of heterogeneity). In addition, the magnitude of heterogeneity was depicted graphically by a 95% prediction region within which we have 95% confidence that the true sensitivity and specificity of a future study should lie.12
Subgroup analyses were performed to evaluate the effect of study characteristics, MRA and aneurysm characteristics, and image evaluation on heterogeneity and diagnostic performance. Differences in diagnostic performance of MRA between subgroups were tested for significance by adding a covariate (coding for subgroup) to the bivariate model.
The relationship between the diagnostic performance and sample size of individual studies was explored by the Deeks funnel plot asymmetry test.13 The diagnostic odds ratio is plotted against 1/root (ESS) as a measure for sample size. ESS stands for effective sample size, and 1/root (ESS) decreases with larger sample size. A regression line is fitted, and values of P<0.1 indicate that there is a nonzero slope coefficient (asymmetry), which implies that there is a relationship between sample size and magnitude of the diagnostic odds ratio. All data were analyzed with STATA 11 software (StataCorp LP, College Station, TX).
Study Identification and Study Characteristics
The literature search yielded a total of 6976 potentially relevant articles (Figure 1). After extensive review of these articles by 2 independent reviewers, 18 articles fulfilled the inclusion criteria of the search.14–31 Six articles were excluded because of duplicate publication of (parts of) the same study population.26–31 In total, 12 articles were included in the meta-analysis.14–25
The study characteristics are shown in Table 1. All included studies had a consecutive design and used DSA as the reference standard. The methodological quality of the included studies was high. Quality assessment scores ranged from 10 to 14 points for the QUADAS-1 tool and from 5 to 7 points for the QUADAS-2 tool. The mean quality score was 12 and 6 points, respectively. The majority of the studies (n=9) used time-of-flight (TOF) MRA technique, and 2 studies evaluated contrast-enhanced MRA (CEMRA). The study by Pierot et al22 evaluated both TOF and CEMRA in all patients, with CEMRA showing higher sensitivity. Therefore, CEMRA data were used for this analysis only.
Results of Meta-Analysis
Within the 12 studies that were included for meta-analysis, a total of 772 aneurysms in 960 patients were analyzed on a per-aneurysm level. Figure 2 shows a Forest plot with sensitivity and specificity per study. Heterogeneity on sensitivity and specificity was high, I2=96.11% (95% confidence interval [CI], 94.82%–97.40%) and 86.66% (95% CI, 80.28%–93.04%), respectively. Pooled sensitivity was 95% (95% CI, 89%–98%), and pooled specificity was 89% (95% CI, 80%–95%). Figure I in the online-only Data Supplement shows the sensitivity and specificity of individual studies in ROC space and the summary ROC curve with prediction and confidence contours. The large 95% prediction region also illustrates the large heterogeneity of study results.
Relationship Between Sample Size and Diagnostic Performance
Figure II in the online-only Data Supplement shows that there is trend toward higher diagnostic performance of studies with large sample size. The regression test is associated with a P value of 0.08.
Finally, we performed subgroup analyses on relevant extracted data. The results of these analyses are shown in Table 2. Studies that evaluated only ruptured aneurysms reported nonsignificant lower sensitivity and specificity compared with studies evaluating both ruptured and unruptured aneurysms. Diagnostic performance was significantly lower in studies with higher prevalence of aneurysms than in studies with lower prevalence, mainly because of lower specificity (P=0.011).
Studies using TOF MRA showed comparable diagnostic performance to studies performed with CEMRA. Among studies using 3-Tesla field strength, both sensitivity and specificity were higher compared with studies using lower field strength, and this difference was close to significance (P=0.054).
Diagnostic performance of studies with digital images was significantly higher than that of studies that used hard copy films (P=0.001). Heterogeneity decreased substantially after excluding the 2 studies in which image evaluation was performed on hard copy films. With respect to influence of freehand versus fixed reconstructions, studies in which radiologists performed freehand 3D reconstructions were associated with significantly higher diagnostic performance than studies where image reviewers were provided with fixed reconstructions in standard planes (P=0.029).
Studies with high methodological quality (QUADAS-1 score ≥13 and QUADAS-2 score=7) did not show significantly different diagnostic performance than studies with suboptimal scores.
False-Negative and False-Positive MRA Results
In total, 72 aneurysms (9% of all aneurysms) were missed by MRA. The size of the missed aneurysms was given in 67 cases: 30 (45%) of these 67 missed aneurysms were <3 mm, another 30 (45%) were between 3 and 5 mm in size, 4 (6%) were between 5 and 10 mm, and 3 were (4%) >10 mm. The location of the missed aneurysms was given in 67 cases. Twenty-seven (40%) of these aneurysms were located at the internal carotid artery. Excluding the study by White et al,24 5 of the reported missed aneurysms were located at the internal carotid artery. There were 43 false-positive findings on MRA. The size of the false-positive findings was specified in 22 cases: 18 (82%) of these were <3 mm, 2 between 3 and 5 mm, 1 between 5 and 10 mm, and 1 >10 mm. The location of the false-positive findings was specified in 33 cases (Table 3).
Comparison With CTA
Three of the included studies16,24,25 evaluated diagnostic performance of MRA and CTA among their study population, with observers blinded to the results of other imaging examinations. In total, 263 patients with 220 aneurysms underwent both noninvasive imaging tests. Comparison of CTA and MRA on a per-aneurysm basis is shown in Table 4.
We performed a systematic review and meta-analysis on studies evaluating MRA in the diagnosis of intracranial aneurysms. In 2000, the first, and until now, only meta-analysis on MRA was published by White et al.6 They reported per-aneurysm pooled sensitivity and specificity of MRA of 87% (95% CI, 84%–90%) and 95% (95% CI, 91%–97%), respectively. Now, 13 years later, the role of MRA in diagnosing intracranial aneurysms needed a re-evaluation. The results of this updated review, including studies that were published after 1998, show increased pooled sensitivity of 95% (95% CI, 89%–98%) and slightly decreased pooled specificity of 89% (95% CI, 80%–95%). Most individual studies reported high sensitivity, but a large variation was observed for specificity with a range from 50% to 100%. These results may be consistent with the need for high sensitivity to be able to rule out the presence of an aneurysm in patients with a negative MRA result. This aim to achieve high sensitivity seems to have resulted in higher rates of false-positive results in some studies, mainly lesions <3 mm.
We chose strict inclusion criteria, which were met by only 12 studies. In total, 52 studies were excluded because of incomplete or inappropriate reporting of patient data and study results, a lack of referral to the reference standard after negative MRA imaging results, or other limitations in study design. Our subgroup analysis showed slightly lower diagnostic performance among studies with optimal QUADAS scores compared with studies with lower scores, indicating possibility of overestimation of our pooled results. However, this difference was not statistically significant, and because overall quality of all included studies was high, we think that the data we report are valid.
In reviews of diagnostic studies, large differences are commonly noted between individual studies, indicating that there is heterogeneity between study results. The differences are too large to be explained by chance, implying that actual test accuracy varies between the included studies.32 This large heterogeneity in sensitivity and specificity estimates was also observed in this review, and subgroup analyses were performed in an attempt to identify potential sources of variation in study results. Three studies (Jäger et al,17 Kouskouras et al,18 and White et al24) produced outliers in terms of sensitivity or specificity. Two of these used analog image evaluation.17,24 After excluding the studies by Jäger and White that evaluated images on hard copy films, diagnostic performance increased and heterogeneity decreased. Five of the 7 missed aneurysms >5 mm came from these 2 studies. Because hard copy films are no longer used for image evaluation and technique and performance differ markedly from digital evaluation, decrease in heterogeneity was expected after exclusion. For the low specificity in the study of Kouskouras et al,18 we could not find a good explanation. However, this small study evaluated only 16 patients with 12 aneurysms.
We further want to highlight 2 important results from the subgroup analyses. First, studies performed on 3 Tesla showed higher diagnostic performance compared with studies on lower field strength. Although this difference did not reach statistical significance, overall diagnostic performance of MRA is expected to increase if 3 Tesla is to replace 1.5 Tesla as state-of-the-art field strength in the near future. No effects were observed on diagnostic performance because of CEMRA versus TOF technique. Therefore, 3 Tesla is the only relevant progression in MRA technique during the past 13 years. Second, we found that image after processing and evaluation technique had a remarkable influence on diagnostic performance, especially on pooled specificity. In studies wherein radiologists evaluated reconstructions on a 3D workstation allowing dynamic viewing from all possible directions, diagnostic performance significantly increased compared with studies in which radiologists were provided with fixed reconstructions in standard planes. In questionable lesions, use of 3D reconstruction software and freehand reconstructions is expected to improve the accuracy of evaluation, resulting in fewer false-positive results.
With regard to the relationship between diagnostic performance and study sample size, the Deeks funnel plot showed that there was a trend toward better results in larger studies. An explanation might be that in centers with large numbers of patients with stroke, radiologists might be more experienced with MRA in the detection of intracranial aneurysms and may, therefore, perform better in studies. The learning curve of experience has been described for CT angiography.33 There is no trend toward higher diagnostic performance in smaller studies, so the funnel plot gives no indication for publication bias.34
The implications of the results of this meta-analysis for the position of MRA in clinical practice need to be discussed. DSA remains the standard of reference because of its higher resolution but is invasive and time-consuming. An alternative noninvasive imaging modality is CTA, which is associated with radiation exposure and possible allergy to iodinated contrast material. MRA does not have these disadvantages, so the question needs to be answered whether diagnostic performance of MRA is comparable with that of CTA. Direct comparison of CTA and MRA did not show superiority of one test compared with the other. However, only 3 studies with ≈200 aneurysms provided direct comparison of CTA with MRA performance, being far too few to draw definite conclusions. Comparison of the pooled data in this meta-analysis with data from recent CTA meta-analysis indicates that sensitivity of MRA has become comparable with that of CTA in the diagnosis of intracranial aneurysms, but specificity of MRA seems slightly inferior compared with CTA, which showed pooled per-aneurysm sensitivity and specificity of 95.0% (95% CI, 93%–96%) and 96.2% (95% CI, 93%–98%), respectively.4 False-negative and false-positive findings on MRA were mainly located at the skull base, which is the same location known for misdiagnosis on CTA.3 Because the presence of overlying bony structures does not play a role in MRA, detection might mainly be hampered by complex vascular anatomy. In a clinical setting of acute nontraumatic subarachnoid hemorrhage, CTA has several advantages compared with MRA. CTA is fast, widely available, less susceptible to motion artifacts, and can be performed directly after subarachnoid hemorrhage is detected on noncontrast-enhanced CT. Furthermore, it has a proven high diagnostic performance.3,4 For MRA imaging, the patient would need to be transferred. MRA might not be available during night hours, might not be eligible in critically ill patients, and is inferior compared with CTA in terms of cost-effectiveness.35 Therefore, MRA would only be the preferred diagnostic tool if performance characteristics were substantially better compared with CTA. We, therefore, recommend that in an acute setting and in high suspicion of an intracranial aneurysm, MRA should only be used as primary diagnostic tool in patients with established adverse reaction on iodinated contrast agent, severely impaired renal function, or previous intracranial coiling.
For a screening setting, MRA might be first choice because of the absence of both radiation exposure and use of iodinated contrast agent. In such a setting, the prevalence of aneurysms is expected to be low. Because none of the included studies evaluated unruptured aneurysms only in a study population with low aneurysm prevalence, interpretation and transfer of results to a screening setting must be performed with caution. To rule out the presence of an intracranial aneurysm, a diagnostic test needs to have a high negative predictive value and high sensitivity. The high sensitivity that was achieved in the studies that were included in this review is promising. However, we could not find a good explanation for the overall high variation among studies on specificity. Pooled specificity of all included studies was 89%, ranging from 50% to 100%, which would result in a considerable high number of false-positively detected lesions requiring further evaluation.
Our meta-analysis focuses on MRA performance and did not evaluate any possible additional value of brain parenchyma imaging because this is hardly evaluated in literature to date. Nevertheless, MR imaging is known to be superior in terms of visualization of the parenchyma compared with CT, and the additional information on MR sequences might be a valid argument to rather perform MR imaging. Early and small areas of ischemia are difficult to detect on CT, even on CT perfusion, and are easily detectable on MR diffusion-weighted imaging sequences. Presence of ischemia could influence further treatment strategy because branches originating from an aneurysm leading to ischemic territories might not be spared. T2-weighted images can be helpful for the detection of completely thrombosed aneurysms and accurate sizing of partly thrombosed aneurysms.36 However, significant amount of thrombus would most probably also be visible on noncontrast-enhanced CT images. In our opinion, if relevant parenchymal pathology is suspected or other additional information about brain parenchyma is crucial, MR imaging including angiography sequences might be a justifiable first choice.
The results of this meta-analysis are consistent with an increase in diagnostic value of MRA over time. Pooled sensitivity is comparable with CTA in diagnosing intracranial aneurysms. Specificity of MRA seems to be slightly lower than the specificity of CTA as reported in recent meta-analysis of CTA but shows considerable variation. Studies on TOF MRA and CEMRA show comparable performance. Three-tesla MRA studies are associated with higher, although nonsignificant, diagnostic performance of MRA compared with studies that used lower field strength. Interactive evaluation with freehand 3D reconstructions significantly increases diagnostic performance.
Published as an abstract in the Radiological Society of North America (RSNA) meeting 2013
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.113.003133/-/DC1.
- Received August 6, 2013.
- Accepted October 23, 2013.
- © 2013 American Heart Association, Inc.
- van Rooij WJ,
- Sprengers ME,
- de Gast AN,
- Peluso JP,
- Sluzewski M
- Westerlaan HE,
- van Dijk JM,
- van Dijk MJ,
- Jansen-van der Weide MC,
- de Groot JC,
- Groen RJ,
- et al
- Higgins JP,
- Thompson SG,
- Deeks JJ,
- Altman DG
- Harbord RM,
- Deeks JJ,
- Egger M,
- Whiting P,
- Sterne JA
- Basiratnia R,
- Norouzi A,
- Hekmatnia A,
- Saboori M
- van Zwam WH,
- Hofman PAM,
- Kessels AG,
- Oei TK,
- Heijboer RJ,
- Wilmink JT
- White PM,
- Teadsale E,
- Wardlaw JM,
- Easton V
- Li H,
- Yan L,
- Li MH,
- Li YD,
- Tan HQ,
- Gu BX,
- et al
- Wu Q,
- Li MH
- Deeks JJ,
- Bossuyt PM,
- Gatsonis C
- Macaskill P,
- Gatsonis C,
- Deeks JJ,
- Harbord RM,
- Takwoingi Y
- Pedersen HK,
- Bakke SJ,
- Hald JK,
- Skalpe IO,
- Anke IM,
- Sagsveen R,
- et al
- Egger M,
- Davey Smith G,
- Schneider M,
- Minder C
- Adams WM,
- Laitt RD,
- Jackson A