Sensitivity of Early Brain Computed Tomography to Exclude Aneurysmal Subarachnoid Hemorrhage
A Systematic Review and Meta-Analysis
Background and Purpose—Emerging evidence demonstrating the high sensitivity of early brain computed tomography (CT) brings into question the necessity of always performing lumbar puncture after a negative CT in the diagnosis of spontaneous subarachnoid hemorrhage (SAH). Our objective was to determine the sensitivity of brain CT using modern scanners (16-slice technology or greater) when performed within 6 hours of headache onset to exclude SAH in neurologically intact patients.
Methods—After conducting a comprehensive literature search using Ovid MEDLINE, Ovid EMBASE, Web of Science, and Scopus, we conducted a meta-analysis. We included original research studies of adults presenting with a history concerning for spontaneous SAH and who had noncontrast brain CT scan using a modern generation multidetector CT scanner within 6 hours of symptom onset. Our study adheres to the preferred reporting items for systematic reviews and meta-analyses (PRISMA).
Results—A total of 882 titles were reviewed and 5 articles met inclusion criteria, including an estimated 8907 patients. Thirteen had a missed SAH (incidence 1.46 per 1000) on brain CTs within 6 hours. Overall sensitivity of the CT was 0.987 (95% confidence intervals, 0.971–0.994) and specificity was 0.999 (95% confidence intervals, 0.993–1.0). The pooled likelihood ratio of a negative CT was 0.010 (95% confidence intervals, 0.003–0.034).
Conclusions—In patients presenting with thunderclap headache and normal neurological examination, normal brain CT within 6 hours of headache is extremely sensitive in ruling out aneurysmal SAH.
Headache accounts for ≈2% of all emergency department (ED) visits.1 A subset of these patients present with abrupt onset of a severe headache reaching peak intensity within 60 s referred to as a thunderclap headache.2 The most serious cause of thunderclap headache is aneurysmal subarachnoid hemorrhage (SAH), which accounts for 4% to 12% of ED patients with a thunderclap headache.3–6 Current clinical practice calls for a noncontrast computed tomography (CT) of the brain followed by a lumbar puncture (LP) if the CT scan is negative to exclude SAH.7–10 This is because the sensitivity of CT scans for detecting subarachnoid blood ranges from 90% to 100% when performed within the first 24 hours after symptom onset. The sensitivity decreases as time from onset to CT elapses because the blood is progressively diluted by the normal flow of cerebrospinal fluid.6,11–16
Recent data suggest that in neurologically intact patients, the sensitivity of modern CT scanners for SAH approaches 100% when performed within 6 hours of headache onset and interpreted by qualified radiologists.11,13,17–20 These data suggest that in this early-presenting population, an LP is not necessary to rule out SAH and an initial negative CT can be considered a rule-out test. An LP is associated with patient anxiety and discomfort and can be complicated by postprocedure headache (15%–20% of patients).21 Traumatic taps, which occur in 10% to 15% of patients, may lead to unnecessary vascular imaging and other downstream consequences.3,20–22
We conducted a systematic review and meta-analysis to determine the diagnostic accuracy of early CT only in the diagnosis of spontaneous SAH. Our objective was to determine the sensitivity, specificity, and positive and negative likelihood ratios (LRs) of a brain CT performed within 6 hours of headache onset using modern generation scanners in the diagnosis of spontaneous SAH.
This was a systematic review and meta-analysis, and it adheres to the preferred reporting items for systematic reviews and meta-analyses (PRISMA).23
We included original research studies of adults with a history concerning for spontaneous nontraumatic SAH and evaluated with noncontrast brain CT scan using modern generation multidetector scanners (16-slice CT technology or greater) within 6 hours of headache onset. Studies involving traumatic SAH, patients younger than 15 years of age, nonhuman studies, older generation scanners, and those in which CT was not performed within 6 hours of headache onset were excluded.
An expert librarian designed a comprehensive search strategy with input from the authors. The electronic search included Ovid MEDLINE, Ovid EMBASE, Web of Science, and Scopus from inception (Ovid MEDLINE and Scopus 1966, Ovid EMBASE 1988 and Web of Science 1975) until April 2015. See Appendix I in the online-only Data Supplement for the terms used in the search. We adjusted the search strategy to account for differences in indexing between databases. Web of Science and Scopus depend heavily on text words, so acronyms were included. We did not apply a language restriction. We also reviewed the related citations: section of PubMed, reference lists of included studies, and the authors’ personal collections.
Two investigators (N.M.D. and A.A.R.) independently screened the titles and abstracts of all records identified from the search strategy (phase I). If either reviewer thought the study might be eligible, we obtained the full report. The same 2 investigators then independently assessed the eligibility of each full report (phase II). We used Cohen unweighted κ to measure chance corrected agreement between reviewers. Discrepancies were resolved by a third author (J.A.E.).
Quality Assessment and Risk of Bias
Data on study quality and risk of bias were abstracted for each study by 1 author (M.F.B.). We assessed the quality of studies of diagnostic accuracy with the revised Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2) tool.24
Two authors (N.M.D. and A.A.R.) independently extracted data from each included article using a standardized data extraction form. We extracted the following data from each study: design, patient demographics, definition of SAH, CT technology, type of radiologist interpreting the CT, clinical setting, number of patients with SAH, and number of missed cases of SAH. When possible, we collected data to construct a 2 by 2 table, including true negatives, true positives, false negatives, and false positives. When data were not sufficiently reported, we sought other sources of information, such as letters to the editors, authors’ reports, and personal e-mail to the authors to acquire missing information.
Diagnostic accuracy measures were pooled using random-effect meta-analysis25 as implemented in OpenMeta[Analyst]26 and tested in a bivariate mixed effects regression model.27 We used a random effects model because it calculates more conservative 95% confidence intervals (CI) and the effects of treatment are assumed to vary around the overall average treatment effect. This is recommended when data are heterogeneous. Results are presented as incidence per 1000 patients and we calculated pooled sensitivity, specificity, LR of a positive and a negative test with 95% CI. LR is the likelihood that a given test result would be expected in a patient with the target disorder (SAH) compared with the likelihood that that same result would be expected in a patient without the target disorder (SAH). It is a different way to incorporate sensitivity and specificity and provide a direct estimate of how much a test result will change the odds of having the disease. LR equals sensitivity/(1 specificity), and the LR of a negative test indicates how much the odds of the disease (SAH) decrease when the CT is negative.
The sensitivity, specificity, and LRs are properties of the test. The positive and negative predictive values are properties of both the test and the population being tested. The predictive value of a test in 2 populations with different disease prevalence will be different.
When a cell has zero count in the 2 by 2 tables, the statistical software will correct adding +0.5 count to all the cells. Meta-analysis heterogeneity was assessed using the I2 statistic.28
We performed an a priori selected sensitivity analysis to exclude the studies with data obtained through letters to the editor and communication with the authors.
Description of Included Studies
Figure 1 shows the study selection process. The search strategy yielded 882 articles. After screening titles and abstracts and removing duplicates, we identified 40 potentially relevant studies. Two authors (N.M.D. and A.A.R.) abstracted data independently and in duplicate. Interobserver agreement for phase II of the review was 87.5% (κ, 0.64; 95% CI, 0.36–0.91) indicating good agreement between reviewers. After full-text review, 5 articles were included in the meta-analysis. The reasons for exclusion after full-text review were that the article did not specify data for patients imaged within 6 hours, the article was not an original study, and the article did not pertain to SAH.
The characteristics of the included studies are shown in Table. Four were cohort studies that reported diagnostic test accuracy, and one was a case-control study. Four had retrospective design13,18,20,29 and one was prospective.11 We estimated that a total of 8907 patients underwent CT within 6 hours. See Table I in the online-only Data Supplement. The mean age of the patients included was 45.3 years (range, 15–87 years) and 60.6% were women.
Quality and Risk of Bias Assessment
Table II in the online-only Data Supplement summarizes the risk assessment using the QUADAS-2 tool for the QUADAS. Overall, there was considerable heterogeneity between studies. There were similarities about the clinical characteristics of included patients: acute headache, normal mental status, no neurological deficit, and similar age and sex distribution. There was significant variation in the incidence of SAH among the studies (Table). Perry et al11 had low bias risk in the applicability of their study, as they included all SAH-suspected patients presenting to an ED.
The index (gold standard) test was a validated method for diagnosis of SAH including CT or cerebrospinal fluid analysis and clinical follow-up. The index test was applied unevenly across the 5 studies, which could have introduced bias.
The studies by Backes et al,13 Stewart et al,18 Blok et al,20 and Mark et al29 used medical records review for ascertainment of the cases and follow-up of the cohorts. In the study by Perry et al,11 patients were identified prospectively the day of the ED visit and then followed by telephone, medical records, review of regional center records, and coroner reports. Any patient who later was diagnosed with an SAH (and survived) would have been transferred to the single regional neurosurgical referral unit. Therefore, unless a patient had a subsequent SAH outside of Ontario, the diagnosis would have likely been captured. We thus considered that the reference standards used in all the studies were appropriate and reproducible.
The studies by Blok et al20 and Mark et al29 only included patients with negative CTs, so we estimated their true positives and negatives. Mark et al29 reported 55 patients with SAH and negative CT and true positives as 1800 patients,30 with ≈30% having a CT within 6 hours and 11 missed cases of SAH, including 7 patients who had vascular anomalies on cerebral angiography. From the data available, one cannot know if even these 7 patients had true SAH or a thunderclap headache and an incidental vascular lesion on imaging.
The study by Blok et al20 reports patients with acute headache, and negative CTs per staff radiologist; an LP was performed in all cases. Among the 760 patients with negative CTs, 52 had cerebrospinal fluid positive for bilirubin and only 1 of these was an SAH (a nonaneurysmal, perimesencephalic SAH diagnosed by review of the original CT, which had initially been reported as negative). They did not report the overall incidence of SAH in the cohort so a 2 by 2 table could not be built. We contacted the senior author of this study who communicated that data on SAH incidence or true positive rates were not available in their cohort.
The case definition of a study will greatly influence the incidence. Because Perry et al11 is the only prospective study performed in EDs, we feel it has the highest potential to be replicated and generalizable. Using the incidence of the Perry et al11 study (12.7% in the early-presenting group) and the proportion of SAH patients who presented within 6 hours (30%), we estimated a 2 by 2 table for the Blok et al20 study. Blok et al20 reported 260 cases of SAH per year and their study period was 6 years, which calculates to 469 SAH among 3600 patients.
When all 5 studies11,13,20,29 were pooled together, we estimated that in the worst-case scenario, 13 of the 8907 patients who underwent CT within 6 hours had a missed SAH (incidence 1.46 per 1000). Overall sensitivity of the CT was 0.987 (95% CI, 0.971–0.994) and specificity was 0.999 (95% CI, 0.993–1.0); Figure 2. The pooled LR of a positive CT was 921.9 (95% CI, 139–6103) and pooled LR of a negative CT was 0.010 (95% CI, 0.003–0.034); Figure 3; Table I in the online-only Data Supplement.
When the study by Mark et al29 is added with 7 missed cases instead of 11 (those with vascular anomalies on angiography, an intermediate case scenario), the pooled 6-hour sensitivity is 0.989 (95% CI, 0.980–0.994) and the pooled specificity is 1.0 (95% CI, 0.993–1.0).
When only the 3 studies11,13,18 that provide direct information in their 2 by 2 tables are included, the pooled incidence of SAH was 19.1%. One of the 1155 patients who underwent CT within 6 hours of headache onset had a missed SAH. This results in an incidence of missed SAH of 0.87 per 1000 with CT within 6 hours of headache onset. The overall sensitivity of the CT in the 3 studies was 0.986 (95% CI, 0.951–0.996), I2 0%, specificity 0.996 (95% CI, 0.974–0.999), I2 28.0%.
We found that the CT miss rate of SAH when performed within 6 hours of the onset of headache was <1.5 in 1000 patients. The sensitivity of the CT was 99% and the LR of a negative CT was 0.010. These results suggest that a negative CT within 6 hours may be considered sufficient to rule out SAH in the following circumstances: a neurologically normal patient, a thunderclap headache presentation, a clear time of onset, and a modern CT scan performed within 6 hours of onset read by an attending radiologist.
Our analysis does not apply to patients who present with atypical features (eg, primary neck pain, syncope, or seizure) or any new finding on neurological examination. Such patients do not meet entry criteria for this particular study and the extremely high sensitivity demonstrated here may not apply in these populations.
There are several ways to interpret the data as reported by Mark et al.29 The most conservative approach is to assume that all 11 cases were true missed SAH (worse case scenario). An intermediate approach would be to assume that only the 7 cases that had associated vascular lesions found were true SAH (intermediate case scenario). Of course, it is possible that even these 7 cases were instances of patients with thunderclap headache and an incidental vascular lesion. Thus to have a conservative approach, we included the study by Mark et al29 with 11 missed cases (worst-case scenario). It is important to note that for the studies of Mark et al29 and Blok et al20 because of the way the data were reported in the articles, we had to estimate certain values to be able to construct a 2 by 2 table using a single prospective study.11 We think this study is the one that best reflects the patient population to which we will be applying the results of this meta-analysis, as it included a larger cohort and it was a nonreferral population.
In addition to these 5 eligible studies, other studies also support the accuracy of CT when performed early after headache onset.12,14,17,29,31,32 Please see Table III in the online-only Data Supplement. Sidman et al14 found CT to be 100% sensitive for diagnosing spontaneous SAH if performed within the first 12 hours but did not specify how many of these patients were imaged in the first 6 hours.19 Bakker et al17 reported that 94 of 1448 consecutive patients with known SAH were CT negative but LP positive. Of the 12 patients who underwent CT within 6 hours, none had a vascular lesion. Of note, this study defined a positive LP as the presence of bilirubin by spectrophotometry (ie, xanthochromia), which is known to be sensitive but lacks specificity.33,34
In an ED population of patients with isolated thunderclap headache who present early enough to undergo CT within 6 hours of symptom onset, the incidence of SAH is reported ≈13% (higher than in patients with thunderclap headache who present later).11 After a negative CT within 6 hours, the post-test probability decreases to ≤0.2%. The results of our analysis indicate that if one applies this 6-hour rule for CT to diagnose SAH, the worst-case miss rate will be 1 to 2 cases per 1000. The harm from missing these cases must be balanced against the potential consequences of routine LP including time, procedure-related pain, anxiety and complications of LP, unnecessary vascular imaging in the roughly 10% to 15% that have traumatic LPs, and most importantly, the downstream consequences—procedural risks and complications in patients who undergo treatments of incidental vascular lesions, and patient anxiety that having an aneurysm engenders and follow-up imaging for those who do not.20
If one were to eliminate the requirement for LP, several important considerations apply (Figure 4). First, our analysis refers only to SAH. Thunderclap headache has a differential diagnosis; if the clinical presentation or epidemiological context suggests another non-SAH diagnosis, further testing beyond CT may be indicated.2 Second, the sensitivity of CT in this group of patients depends on factors related to the CT scan and its interpretation. In the studies by Perry et al,11 Stewart et al,18 and Blok et al,20 general attending level radiologists read most of the CTs and in the Backes et al13 study, neuroradiologists interpreted the scans. Trainees and nonexperts have a higher rate of errors in interpretation.35 In the study by Perry et al,11 there were 4 instances of scans read as negative by emergency physicians or radiology trainees, and all subsequently read correctly as positive by the attending radiologist.11 In the studies by Mark et al29 and Blok et al,20 some of the scans initially read as negative by general radiologists were later over read as positive.20,36,37 It is therefore critical that individuals experienced in reading brain CTs interpret the scan and that the clinician clearly communicates the indication for the scan to the radiologist.
Our analysis has several limitations. First, we included only studies involving ED patients presenting with complaints concerning for nontraumatic SAH and with CT scans performed within 6 hours. Because of the clinical heterogeneity of studies pertaining to this topic, only 5 were ultimately included in our meta-analysis. Although the number of included studies is small, we are confident that we included all pertinent studies given the rigor of our search strategy. Second, in the study that contributed the largest number of patients, LP was not performed in all patients, which could have led to overestimation of CT sensitivity.11 However, the nature of the follow-up in that study (telephonic follow-up, ability to gather information from regional health and coroner records and the fact that the area contains a single regional neurosurgical center) makes this possibility unlikely. Third, the included studies had methodological heterogeneity and have incidences of SAH that seem higher than what is typically seen in clinical practice. Fourth, we recommend to the readers to be careful when evaluating heterogeneity of diagnostic test accuracy reviews relying solely with the I2, as the included studies differ in the selection of their cohorts and incidences. Finally, the way outcomes were defined and measured (CT only versus CT plus LP) were different. Despite these differences in the definition of the outcomes, we did not see differences in the observed intervention effects. Applying these results to a population with lower prevalence than the one of the included studies increases the negative predictive value, meaning a negative CT is more likely to be a true negative.
In patients presenting with thunderclap headache and a normal neurological examination, a negative brain CT scan within 6 hours of headache onset is highly sensitive in ruling out aneurysmal SAH when the CT scan is technically adequate, and it is interpreted by an experienced radiologist.
The authors thank Ms Patricia Erwin for the literature search.
Dr Edlow gives expert testimony for cases of neurological emergencies for both plaintiff and defense firms.
Guest Editor for this article was Eric E. Smith, MD.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.115.011386/-/DC1.
- Received September 10, 2015.
- Revision received December 7, 2015.
- Accepted December 14, 2015.
- © 2016 American Heart Association, Inc.
- Goldstein JN,
- Camargo CA Jr,
- Pelletier AJ,
- Edlow JA.
- Edlow JA,
- Panagos PD,
- Godwin SA,
- Thomas TL,
- Decker WW
- Connolly ES Jr,
- Rabinstein AA,
- Carhuapoma JR,
- Derdeyn CP,
- Dion J,
- Higashida RT,
- et al
- Perry JJ,
- Stiell IG,
- Sivilotti ML,
- Bullard MJ,
- Emond M,
- Symington C,
- et al.
- van der Wee N,
- Rinkel GJ,
- Hasan D,
- van Gijn J.
- Backes D,
- Rinkel GJ,
- Kemperman H,
- Linn FH,
- Vergouwen MD.
- Fishman RA.
- Bakker NA,
- Groen RJ,
- Foumani M,
- Uyttenboogaart M,
- Eshghi OS,
- Metzemaekers JD,
- et al.
- Stewart H,
- Reuben A,
- McDonald J.
- Wallace BC,
- Dahabreh IJ,
- Trikalinos TA,
- Lau J,
- Trow P,
- Schmid CH.
- Higgins JP,
- Thompson SG,
- Deeks JJ,
- Altman DG.
- Mark DG,
- Hung YY,
- Offerman SR,
- Rauchwerger AS,
- Reed ME,
- Chettipally U,
- et al.
- Perry JJ,
- Sivilotti ML,
- Stiell IG,
- Wells GA,
- Raymond J,
- Mortensen M,
- et al.