Variation in Outcome After Subarachnoid Hemorrhage
A Study of Neurosurgical Units in UK and Ireland
Backgrounds and Purpose— The purpose of the study was to describe the characteristics, management, and outcomes of patients with confirmed aneurysmal subarachnoid hemorrhage and to compare outcomes across neurosurgical units (NSUs) in the UK and Ireland.
Methods— A cohort of patients admitted to NSUs with subarachnoid hemorrhage between September 14, 2001 and September 13, 2002 was studied longitudinally. Information was collected to characterize clinical condition on admission and treatment. Death or severe disability, defined by the Glasgow Outcome Score–Extended, was ascertained at 6 months.
Results— Data for 2397 patients with a confirmed aneurysm and no coexisting neurological pathology were collected by all 34 NSUs in the UK and Ireland. Aneurysm repair was attempted in 2198 (91.7%) patients (surgical clipping, 57.7%; endovascular coiling, 41.2%; other repair, 1.0%). Most patients (65.0%) were admitted to the NSU on the same day or the day after their hemorrhage; 32.0% of treated patients had the aneurysm repaired on the day of admission to the NSU (day 0), day 1 or day 2 and a further 39.3% by day 7. Glasgow Outcome Score–Extended at 6 months was obtained for 90.6% of patients (2172), of whom 38.5% had an unfavorable outcome. The median risk of an unfavorable outcome for all patients was 31% (5th and 95th percentiles, 12% and 83%), depending on prerepair prognostic factors. After adjustment for case-mix, the percentage of patients with an unfavorable outcome in each NSU did not differ significantly from the overall mean.
Conclusions— In this study that collected representative data from the UK and Ireland, there was no evidence that the performance of any NSU differed from the average.
In 1998, after incidents that undermined public confidence in the NHS,1,2 the UK government promoted clinical governance and national comparative audit.3 The concept of audit, typically by monitoring that care is provided in accordance with up-to-date guidelines, can present a challenge in surgical and other fields in which there is often a limited evidence base to underpin practice.4 In such cases, the public can be reassured by monitoring the patients’ health after treatment, showing that outcome does not differ markedly between hospitals.5,6
The Society of British Neurological Surgeons, in collaboration with the Clinical Effectiveness Unit of the Royal College of Surgeons, responded by initiating a national comparative study of the outcome of treatment for subarachnoid hemorrhage (SAH) in all neurosurgical units (NSUs) in the UK and Ireland. Similar studies have been performed in other surgical disciplines.7,8
Independent of the clinical governance agenda, there is a lack of nationally, or locally, representative information about variations in practice and outcome after treatment for a wide range of conditions. It is often surprising to the public that information of this kind is rarely available.1 This was true for SAH at the time of setting up this study. No British study had measured outcome after SAH on a national basis or attempted to compare outcomes between NSUs. Therefore, the study had the added benefit of providing high-quality descriptive information about the outcome of SAH after repair of an aneurysm.
This article has three objectives: (1) to characterize patients, their treatment, and their outcomes in a representative sample of patients receiving treatment for SAH in NSUs in the UK and Ireland; (2) to describe outcomes for patients with differing severity of physical signs on admission to a NSU; and (3) to compare average outcomes for NSUs after adjusting for variations in case-mix.
Materials and Methods
The National Study of SAH was designed as a prospective longitudinal cohort study of consecutive patients admitted to NSUs in the UK and Ireland after experiencing SAH. The study was approved by a UK multicenter research ethics committee.
Study Centers and Patients
All 34 NSUs in the UK and Ireland recorded data on patients admitted to NSUs from September 14, 2001 to September 13, 2002. Patients were eligible for inclusion if they had a SAH confirmed by a CT scan or a lumbar puncture. Patients with traumatic SAH (ie, SAH caused by head injury) and those aged younger than 16 years were excluded. Analyses in this article are restricted to patients with an aneurysm confirmed by digital, CT, or MR angiography.
Consultants were asked to complete data collection forms at the time of death or discharge and to send completed forms to the Clinical Effectiveness Unit of the Royal College of Surgeons for data entry. Data were collected on patients’ preoperative risk factors, clinical management, and hospital outcome. Preoperative risk factors included neurological condition on admission (World Federation of Neurological Societies grade), size and site of aneurysm, amount of blood detected on CT scan, and comorbid conditions. To avoid bias by omitting patients in poor clinical condition, the multicenter research ethics committee provided approval for the transmission of nonidentifiable data to the Clinical Effectiveness Unit of the Royal College of Surgeons without the need for patients’ consent.
Patient outcome was monitored for 6 months after discharge. Vital status was obtained by tracing patients in the UK population register through the Office for National Statistics. Functional status (a patient’s ability to perform activities of daily living after the hemorrhage compared to before the hemorrhage) was measured by the Extended 8-point Glasgow Outcome Score,9 administered by postal questionnaire to patients or their care-givers. If there was no response to repeat questionnaires, the Clinical Effectiveness Unit of the Royal College of Surgeons contacted the lead clinician at the NSU to provide a proxy assessment based on the most recent information available.
The Extended 8-point Glasgow Outcome Score classifies patients into 8 categories that correspond to a favorable (good recovery or moderate disability, scores 1–4) or unfavorable (severe disability or death, scores 5 to 8) outcome. The primary outcome was an unfavorable outcome at 6 months after discharge. Secondary outcomes, such as destination at discharge from NSU and 30-day mortality, were also collected.
Completeness of ascertainment of eligible patients was checked by research staff by visiting NSUs and searching hospital information systems (eg, NSU admission or theater logs) for the study period. Details of patients identified as potentially eligible but who were missing from the study were given to the NSUs for verification and, if patients were found to be eligible for inclusion, NSUs were encouraged to submit data retrospectively. The study also identified patients for whom data were missing or inconsistent and NSUs were asked to complete or verify the data. Finally, case notes of 10% of patients in each NSH were independently validated by Clinical Effectiveness Unit of the Royal College of Surgeons staff. Data were re-abstracted from the notes and compared with the original data submitted.
Patient characteristics, mode of repair, and outcomes were tabulated. Logistic regression was used to investigate the association of different patient case-mix characteristics with outcome in univariable and multivariable models. Missing case-mix data were coded and entered as a separate category, allowing all participants to be retained in the analysis. Robust standard errors were calculated to take into account clustering of patients within NSUs using STATA version 9 (Stata Corporation).
Both unadjusted and case-mix–adjusted logistic regression models were fitted (adjusted for age, sex, admission neurological condition, CT blood, size and site of aneurysm, comorbid conditions on admission, and prerepair deterioration). The probability of an unfavorable outcome for each patient was calculated from each model.
The ratio of the observed to expected number of unfavorable outcomes for each center was calculated with an exact confidence interval based on the Poisson distribution with and without case-mix adjustment. Without adjustment, the expected number was the number of eligible patients in a center multiplied by the proportion of patients with an unfavorable outcome in the whole study. With case-mix adjustment, the expected number of patients was the number of eligible patients in a center multiplied by the sum of the predicted probabilities of unfavorable outcome for individual patients in that center. All probability values are 2-sided, and P<0.05 was considered to be statistically significant.
Clinical data were received for 2397 patients with a confirmed aneurysmal SAH with no coexisting neurological pathology. Figure 1 shows a flow diagram of patients included in the study. Characteristics of patients are shown in Table 1, separately for patients who underwent repair (2198) of the aneurysm and those who did not (199). Compared to the repaired patients, patients in whom no repair was attempted were in a much poorer medical condition on admission as measured by the World Federation of Neurological Societies grade.
Based on hospital information systems for all but 6 NSUs, we estimated that data on 87.4% of all eligible patients were collected (median by NSU, 89.6%; interquartile range, 78.7% to 99.9%). After data cleaning, the average completeness of individual case-mix data items was 97.6%, with no data field having <90.0% completeness. Case note validation for 10% of patient notes found that submitted data agreed with re-abstracted data for >90% of items.
Losses to follow-up were <10% for the Glasgow Outcome Scale (favorable or unfavorable outcome) at 6 months. The percentage of patients with missing outcome varied considerably between centers (median, 4.8%; interquartile range, 2.6–17.6%) but did not vary by baseline characteristics.
Among patients in whom a repair was attempted (2198), 1269 (57.7%) were surgically clipped, 905 (41.2%) were coiled (endovascular occlusion), and a further 24 (1.1%) underwent other procedures such as wrapping the aneurysm with muslin or occlusion of the aneurysm with onyx glue. The majority of patients, whether the aneurysm was subsequently repaired, was admitted to the NSU on the same day or the day after their hemorrhage (65.0%).
Prerepair deterioration was reported in 528 patients (22%), caused by cerebral ischemia in 179, caused by hydrocephalus in 161, and caused by rebleeding in 142; in 195 patients it was caused by other complications (in the database, >1 reason for prerepair deterioration could be recorded). Of the patients who re-bled, 75 did not undergo repair. In all patients who underwent repair, 3.3% of patients were treated on the day of admission to the NSU (day 0), 12.4% on day 1, 16.3% on day 2, and nearly three quarters (71.3%) by day 7.
Overall, 11.3% patients died in hospital, 41.3% were discharged home, and 47.4% were discharged to the referring hospital or a rehabilitation unit (Table 2). Six months after discharge, 837 (38.5%) of patients had an unfavorable outcome (34.6% in repaired and 81.0% in unrepaired patients). The influences of prerepair characteristics on the odds of an unfavorable outcome at 6 months are shown in Table 3. World Federation of Neurological Societies grade on admission is most strongly associated with outcome, followed by preoperative deterioration. Patients’ risk for an unfavorable outcome varied considerably (median, 31.2%; 5th and 95th percentiles, 11.9% and 83.3%), depending on prerepair prognostic factors, ie, age, sex, presenting neurological condition (World Federation of Neurological Societies grade), amount of blood detected on the CT scan, site and size of aneurysm, preexisting comorbidities, and prerepair deterioration.
No significant difference in outcome was observed between patients who had their aneurysms clipped and those who had their aneurysms coiled (case-mix adjusted OR, 0.92, 95% CI, 0.74–1.15 for coiled compared with clipped aneurysms). The effect of timing of repair on outcome was investigated. In the multivariate risk-adjusted logistic regression model, no association between outcome and timing of repair (modeled as a continuous variable) was found. However, there was some evidence of an interaction between timing of repair and mode of repair (P=0.053); the risk of an unfavorable outcome decreased with increasing time from ictus for patients who had their aneurysms clipped but increased with increasing time from ictus for those who had their aneurysms coiled (not significantly).
Comparison of Outcomes by NSU
The distributions of predicted case-mix–adjusted probabilities of an unfavorable outcome for patients with confirmed aneurysmal SAH for each center are summarized in Figure 2 as box plots. The median probability ranged from 23% to 70% across NSUs. This Figure also shows that the median probability varied relatively little for the majority of NSUs (from 30% to 40% for the NSUs in the center of the distribution), ie, there was little case-mix variation between these NSUs. The same percentages apply to the time from ictus to repair.
Figure 3 shows both unadjusted and adjusted observed/expected ratios for NSUs, ordered according to their median unadjusted probabilities of an unfavorable outcome (as in Figure 2). First, this figure shows that confidence intervals for all NSUs include 1, ie, outcomes were consistent with expectation. Second, the difference between adjusted and unadjusted observed/expected ratios is greatest for NSUs with the most extreme median predicted case-mix–adjusted probabilities of an unfavorable outcome, ie, left and right ends of the figure. Third, confidence intervals are generally wide because the total numbers of patients, and of patients with unfavorable outcomes, were quite small (10–142 and 3–61, respectively). No relationship was found between the volume of patients recruited to the study in each center and outcome (supplemental Figure I, available online at http://stroke.ahajournals.org).
This study has described representative data about the characteristics of patients presenting for repair of a confirmed aneurysmal SAH, their management, and their health outcomes 6 months after discharge. Data were submitted for the majority of eligible patients and the primary outcome was determined for >90% of patients for whom data were submitted. Patients’ risk of an unfavorable outcome before repair varied considerably, but 86% were alive 6 months after discharge and 62% had a good recovery or only moderate disability. The average patient risk also varied across NSUs, although most of this variation arose from a minority of NSUs (Figure 2). Neurological condition on admission to the NSU and clinical deterioration before repair were the factors most strongly associated with an unfavorable outcome. After adjusting for case-mix, the proportions of patients with an unfavorable outcome in NSUs did not differ significantly from the study average.
Strengths and Weaknesses of the Study
We aimed to capture data for all patients admitted to NSUs with SAH during the study period. Despite limited funding and voluntary participation of staff in NSUs, we nearly achieved this aim; all NSUs and almost all consultant neurosurgeons managing SAH patients in UK and Ireland contributed data to the study. Data quality was validated by site visits, which ensured a high level of ascertainment of eligible patients in the study. The total number of patients recruited was consistent with expectation based on existing estimates of the population incidence of SAH.10,11 We remain uncertain about the exact number missed (estimated ≤12.6%), because we could not confirm the eligibility of those identified as possibly missing. The study was not able to collect data on patients who were admitted to District General Hospitals with SAH but who, for a variety of reasons, were not transferred to a neurosurgical unit. Thus, it does not reflect the whole population of SAH patients who survive the initial hemorrhage.
Data were not always collected at the time of discharge as requested in the protocol, and some were missing despite the process of feeding back data queries to NSUs. However, we have no evidence to suggest that recruitment or measurement of risk factors and outcomes were biased, ie, were influenced by patients’ characteristics or management. Based on other analyses,12 ascertainment and data quality depended only on the availability of staff time. Therefore, this limitation will have resulted in less precise, but not biased, estimates of average NSU outcome.
Compared to most observational studies, we established detailed methods for validating patients’ prerepair characteristics and management. The amount of missing outcome data is a greater concern and is higher than in well-resourced clinical trials.13 However, there was no evidence that the amount of missing outcome data was associated with patient characteristics or with average outcome for a NSU. Tracing patients through the Office for National Statistics meant that vital status at 6 months was known in 97% of patients and functional outcome was known for 91%.
Comparing average outcome between units of health care provision (hospital or consultant team) is complex (see Appendix). Intrinsic challenges include small sample sizes and rare outcomes, leading to inadequate power to detect important differences in performance; imperfect characterization of all aspects of outcome that are important to patients; and residual confounding from imperfectly measured case-mix and incomplete statistical risk adjustment. The problem of small sample sizes per center is not easily solved, especially in fields such as neurosurgery and neuroradiology, in which the numbers of procedures annually are small both per center and per operator. Averaging outcomes over longer periods of time may provide better precision, but raises the question of whether increasingly historical data should carry the same weight as more recent data. This is particularly true when there are rapid changes in clinical practice, such as a shift from clipping to coiling, as seen in this study. Therefore, the lack of significant differences in operative outcomes between NSUs in this study should not be assumed to provide reassurance. It is important to try to monitor continuously, to explore consistency of performance over time, to hypothesize about reasons for variation, and to test hypotheses using the available data. Performance estimates should be considered as the starting point for further investigation, rather than as the basis for decisions about health care provision or judgments about clinical governance.6,14,15
The importance of the study is demonstrated by the fact that no previous UK-wide study to our knowledge has characterized outcomes after SAH or compared outcomes between NSUs. This study means that it is now possible to provide patients and their families with empirical estimates of prognosis, based on their prerepair characteristics.
During the study period, the rate of coiling increased by >10%, from 35.2% in the first 6 months (September 2001–February 2002) to 46.1% in the latter 6 months (March–September 2002). The increase in the proportion of patients who had their aneurysms coiled may have occurred partly in response to cessation of recruitment to, and subsequent publication of results from, the International Subarachnoid Aneurysm randomised Trial (ISAT) of coiling vs clipping.13,16 Our study did not detect any difference in outcomes by mode of repair, unlike in the ISAT. However, estimates of effectiveness in ISAT and in our study are not equivalent and it is important to consider the limitations of treatment comparisons based on observational studies.17
Our study did not detect any differences in outcome by timing of repair, although there was a statistically significant interaction between timing and mode of repair. However, analysis of the effect of timing of repair in observational studies is problematic because the decision to treat patients early or late could have been influenced by many factors, including variables that were not collected and that therefore could not be controlled for in the analysis (eg, changes in clinical condition after admission, referrals to centers with coiling facilities).
Some patients may have not been treated early because of the length of time it took for them to present at an emergency department; almost 10% of patients were not admitted to any hospital in the first week after their hemorrhage. However, the majority (75.3%) was admitted to emergency departments on the same day as the hemorrhage, of which 46.7% were admitted to the NSU on the same day. After admission to the NSU, of those treated, 32.0% were treated within the first 2 days, and 71.3% within 1 week. The National Study of SAH did not have the resources to investigate reasons for delayed admissions.
Some delays may have been caused because of emergency departments in referring hospitals failing to initiate admissions to a NSU or because NSUs would not accept referrals. However, the same issues were identified in UK centers that participated in the ISAT and they are being studied in detail by the ISAT investigators with the aim of facilitating admission to NSUs. The main preventable consequence of delay in treatment of a ruptured cerebral aneurysm is rebleeding. This occurred in 5.9% of patients in the study and an unknown number who did not reach neurosurgical care. Because the outcome for such patients is grave, this may be the single most readily remedial factor in improving SAH care in the UK. There is no evidence from this study or unpublished data from ISAT (A. Molyneux, personal communication) to suggest that early repair worsens overall outcome. Thus, for patients in whom treatment is potentially feasible, emergency transfer to a neurosurgical unit with access to both clipping and coiling is likely to provide the best care.
Implications for Future Representative Studies of Health Outcomes
Collecting high-quality data for a study of this kind is both critically important and demanding. This study required the collection of information, often duplicating the recording of identical or similar information in patients’ case notes. Modernization of information technology in health services should eliminate both the need for duplication of data collection and this source of transcription errors. It may also allow the more complete ascertainment of the condition by identifying patients who are not, for whatever reason, transferred to tertiary care. Obtaining consent (a requirement for research) in populations such as these in which a proportion of patients cannot be consented because of neurological condition on admission presents problems. Some institutional review boards/ethics research committees allow data collected routinely in the course of health care provision to be used anonymously for research purposes. Modernization of information technology may solve this problem by making data collection a by-product of the delivery of care.
Collecting high-quality information about health outcomes after treatment (preferably from patients themselves) will always be resource-intensive compared to relying on routinely collected clinical outcomes (often short-term) or survival. With increasing time after an event or intervention, more participants become lost to follow-up, making analyses susceptible to attrition bias (informative censoring). The estimation of both prognosis and performance requires outcomes that are meaningful and relevant to patients. Choosing to use routine data only (the easy option, both practically and scientifically) at best presents an incomplete picture and at worst misleads.
In conclusion, this nationwide study has described representative data about the characteristics of patients presenting for treatment of a confirmed aneurysmal SAH, their management, and their health outcomes 6 months after discharge. Neurological condition on admission to the NSU and clinical deterioration before repair were the factors most strongly associated with an unfavorable outcome. No individual NSU differed significantly from the national average after adjusting for case-mix.
Justification of Methods
Choices in the methods of analysis, and the effects of these choices, are not often described. We intend that the approach taken in this article (considering variation between NSUs as “fixed effects”) should first and foremost be understandable. Some statisticians argue that this approach does not represent the data in the most plausible manner. Multi-level or Bayesian analyses (considering variation between NSUs as “random effects”), which make more explicit assumptions about the likely prior distribution of average performance estimates between the units being compared, address this criticism but tend to widen confidence intervals and shift performance estimates toward performance consistent with expectation, especially when data are sparse.
Sources of Funding
The National Study of Subarachnoid Haemorrhage was initially funded for 1 year by the Department of Health through the Academy of Medical Royal Colleges’ Outcome Initiative. Additional funding for the project coordinator (J.L.) was provided through an MRC Training Fellowship for Health Services Research. The funding sources did not have any input into the study design, data collection, analysis, and interpretation of data, the writing of the report, and the decision to submit the paper for publication.
A.J.M. is a consultant to Micrus Corp, Inc, and has a stockholding in the company. His unit has also received research support from Boston Scientific, Inc. There are no other conflicts to report.
- Received February 18, 2008.
- Accepted June 15, 2008.
Bristol Royal Infirmary I. Learning from Bristol: the report of the public inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984–1995. http://www.bristol-inquiry.org.uk/. Accessed 2001.
Shipman Inquiry: The First Report. http://www.the-shipman-inquiry.org.uk/reports.asp. The Stationary Office. London, UK. Accessed 2002.
Department of Health. A first class service: quality in the new NHS. London; 1998.
Goldstein H, Spiegelhalter DJ. League tables and their limitations: statistical issues in comparisons of institutional performance. J R Stat Assoc A. 1996; 59: 385–443.
Dawwas MF, Gimson AE, Lewsey JD, Copley LP, van der Meulen JH. Survival after liver transplantation in the United Kingdom and Ireland compared with the United States. Gut. 2007; 56: 1606–1613.
Linn FH, Rinkel GJ, Algra A, van Gijn J. Incidence of subarachnoid hemorrhage: role of region, year, and rate of computed tomography: a meta-analysis. Stroke. 1996; 27: 625–629.
de Rooij NK, Linn FH, Van der Plas JA, Algra A, Rinkel G. Incidence of subarachnoid haemorrhage: a systematic review with emphasis on region, age, gender and time trends. J Neurol Neurosurg Psychiatry. 2007; 78: 1365–1372.Epub 2007 Apr 30.
The Steering Group for The National Study of Subarachnoid Haemorrhage. National Study of Subarachnoid Haemorrhage, Final Report: February 2006. http://www.rcseng.ac.uk/rcseng/content/publications/docs/nat_study_subarachnoid_haem_feb2006.html. 2006.
Molyneux A, Kerr R, Stratton I, Sandercock P, Clarke M, Shrimpton J, Holman R; International Subarachnoid Aneurysmal Trial Collaborative Group. International Subarachnoid Aneurysm Trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised trial. Lancet. 2002; 360: 1267–1274.
Mant J. Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care. 2001; 13: 475–480.
Molyneux AJ, Kerr RS, Yu LM, Clarke M, Sneade M, Yarnold JA, Sandercock P; International Subarachnoid Aneurysmal Trial Collaborative Group. International subarachnoid aneurysm trial (ISAT) of neurosurgical clipping versus endovascular coiling in 2143 patients with ruptured intracranial aneurysms: a randomised comparison of effects on survival, dependency, seizures, rebleeding, subgroups, and aneurysm occlusion. Lancet. 2005; 366: 809–817.