Quality of In-Hospital Stroke Care According to Evidence-Based Performance Measures
Results From the First Audit of Stroke, Catalonia (Spain) 2005/2006
Background and Purpose— Evidence-based standards are used worldwide to determine quality of care. We assessed quality of in-hospital stroke care in all acute-care hospitals in Catalonia by determining adherence to 13 evidence-based performance measures (PMs) of process of care.
Methods— Data on PMs were collected by retrospective review of medical records of consecutive stroke admissions (January to June, 2005). Compliance with PMs was calculated according to 3 hospital levels determined by their annual stroke case-load (level 1, <150 admissions/yr; level 2, 150 to 350; and level 3, >350). We defined sampling weights that represented each patient’s inverse probability of inclusion in the study sample. Sampling weights were applied to produce estimates of compliance. Factors that predicted good/bad compliance were determined by multivariate weighted logistic regression models. An external monitoring of 10% of cases recruited at each hospital was undertaken, after random selection, to assess quality of data.
Results— We analyzed data from 1791 stroke cases (17% of all stroke admissions). Global interobserver agreement was 0.7. Eight PMs achieved compliances ≥75%, 4 of which were more than 90%, and the remaining showed adherences ≤62%. Analysis of compliance across hospital levels displayed some significant differences that persisted after multivariate analysis. We observed lower adherences to “early mobilization,” “assessment of rehabilitation needs,” and “prescription of anticoagulants for atrial fibrillation” in females and in the elderly.
Conclusions— In 2005, in-hospital stroke care in Catalonia was heterogeneous across hospital levels. Rehabilitation-related measures showed poor compliances compared to acute care-related ones, which achieved more satisfactory adherences.
The development of systems and tools for monitoring the quality of hospital-based stroke services has drawn considerable attention in the last years as concern has been expressed about differences and variations in the standards of care across states, regions, and countries. A good example of this is the UK initiative of monitoring the quality of stroke care and services through a series of national audits commissioned by the National Health System.1–5 In line with the UK stroke audits, other experiences have been reported.6–8 In addition to the need of monitoring quality of stroke services, it is crucial to ensure that knowledge from recent research studies is made available to everybody and is definitely translated into clinical practice. Clinical practice guidelines (CPG) may represent the standardization of processes and interventions for a particular condition, and are aimed at reducing variations in medical practice in order to guarantee an optimum level of quality and improve health care.
In Spain, health care competencies are decentralized and transferred to the Autonomous Communities that are responsible for management and distribution of the health care–related budget. In Catalonia (capital: Barcelona, population 2008: 7.5 million inhabitants, area: 32 000 squared Km), stroke causes more than 12 000 hospital admissions per year, representing 2% of all admissions to acute-care hospitals.9 It is the leading cause of death among women and the second one among men, and it is the most common cause of nontraumatic disability.10,11 In Catalonia, acute stroke care is provided through a network of 48 publicly financed acute-care hospitals distributed across the territory according to demographic criteria: larger hospitals are located in and around Barcelona, whereas middle- and small-sized hospitals cover the rest of the territory where demographic density is much lower.
The magnitude of stroke figure led to the creation of a Stroke Programme (SP) in 2004. As a part of a comprehensive Stroke Strategy, initial objectives of the SP included the assessment of stroke care quality across the territory before and after the release of local stroke CPG, which was made available in November 2005.12 The aim of this study was to assess the quality of in-hospital stroke care in all acute-care hospitals of the Catalan network of public hospitals by determining the adherence to 13 evidence-based performance measures (PM), and to analyze whether hospital size did have any influence on the adherence to these PM. This study is the first part of a larger project where we hypothesize that the release and implementation of the Catalan Stroke CPG, along with territory-based interventions delivered by the SP, will help to reduce variations in stroke care across the country.
Sampling and Data Collection
We retrospectively studied consecutive stroke patients admitted to all acute-care hospitals belonging to the Catalan network of publicly financed hospitals (n=48). Stroke cases were defined by the following ICD-9 diagnostic codes: 431 (intracerebral hemorrhage), 433.01 (occlusion and stenosis of precerebral arteries with cerebral infarction), 434 (occlusion of cerebral arteries), and 436 (acute, but ill-defined, cerebrovascular disease). Patients with transient ischemic attacks and subarachnoid hemorrhages were not included. Participating hospitals were categorized into 3 different levels according to the annual number of stroke admissions shown by the Register of the Minimum Basic Data Set for Acute-care Hospitals (CMBD-AH, Catalan acronym for Hospital Discharges Database) for 2005: level 1, <150 stroke admissions; level 2, 150 to 350 admissions; and level 3, >350 admissions. The main features of each hospital category are described in Table 1. Hospitals recruited 20, 40, or 60 stroke cases depending on their level (1, 2, and 3, respectively). These cases corresponded to consecutive stroke admissions from the first of January 2005 onward until the completion of the number of cases appointed to each hospital, and were restricted to those admitted within January to June, 2005 to avoid any possible effect of the CPG on the stroke care. The sample represented approximately a 17% of all stroke admissions in 1 year (overall population: 10 435 stroke admissions), and was representative of stroke patients admitted to each participating hospital.
All trusts appointed a member of the stroke team as the local investigator, who was in charge of identifying the local sample of stroke patients from the archives of the hospital and of data collection. Data were retrospectively collected (March to June, 2006) from the patient’s case note to fulfill an electronic questionnaire accessed via the website of the Catalan Ministry of Health. The electronic database satisfied legal requirements for protection of personal data. An external audit consisting of double data collection and entry of approximately 10% of cases recruited at each study site, and selected randomly from the trust patients’ list, was undertaken to check quality of data.
Audit Tool: Selection of Evidence-Based Performance Measures
After the completion of the Catalan Stroke CPG, we requested the members of the CPG Board and the Standing Commission of the SP to list relevant recommendations that represented quality standards of stroke care from a pool of more than 250 included in the CPG. Relevance was defined as either clinical or scientific relevance. The latter had been defined in the Catalan Stroke CPG according to the Scottish Intercollegiate Guidelines Network classification of evidence. This initial step returned 43 measures that were grouped into 6 dimensions of care: quality of medical records (12 PMs), initial interventions (4 PMs), neurological assessment (12 PMs), assessment of rehabilitation needs (6 PMs), management of medical complications (4 PMs), and initial preventive measures (5 PMs). A subsequent selection was undertaken to choose, from those initial 43 measures, a reduced set of both clinically and scientifically relevant measures. Thus, we identified 13 PMs distributed across the aforementioned dimensions. The level of evidence and grade of the recommendations supporting these 13 evidence-based PM (the “top 13”) were distributed as follows: 8 grade A recommendations, 3 grade B recommendations, and 2 grade C recommendations (determination of “baseline glycemia” and “baseline blood pressure (BP)” were selected mostly because of their clinical relevance).
On the basis of the sampling design (20, 40, or 60 cases depending on the hospital level/annual number of stroke admissions), and using the number of admissions during 2005 by hospital level, we defined sampling weights that represented each patient’s inverse probability of inclusion in the study sample. We applied the sampling weights to produce territory-representative estimates of compliance with the PM.
For each hospital level we calculated the mean percent compliance and 95% confidence intervals for each PM. For each estimate, the numerator was the number of patients with documented PM compliance and the denominator was the number of valid cases for the measure being evaluated. Failure to document the provision of care was considered noncompliance. A description of PM included in the “top13” is summarized in Table 2.
Proportions of compliance were compared among hospital levels using weighted logistic regressions with each PM as the dependent variable and hospital level as the explanatory variable. Adjusted ORs for compliance with each PM were estimated by adding the variables age, sex, and previous history of diabetes mellitus, dyslipidemia, hypertension, coronary heart disease (CHD), peripheral vascular disease (PVD), and valvulopathy together with hospital level in multivariate weighted logistic regression models.
We applied sample weights and performed all analyses using STATA (Stata Statistical Software: Release 9, StataCorp LP).
1805 cases were registered, and after excluding 5 duplicates and 9 empty cases (patients that had been communicated to the online system but for whom no information was available at all), we finally studied 1791 stroke patients. Mean age was 75.6±12.5 years, and 53.9% (n=966) were men. These cases represented an overall reference population of 10 435 stroke admissions per year distributed among hospital levels as follows: 1291 stroke admissions to level 1; 3683 to level 2; and 5461 to level 3. The most common risk factors were: hypertension (66.2%), diabetes (30.2%), dyslipidemia (27.6%), and previous stroke (25.6%). The overall weighted 7-day in-hospital mortality was 9.1% (95% CI: 7.7 to 10.6), and the weighted percentage of patients that were treated with intravenous (iv) tPA within the first 3 hours of stroke, considering all (ischemic and hemorrhagic) stroke admissions, was 3.8% (95% CI: 2.7 to 4.8). The weighted rate of i.v. thrombolysis raised to 8.0% (95% CI: 5.6 to 10.2) when we considered ischemic stroke admissions to level 3 hospitals only, the level to which all reference hospitals with capacity to administer i.v. tPA in 2005 belonged to. The main characteristics of the sample considering hospital levels are shown in Table 3.
A total of 167 (9.3%) cases were externally audited after random selection to check the quality of the data collected at each study site. The global interrater agreement was 0.7, and more than 75% of variables achieved kappa values above 0.6 (according to Landis & Koch).13
The overall compliance with the “top 13” PM is shown in Table 2. Eight PMs achieved compliances ≥75%, with determination of BP and glycemia at baseline, neuroimaging <24 hours, and antithrombotics at discharge being over 90%. Screening of dysphagia (mean proportion of compliance: 33.3%), early mobilization (48.7%), assessment of rehabilitation needs (38%), anticoagulation for atrial fibrillation (AF; 50.3%), and management of dyslipidemia (62%) showed poorer compliances. Moreover, the analysis of compliance across hospital levels displayed some significant differences (Table 4, Figure). Logistic regression models considering each one of the evidence-based measures as dependent variables showed prior diabetes, hypertension, and dyslipidemia to be independent predictors of a greater adherence to baseline determination of glycemia (OR=2.72; 95% CI: 1.42 to 5.19), management of hypertension (OR=6.81; 95% CI: 4.47 to 10.4), and management of dyslipidemia (OR=1.61; 95% CI: 1.12 to 2.31), respectively. Furthermore, a prior valvulopathy was an independent predictor of a better compliance with prescription of antithrombotics at discharge (OR=3.05; 95% CI: 0.9 to 10.3), and of a worse compliance with prescription of aspirin within the first 48 hours (OR=0.55; 95% CI: 0.32 to 0.92). Female sex was predictive of poorer adherences to “neuroimaging within the first 24 hours after stroke” (OR=0.62; 95% CI: 0.38 to 0.99), “early mobilization” (OR=0.7; 95% CI: 0.53 to 0.93), and “assessment of rehabilitation needs” (OR=0.7; 95% CI: 0.52 to 0.94). Similarly, increasing age determined a poorer compliance with “early mobilization” (OR=0.99; 95% CI: 0.97 to 1), and “prescription of anticoagulants for AF” (OR=0.96; 95% CI: 0.94 to 0.99).
The present study illustrates what in-hospital stroke care was like in all 48 acute-care hospitals belonging to the public network of Catalonia (Spain). The main results of this study suggest that quality of in-hospital stroke care was improvable and heterogeneous across hospital levels. Screening of dysphagia, early mobilization, assessment of rehabilitation needs, anticoagulation for AF, and management of dyslipidemia achieved poor compliances. Furthermore, sex, age, some aspects of patients’ medical history, and hospital level predicted better/worse adherences to some quality measures. Admission to a level 3 hospital was predictive of better adherences to acute stroke care PMs and correct management of dyslipidemia, whereas patients admitted to small-sized hospitals had better chances to receive adequate rehabilitation-related PMs. It is important to emphasize that the results of the first Catalan audit of stroke illustrate the quality of stroke care in the first half of 2005, before local CPG on stroke were released, and some changes could have happened in the meantime. Furthermore, because Catalonia is a territory with a homogeneous Health System, the results are representative of the whole territory, and the three hospital levels. However, conclusions cannot be drawn at a hospital level because of reduced sample size.
Hospital size has been identified by others as a determinant of quality of stroke care.14–16 However, and regardless of the many apparent reasons whereby larger hospitals may provide optimal care (better resourced in terms of technical equipment and professionals, continuous training, stroke faculty, dedicated nurses, stroke units, etc), the results of the current study might be taken as unexpected as admission to level 3 hospitals (academic/large hospitals) did not systematically predict higher standards of care compared to level 1 and level 2 hospitals.
Our study shows independent sex- and age-related differences in stroke care: whereas women were less likely to undergo urgent neuroimaging after stroke and to receive adequate rehabilitation-related PMs, elder patients (>80 years old) had lower odds of anticoagulation for AF, and of early mobilization after stroke. Many publications have found sex differences in care for coronary heart disease and stroke. Women are less likely to receive referrals for revascularization procedures or to receive medications indicated on evidence-based guidelines.17 In a recent study, women with a stroke had lower odds of receiving tPA treatment and lipid investigation.18
There are some limitations to our study. First, it consisted of a retrospective review of medical records. Only that information available on the records was considered, and when information was lacking the intervention was declared “nonachieved.” Therefore, the results represent the worst possible scenario and might underestimate the real adherence to the selected quality measures. We acknowledge an information bias as the real noncompliance and the information lacking in the medical records cannot be distinguished in this study. However, we believe the bias was minimized in doing so. First, because it is more likely that the absence of information in medical records corresponds to unperformed interventions rather than performed but not informed interventions, and second, because the latter would also be punishable as it reflects poor quality of medical records.
Second, we only measured the process of care and did not investigate the outcome of that process in terms of medium/long-term mortality/disability. Moreover, although we collected data on in-hospital mortality, we did not perform analyses to determine whether mortality rates were related to quality of care. A few studies have attempted to demonstrate a connection between process of care and outcome in stroke but the relationships are generally weak.19–21 With the exception of “baseline glucose” and “baseline BP” there is robust evidence from randomized trials for the remaining PMs selected in this study.
Third, we did not include “i.v. thrombolysis” and “admission to a stroke unit” (SU) in the “top 13”. Back in 2005, Catalonia did not have a countrywide model of organized acute stroke care and only 5 academic hospitals delivered i.v. tPA to candidate patients living in their primary referral area (2 more hospitals were implemented at the end of 2005). Furthermore, SU were scarce and available only at 6 hospitals (semi-intensive type), whereas the remaining hospitals had different resources for the admission of stroke patients (neurological or medical wards). Because SU and i.v. thrombolysis cannot be reasonably implemented in hospitals admitting a low number of stroke patients, one of the very first purposes of the Catalan SP was to define referral pathways and patients’ flows, to designate new reference hospitals, and to promote the creation of SU in large and middle-sized centers. Our current model of acute stroke care is based on territory-related networks of primary care practices, community and local reference hospital(s) with capacity to emergently assess and treat candidate acute stroke patients in an SU. This model was gradually implemented in 2006 and is being tested currently. Additionally, because this network of hospitals includes a wide range of centers, we looked for a set of simple evidence-based measures of the process of care that could be reached and tested at all participating hospitals.
Achieving performance excellence in health care requires that PMs are measured and compared with prior performances. Assessing stroke care performance is relevant because it allows gaining insight and identifying opportunities for improvement.22,23 The Catalan stroke strategy initiated in 2005 by the SP comprises two different approaches. On the one hand, suspicions over territory-related variations and inequalities in stroke care existed but information about quality of stroke services was lacking. The Stroke Guidelines project, launched in November 2005, included a CPG to ensure that recent knowledge on stroke was available to everybody in an attempt to reduce variations in medical practice, and an audit to detect weak aspects of stroke care. In line with this, several studies have shown that guidelines are useful to standardize processes.3,24–27 On the other hand, we recognized the need for organizing and planning acute stroke care in a nationwide level given the practical difficulties to set up specialized stroke services at all acute-care hospitals. By defining a territory-based model of acute stroke care supported by a series of local networks of primary care centers, community and academic hospitals, we wanted to establish a fairer model.
As mentioned earlier in this article, the first audit of stroke is not an isolated experience. The collection of data for the second audit has just finished, and the aim is to compare quality of stroke care before and after the release of CPG. In between, there have been several actions driven to disseminate the data. Members of the SP have periodic meetings with members of local stroke committees. These 10 multidisciplinary committees are made up of different stroke specialists (neurologists, nurses, rehabilitation physicians, physiotherapist, etc) and the local health authorities responsible for budget assignment. Meetings were carried out throughout the country when the results of the audit were available. All participating hospitals received a report on their scores, and weak aspects of care, as well as the positive ones, were identified and discussed in depth. Furthermore, local committees agreed on specific actions addressed to improve poor aspects of care. Thus, whereas the audits provide knowledge on the evidence-practice gap, the implemental arm of the SP must guarantee that institutional efforts are made to improve weak aspects of care. The challenge is to run periodic audits that incorporate assessments of the most recent knowledge to ensure that all stroke patients get what they really need: high-quality medical care.
The authors thank Maria Luisa de la Puente, Director of the General Directorate for Health Planning and Evaluation, for her invaluable support to the Stroke Programme, as well as all participating hospitals and local collaborators.
Sources of Funding
This study was funded by a grant from the “Fondo de Investigación Sanitaria, Instituto de Salud Carlos III” (PI05/2709), and the General Directorate for Planning and Evaluation from the Catalan Ministry of Health.
- Received June 26, 2008.
- Revision received August 28, 2008.
- Accepted September 8, 2008.
Rudd AG, Hoffman A, Irwin P, Pearson M, Lowe D, Lowe D, Intercollegiate Working Party for Stroke. Stroke units: research and reality. Results from the National Sentinel Audit of Stroke. Qual Saf Health Care. 2005; 14: 7–12.
Rudd AG, Lowe D, Irwin P, Rutledge Z, Pearson M. National stroke audit: a tool for change?. Qual Health Care. 2001; 10: 141–151.
Rudd AG, Hoffman A, Irwin P, Lowe D, Pearson MG. Stroke unit care and outcome: results from the 2001 National Sentinel Audit of Stroke (England, Wales, and Northern Ireland). Stroke. 2005; 36: 103–106.
The impact of standardized stroke orders on adherence to best practices. Neurology. 2005; 65: 360–365.
Heuschmann PU, Biegler MK, Busse O, Elsner S, Grau A, Hasenbein U, Hermanek P, Janzen RW, Kolominsky-Rabas PL, Kraywinkel K, Lowitzsch K, Misselwitz B, Nabavi DG, Otten K, Pientka L, von Reutern GM, Ringelstein EB, Sander D, Wagner M, Berger K. Development and implementation of evidence-based indicators for measuring quality of acute stroke care: the Quality Indicator Board of the German Stroke Registers Study Group (ADSR). Stroke. 2006; 37: 2573–2578.
Hinchey JA, Shephard T, Tonn ST, Ruthazer R, Selker HP, Kent DM. Benchmarks and determinants of adherence to stroke performance measures. Stroke. 2008; 39: 1619–1620.
Activity of the Healthcare and Social-Healthcare Network of Catalonia 2006. Report on Registers of the Minimum Basic Data Set (CMBD) (web site). Barcelona: Servei Català de la Salut (Departament de Salut, Generalitat de Catalunya); 2007 (cited June 10, 2008). Available from: http://www10.gencat.net/catsalut/cat/prov_cmbdpublicacions.htm.
Master Plan for Diseases of the Circulatory System (monograph on the internet). Barcelona: Department of Health (Generalitat de Catalunya); 2006 (cited June 20, 2008). Available from: http://www.gencat.net/salut/depsan/units/sanitat/html/en/dir228/index.html.
Tresserras R, Castell C, Pardell H. Evaluation of the targets on cardiovascular diseases. In: Salleras L, Tresserras R, editors. Evaluation of the targets of the Health Plan for Catalonia for the year 2000 (monograph on the internet). Barcelona: Department of Health (Generalitat de Catalunya); 2005 (cited June 20, 2008). Available from: http://www.gencat.net/salut/depsan/units/sanitat/html/en/dir228/index.html.
Guia de pràctica clínica de l’ictus. Catalunya 2007 (web site). Barcelona: Agència d’Avaluació de Tecnologia i Recerca Mèdiques; 2007 (cited June 18, 2008). Available from: http://www.gencat.net/salut/depsan/units/sanitat/html/ca/dir491/index.html.
Gillum LA, Johnston SC. Characteristics of academic medical centers and ischemic stroke outcomes. Stroke. 2001; 32: 2137–2142.
Saposnik G, Baibergenova A, O'Donnell M, Hill MD, Kapral MK, Hachinski V. Hospital volume and stroke outcome: does it matter? Neurology. 2007; 69: 1142–1151.
Gargano JW, Wehner S, Reeves M. Sex differences in acute stroke care in a statewide stroke registry. Stroke. 2008; 39: 24–29.
McNaughton H, McPherson K, Taylor W, Weatherall M. Relationship between process and outcome in stroke care. Stroke. 2003; 34: 713–717.
Weir N, Dennis MS. Towards a national system for monitoring the quality of hospital-based stroke services. Stroke. 2001; 32: 1415–1421.
Measuring and improving quality of care : A report from the American heart Association/American college of cardiology first scientific forum on assessment of healthcare quality in cardiovascular disease and stroke. Stroke. 2000; 31: 1002–1012.
Holloway RG, Vickrey BG, Benesch C, Hinchey JA, Bieber J. Development of performance measures for acute ischemic stroke. Stroke. 2001; 32: 2058–2074.
Duncan PW, Horner RD, Reker DM, Samsa GP, Hoenig H, Hamilton B, LaClair BJ, Dudley TK. Adherence to postacute rehabilitation guidelines is associated with functional recovery in stroke. Stroke. 2002; 33: 167–177.
Hammond R, Lennon S, Walker MF, Hoffman A, Irwin P, Lowe D. Changing occupational therapy and physiotherapy practice through guidelines and audit in the United Kingdom. Clin Rehabil. 2005; 19: 365–371.
Scottish Intercollegiate Guidelines Network (web site). Edinburgh: Scottish Intercollegiate Guidelines Network; 2004 (cited 10 Jun 2008). Available from: http://www.sign.ac.uk/guidelines/fulltext/50/annexb.html.