| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2003;34:713.)
© 2003 American Heart Association, Inc.
Original Contributions |
From the Medical Research Institute of New Zealand, Wellington (H.M.); Health and Rehabilitation Research Unit, School of Health Professions, University of Southampton, Southampton, UK (K.M.); and Rehabilitation Teaching and Research Unit, Wellington School of Medicine and Health Sciences, University of Otago, Otago, New Zealand (W.T., M.W.).
Correspondence to Dr Harry McNaughton, Medical Research Institute of New Zealand, PO Box 10055, Wellington, New Zealand. E-mail hkmcn{at}paradise.net.nz
| Abstract |
|---|
|
|
|---|
Methods This was a prospective study of consecutive patients with acute stroke admitted to each of the 3 general hospitals in 1 region who were followed up for 12 months after hospital discharge. Process was measured by use of the Royal College of Physicians Stroke Audit Package, and outcomes were measured with a range of disability, health status, handicap, and independence measures, as well as mortality.
Results One hundred eighty-one patients were recruited. There was evidence for a relationship between some process variables and outcomes at hospital discharge, but the relationships were generally weak. None of the process variables remained in regression models of functional outcomes at 12 months. The hospital with the best process scores had the worst case mix-adjusted outcomes.
Conclusions The link between stroke process and outcome, through the use of currently available measures of process such as the Royal College of Physicians Stroke Audit Package, is not straightforward. Ongoing work may clarify some of these issues and provide guidance to stroke clinicians on how best to improve existing services.
Key Words: outcome process assessment (health care) stroke
| Introduction |
|---|
|
|
|---|
The relationship between process and outcome depends critically on how closely tied to outcomes the particular aspects of process might be. Mant and Hicks4 use the example of thrombolysis and myocardial infarction to justify the measurement of process rather than outcome. Here, the time from onset of symptoms to injection of thrombolytic agent is known to influence reperfusion of the coronary vessels, the degree of infarcted myocardium, and outcomes in terms of mortality. Hence, measurement of delays in people receiving thrombolysis might reasonably be expected to translate into measurement of worse outcomes in a sample of sufficient size. However, in stroke care (for which Mant and coworkers5 also advocate measurement of process rather than outcome), the situation is not as clear cut. Indeed, the possible items of process to be measured may not necessarily reflect what is having an impact on outcome. Currently, we remain unclear as to what it is about stroke unit care that leads to lower mortality. It may be early medical management, multidisciplinary rehabilitation teams, the enthusiasm of the nursing staff working in stroke services, early mobilization, or a number of other possible factors. Whether these items, reflected in process measures, could be used as a proxy for outcome measurement remains uncertain.
Various authors have attempted to demonstrate a connection between process and outcome in stroke,6,7 but the relationship, particularly that between postinitial stroke incident care and longer-term outcome, is poorly understood. We report here our experience with the Royal College of Physicians Stroke Audit Package (RCPSAP8) in measuring process and comparing this to case mix-adjusted outcome at hospital discharge and 12 months after hospital discharge.
| Methods |
|---|
|
|
|---|
77% of eligible patients with stroke. The main reason for nonenrollment was failure to provide informed consent or an inability to do so because of death or early discharge. Assessments of prestroke function, stroke variables, and initial disability with the Functional Independence Measure (FIM)10 were made within the first week of admission. Stroke type was classified by use of the Oxfordshire Community Stroke Project classification.11 Patients were followed up until death or 12 months after hospital discharge in the community. Outcome measures included the modified Rankin Scale (MRS),12 Short Form (SF)-36,13 mortality, and place of residence. Discharge FIM was the last FIM assessment before discharge, which occurred a median of 3 days before discharge. Poor discharge outcome was defined as death, a move from home to residential care, or a move from rest home-level to hospital-level care. Process was measured with the RCPSAP and 4 additional process indicators. The RCPSAP was chosen because it is a well-established quantitative measure of process with adequate interrater reliability.14,15 Its 60 items cover acute and rehabilitation aspects of care for inpatients after stroke. The RCPSAP items were scored according to the manual that comes with the audit package, which is detailed elsewhere.16 Case notes were reviewed by a single reviewer (H.M.) retrospectively at least 6 months after hospital discharge. A summary score for the RCPSAP (RCPSAP Sum) was derived in the following way. For each patient, each of the 60 items of the RCPSAP was scored 1 point for "yes" and no points for "no" or "no, but" values. The 60 scores were then summed; this total was divided by the number of items with a "yes" or "no" response; and this fraction was expressed as a percentage (ie, "no, but" items were excluded). This means that when sections of the process audit could not be completed for some patients (such as the rehabilitation section for patients discharged from a medical ward or who died in hospital), it was still possible to score 100% as an overall score based on the items relevant to that patients care. The 4 specific process indicator items17 were as follows: proportion of people having a swallow assessment before feeding (swallow); proportion of people having a CT head scan (CT done); proportion of people with an adequate clinical formulation (includes neurological deficits, part of the brain affected, and risk factors; clinical formulation); and proportion of people having a multidisciplinary team meeting (MDT meeting) to discuss a management plan or progress.
Statistical Analysis
Categorical variables were analyzed by use of
2 tests, and continuous variables meeting normality assumptions were analyzed by use of t tests. When normal assumptions were not met, nonparametric statistics were used. Regression equations were developed for various response variables as detailed in the Results section for each variable. Analyses were undertaken with SPSS 10.0 and SAS software.
| Results |
|---|
|
|
|---|
|
Thirty-five patients (19%) died in hospital; another 18 patients (cumulative, 31%) died by 12 months after hospital discharge. Data on functional outcomes at 12 months are based on 118 survivors of stroke whom we could follow up. Hospital process scores could be obtained for all 181 patients.
Ninety patients (50%) were transferred for inpatient rehabilitation. Mean length of hospital stay was 30.5 days (median, 21 days; interquartile range, 8 to 47). Median times to death in hospital and transfer to inpatient rehabilitation were 9 and 10 days, respectively.
RCPSAP Sum scores were normally distributed, with a mean of 62.7 (range, 18 to 100; SD, 12.9). The percentage of all subjects with "yes" scores on the 4 specific process items were as follows: swallow, 23%; CT done, 83%; clinical formulation, 34%; and MDT meeting, 51%.
There were no significant differences in RCPSAP Sum scores according to patient characteristics such as age, sex, ethnicity, initial disability, or stroke type. However, mean RCPSAP Sum scores were significantly higher in the group who were transferred for inpatient rehabilitation (n=90; mean RCPSAP Sum, 67.9; 95% CI, 65.6 to 70.1) than those not transferred (n=91; mean RCPSAP Sum, 57.6; 95% CI, 55.0 to 60.3; P<0.001). Subjects <75 years of age were more likely than those >75 years of age to have had an adequate swallow assessment (30.2% versus 16.8%, P=0.03), a CT scan (95.3% versus 72.6%, P<0.001), and an adequate clinical formulation (41.9% versus 27.4%, P=0.004). Subjects transferred for inpatient rehabilitation were more likely than those who were not to have had a CT scan (93.3% versus 73.6%, P<0.001), an adequate clinical formulation (44.4% versus 24.2%, P=0.004), and an MDT meeting (86.7% versus 16.5%, P<0.001).
Comparing Results for Process and Outcome
Comparison of findings was made in 3 ways: (1) univariate analysis calculating correlations between process and outcome variables, (2) multivariate analysis of process and nonprocess variables against various outcomes, and (3) comparison of the 3 hospitals in the study in terms of process and case mix-adjusted outcome.
Univariate Analysis
There was no significant relationship between RCPSAP Sum and discharge FIM, poor discharge outcome, or the physical or mental component summary scale of the SF-36. However, there was a significant difference in RCPSAP Sum between nonsurvivors (mean, 57.1; 95% CI, 52.9 to 61.4) and survivors (mean, 64.1; 95% CI, 62.0 to 66.1) at hospital discharge (P=0.004). When the MRS is expressed as a binary variable (0 to 2 versus 3 to 5), subjects with the worst functional outcome (MRS, 3 to 5) had better process scores than those with a better functional outcome (mean RCPSAP Sum, 66.5 [95% CI, 63.3 to 69.7] versus 60.9 [95% CI, 57.4 to 64.4]; P=0.02).
Regression
The first response variable modeled was discharge FIM. The following potential explanatory variables were entered into the model: age, sex, prestroke FIM, initial poststroke FIM, European ethnicity, total anterior circulation infarct versus other types of stroke, RCPSAP Sum, clinical formulation, MDT, CT recorded, and swallow. With the use of a stepwise procedure with P=0.1 to enter and P=0.15 to leave the model, the final model presented in Table 2 is derived: older age and being European were associated with a lower FIM score at discharge. Higher initial FIM and prestroke FIM, having an MDT recorded, and higher RCPSAP Sum were associated with higher FIM at discharge. The R2 for this model was 77.5%, and normality assumptions were met.
|
The first logistic regression model used the 5 process variables to predict the poor discharge outcome variable through a backward selection procedure, with P=0.1 for staying in the model. The 2 variables that remained associated with poor discharge outcome were CT not recorded (odds ratio [OR], 2.07; 95% CI, 0.93 to 4.62; P=0.08) and swallow assessment not recorded (OR, 2.48; 95% CI, 1.10 to 5.61; P=0.03).
The second model used the 4 case mix variables found to be significant in the regression model outlined above (age, initial FIM, prestroke FIM, and being European) to see whether, after adjustment for these variables, the 2 process measures identified above still predicted a poor discharge outcome. Again, a backward selection process was used, with P=0.1 for staying in the model. The variables remaining in the model are described in Table 3; a higher prestroke and initial FIM were associated with a lower probability of poor discharge outcome, whereas being European and having no swallowing recorded were associated with a higher probability of poor discharge outcome. The model fitting test (Hosmer and Lemeshow test) was not significant, with a
2 of 5 on 8 df (P=0.76).
|
Response variables at 12 months after hospital discharge (MRS >2, physical component summary score of the SF-36) were also modeled in a similar fashion, and the process variables were found to have no explanatory power when combined with case mix variables.
Comparing Hospitals
One way to investigate assumptions behind the "league table" approach is to test the hypothesis that hospitals with the best process scores also have the best outcomes. The results for the process variables and selected case mix-adjusted outcomes are presented in Table 4. Case mix-adjusted outcomes were calculated from regression equations as described above with dummy variables used to separate the 3 hospitals. On ranking, hospital 1 had the poorest scores on the process variables but highest scores on outcomes, and hospital 2 had the highest mean RCPSAP Sum score but the poorest outcomes.
|
| Discussion |
|---|
|
|
|---|
This was a small, observational study, and some caution is required when the results are considered. First, the study sample did not include all patients with stroke admitted during the study period. Case note review showed that 77% of all eligible stroke subjects were enrolled. Those not enrolled were mainly patients with minimal disability who were discharged early or those with very severe strokes, some of whom died soon after admission. Nevertheless, hospital mortality, proportions of subjects with each stroke type, and initial disability were similar to comparable studies in the United Kingdom, Australia, and New Zealand, and the sample may be considered generally representative of stroke patients admitted to hospital in these countries.18 Second, the assessment of process was not blinded, so there was a possibility of expectation bias on the part of the single reviewer who would have had the opportunity to be aware of inpatient outcome during case note review. Third, none of the 3 hospitals in the study had a stroke unit. It could be argued that the issue of improving aspects of the process of care should apply only once a stroke unit is in place. All the hospitals in this study offered general rehabilitation wards, generally geriatric rehabilitation without a specific focus on stroke, and 50% of all patients were transferred to these wards. The Stroke Unit Trialists Collaboration systematic review1 was unable to determine an extra benefit for stroke units over generic rehabilitation wards in terms of outcome after stroke. Selection bias may be responsible for the association in our results between higher RCPSAP Sum scores and survival. Higher RCPSAP Sum scores (and having an MDT meeting) were also associated with transfer for rehabilitation. In these hospitals, patients thought likely to die are not transferred for rehabilitation. Fourth, our choice of case mix variables could be criticized because they were not externally validated. However, the final models are similar to a recently reported, externally validated model of outcome after stroke,19 apart from the inclusion of ethnicity. In our cohort, Maori and Pacific people had significantly different outcomes than people of European descent,9 so this variable was included in the model.
The Scottish Stroke Outcomes Group described variations in case fatality after stroke at 5 hospitals in Scotland.6 They were able to show that most of that variation was likely due to differences in case mix. However, 1 hospital in that study had significantly higher case fatality after adjustment for case mix. The authors surmise that the differences in stroke process that were recorded (in particular, no specialized and organized stroke care, many fewer CT head scans, and lowest overall process scores) may have been responsible and suggest that the quality of care at this hospital was deficient. They measured process using the RCPSAP and some other items such as whether the patient was admitted to a stroke rehabilitation unit. Their "compliance per record" factor, which varied between a median of 53 and 79 among the 5 hospitals, is very similar to our RCPSAP Sum variable, which varied between a median of 60 and 67 among our 3 hospitals. Put another way, if the worst hospital in the Scottish study is excluded, no difference in case mix-adjusted outcome is observed from the remaining 4 hospitals with large variations in process, eg, a compliance per record factor range from 62 to 79, a 2-fold difference in admission rates to a stroke rehabilitation unit, and a 2.5-fold difference in the number of patients discharged from the care of a stroke consultant. In an unpublished study, we repeated the measurement of process and outcome in a further cohort of consecutive stroke patients, but despite a statistically significant increase in RCPSAP Sum scores to a mean 77.2, no difference in mortality or functional end points was measurable after adjustment for case mix. In the present study, the numbers of patients at each hospital were small; thus, the power to detect even sizeable differences in treatment effect would be small.
At least part of the difficulty in determining the true relationship between process and outcome may relate to the specific process measure. The RCPSAP is predominantly a measure of medical documentation, focusing heavily on acute care. It covers issues such as whether blood pressure is recorded, whether an examination for visuoperceptual deficit is documented, and whether an ECG is in the medical notes. It does not cover issues that may be directly relevant to the efficacy of inpatient stroke care such as management of fever, elevated blood pressure, or early mobilization or what many of the (nonmedical) members of a multidisciplinary stroke team actually do. In scoring the RCPSAP, we used simple summation, rating each item as being of equal importance. A similar approach has been taken by others.6 We tested different ways of weighting individual items and groups of items of the RCPSAP but did not find any significant differences in the overall results obtained. Nevertheless, Table 4 shows that similar RCPSAP Sum scores can hide sizeable differences in individual process components between different hospitals. The Intercollegiate Stroke Audit Package20 has been developed since we undertook this study and may supplant the RCPSAP as the standard for quantitative audit of the process of stroke care. An alternative approach is that of Hoenig and colleagues,21 who have attempted to define the key characteristics of stroke rehabilitation services, aiming to relate these prospectively to differences in outcome. Holloway and colleagues22 have provided a series of performance measures in acute stroke care based on consensus of experts.
Evans and colleagues7 have presented the most compelling data to date to support the contention that better process of care is associated with better outcomes. They prospectively and comprehensively measured aspects of stroke care, including assessment and treatment in the first week, rehabilitation interventions, and secondary prevention. Many of these items are in common with those in the RCPSAP. Significant factors in multivariate analyses with good functional outcome as the end point (MRS <4) were stroke unit management, early feeding, prevention measures for aspiration, stroke progression, chest infection, and dehydration. What constituted prevention measures was not specified. Attempts were made to reduce bias in this unblinded assessment by using 2 observers. Ultimately, however, the various prevention measures are likely to have been strongly associated with stroke unit care, which was the primary intervention. We do not know whether improving these aspects of stroke process instead of, or in addition to, providing stroke unit care improves results.
We believe that the currently available evidence does not provide an evidenced-based rationale for limiting the focus to improving stroke process as the best way of improving stroke outcome, apart from the provision of stroke unit care for as many people as possible. Once stroke unit care is provided, however, the problem of how to continuously improve the quality of that service remains. The work of Evans et al7 points to preventive measures as important, and better definition of these measures would be helpful. Ongoing work with which we are involved is aimed at clarifying the nature, quality, and quantity of interactions between clinicians and stroke patients within the stroke rehabilitation setting, thereby identifying behaviors associated with better outcomes. Such investigation may enable more concrete recommendations to be made about optimum structures and process within the overall framework of stroke unit care. The approach of Hoenig and colleagues21 in trying to define the components of care at different levels of organization might also allow future research to test the relationship between different levels of organization and stroke outcomes.
In the meantime, despite the many difficulties involved, direct measurement of outcome, including mortality and functional end points, ensures measurement of the variables of interest rather than a reliance on assumptions of process measures as a proxy for outcome. Further refinement of case mix-adjustment systems is required both for interservice comparisons and for comparison of a single service at different time points if direct outcome measurement is to become the standard for quality assessment of stroke services.
| Acknowledgments |
|---|
Received July 5, 2002; revision received September 12, 2002; accepted September 25, 2002.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
M. A. Mohammed, J. Mant, L. Bentham, A. Stevens, and S. Hussain Process of care and mortality of stroke patients with and without a do not resuscitate order in the West Midlands, UK Int. J. Qual. Health Care, April 1, 2006; 18(2): 102 - 106. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. K. Kapral, A. Laupacis, S. J. Phillips, F. L. Silver, M. D. Hill, J. Fang, J. Richards, J. V. Tu, and for the Investigators of the Registry of the Canad Stroke Care Delivery in Institutions Participating in the Registry of the Canadian Stroke Network Stroke, July 1, 2004; 35(7): 1756 - 1762. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. G. Rudd and D. B. Matchar Health Policy and Outcome Research in Stroke Stroke, February 1, 2004; 35(2): 397 - 400. [Full Text] [PDF] |
||||
![]() |
D. S. Likosky, W. H. Likosky, and T. L. Cress Re: Relationship Between Process and Outcome in Stroke Care Stroke, September 1, 2003; 34 (9): e158 - e158. [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2003 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |