(Stroke. 2002;33:2593.)
© 2002 American Heart Association, Inc.
Original Contributions |
From the Brooks Center for Rehabilitation Studies, University of Florida, and Rehabilitation Outcomes Research Center, North Florida/South Georgia Department of Veteran Affairs, Gainesville, Fla (P.W.D.); Kansas City Department of Veterans Affairs Medical Center, Kansas City, Mo (D.T., D.M.R.); and Center on Aging, University of Kansas Medical Center, Kansas City, Ks (P.W.D., S.M.L., S.P., D.M.R., S.S.).
Correspondence to Pamela W. Duncan, PhD, FAPTA, Brooks Center for Rehabilitation Studies, College of Health Professions, University of Florida, Health Science Center, Box 100185, Gainsville, FL 32610-0185. E-mail pwduncan{at}hp.ufl.edu
| Abstract |
|---|
|
|
|---|
Methods Two hundred eighty-seven patient and proxy pairs from the Kansas City Stroke Registry participated in the study. All patients were assessed in their home or nursing facility between 90 and 120 days after stroke with the use of the modified Rankin Scale Motricity Index (strength), Barthel Index (activities of daily living), Lawton assessment (instrumental activities of daily living), Folstein Mini-Mental State Examination (cognition), and the SIS. Eligible proxies were individuals who were aged
18 years, had known the patient for at least 1 year, and saw the patient at least once each week. All proxy interviews were conducted within 7 days of (before or after) the patients interview.
Results Three hundred seventy-seven patients from the Kansas City Stroke Registry were eligible for the study. Seventy-seven patients or proxies refused participation. Thirteen patients of the consenting patient-proxy pairs were too aphasic or cognitively impaired to complete the interviews and were dropped from the study. Proxies scored patients as more severely affected than patients scored themselves on the SIS-16 and in 7 of 8 domains of the full SIS (5 were statistically significant at
=0.05). The proxy bias toward overrating the severity of the patients condition tended to increase as the severity of the stroke increased. However, the magnitude of the biases between patient and proxy means, as measured by effect size, was small (range, -0.1 to 0.4). The strength of the agreement, as measured by intraclass correlation coefficients, between proxy and patient ranged from 0.50 to 0.83. Agreement was best for the observable physical domains. Both patient and proxy scores in all domains were significantly different across Rankin categories. Concurrent validity for both patient and proxy correlations with the Folstein Mini-Mental State Examination, Barthel Index, Lawton instrumental activities of daily living, and Motricity Index was good to excellent (range, 0.37 to 0.78).
Conclusions Proxies provide valid information for assessment of stroke outcomes. There are significant differences between patient and proxy reporting on SIS domains and the SIS-16. However, the observed biases are small and not clinically meaningful.
Key Words: disability evaluation outcome quality of life stroke
| Introduction |
|---|
|
|
|---|
One of the limitations of the use of self-report measures to assess outcomes after stroke is that many stroke survivors have cognition and communication problems. In a previous stroke outcome study, 25% of the subjects were excluded from health-related quality of life assessments because of cognitive and language disorders.2 In a large study that used mail-administered quality of life questionnaires, 50% of the stroke subjects were unable to complete the questionnaires by themselves.3 Study results can be seriously compromised and misleading if subjects who are suffering from severe deficits are excluded. The inclusion of proxy data will increase sample size, improve generalizability of the studies, and reduce sample bias.4
Good proxy-patient agreement is a necessary criterion for the usefulness of a stroke outcome measure.5 The development of the SIS requires that the reliability of proxy-patient responses to each domain be evaluated. The purposes of this study were to compare proxy-patient responses on each domain of the SIS and the SIS-16, estimate the biases, and evaluate the validity of patient and proxy scores. In addition, patient and proxy characteristics that affected agreement or bias were also evaluated.
| Subjects and Methods |
|---|
|
|
|---|
To be eligible for participation in the Stroke Registry, a patient had a documented diagnosis of stroke and
50 years. The World Health Organization definition of stroke was used for the purposes of this study.6 A stroke was defined as symptoms of rapid onset and of vascular origin reflecting a focal disturbance of cerebral function, excluding isolated impairment of higher function.6 These deficits must have persisted for >24 hours. In addition, patients could not have any of the following exclusion criteria: stroke onset >28 days, stroke due to subarachnoid hemorrhage, deficits from a previous stroke, not expected to live 1 year, New York Heart Association class IV heart failure, not community dwelling before stroke, not independent in basic ADL before stroke, living >50 miles from participating hospital, progressive or severe neurological disease, amputee, obtunded, comatose, or unable to follow a 3-step command.
If, on review of the medical record, a patient was found to be eligible for the Stroke Registry, consent to approach the patient about the Stroke Registry was obtained from the primary physician. The Stroke Registry was then explained to the patient, and informed consent was obtained.
Baseline Measures
At enrollment, a baseline assessment conducted by the research staff of nurses and physical therapists assessed patients demographics, stroke type, prior functional status, and stroke severity with the use of the National Institutes of Health Stroke Scale7 and Barthel Index.8
Ninety-Day Follow-Up Assessments
All Stroke Registry patients received a follow-up telephone assessment at 90 days after stroke. Data were collected from the patient or a proxy. This telephone call consisted of a brief cognitive screen, the Hodkinson Cognitive Screen,9 to determine the patients ability to accurately answer questions about health status. If the patient was unable to communicate or score
5 on the Hodkinson Cognitive Screen,9 a proxy for the patient completed the telephone interview. During the telephone interview, the patients basic ADL and IADL were assessed with the Barthel Index8 and Lawton IADL.10 At the conclusion of this call, patients were asked to participate in the present study. Those who agreed were asked to identify a proxy. Eligible proxies were individuals who were aged
18 years, had known the patient for at least 1 year, and saw the patient at least once each week.
Patient Interview
All patients were assessed in their home or nursing facility by a masters level research associate between 90 and 120 days after stroke. Patients 3-month stroke severity was characterized by the modified Rankin Scale.11 Strength was measured by the Motricity Index.12 The Folstein Mini-Mental State Examination (MMSE) was used to rate cognitive ability.13 After these measures were gathered, patients were interviewed with the use of the SIS.
Proxy Interview
All proxy interviews were conducted within 7 days of (before or after) the patients interview to minimize any actual change in the patients status between interviews. Both the patients and the proxies were blinded to each others responses. Before participation, each proxy signed an informed consent. Proxy demographics including age, sex, race, education, and relationship to the patient were gathered, as was information about the amount of time spent with the patient each week. Proxies were also asked to rate, on a scale of 0 to 100, how well they felt they knew the patient. Proxy mental status was then determined with the Folstein MMSE.13 Proxy health status was assessed by an interviewer-administered Medical Outcome Survey 36-item short-form health survey (SF-36).14,15 After this information was gathered, proxies were interviewed about patients stroke recovery with the SIS.
Data Analysis
Descriptive statistics were used to characterize the patients, proxies, and their relationships. Several analytical strategies were used to examine patient-proxy agreement. First, mean responses of the patients and proxies were compared for each of the individual SIS domains with the use of a paired-samples Students t test. To examine systematic bias, effect size was calculated by dividing the mean difference score by the standard deviation of the difference score.16 An effect size of 0.2 was considered a small bias, 0.5 a moderate bias, and 0.8 a large bias.17 Second, intraclass correlation coefficients (ICCs) between patient and proxy SIS assessments were computed with the use of the SAS VARCOMP procedure and minimum variance quadratic unbiased estimation. Guidelines used for the ICC as a measure of the strength of agreement were as follows: <0.40, poor agreement; 0.40 to 0.75, fair to good agreement; and 0.75 to 1.00, excellent agreement.18 Third, ANOVA was used to examine whether differences between proxy and patient assessments varied across the range of stroke severity (Rankin Scale 0 to 5). Finally, we used 17 regression models with interaction for each SIS domain to identify any patient and proxy characteristics that may have affected patient-proxy differences after controlling for the patient-proxy average SIS score.
To evaluate discriminant validity of patient and proxy scores, we examined differences in mean scores on each domain of the SIS across Rankin grades using ANOVA. We also assessed relationships between standard measures of stroke outcomes and select SIS domain scores with Pearson correlation coefficients. SAS version 8.00 was used for all statistical analyses.
| Results |
|---|
|
|
|---|
|
|
|
|
Comparison of Patient and Proxy Responses
Agreement between patient and proxy SIS domain scores and the SIS-16 was examined in 287 patients for whom both patient and proxy information was available. Table 5 presents patient and proxy mean scores, patient-proxy differences, the magnitude of the differences (bias), and ICCs between patient and proxy reporting. Differences between patient and proxy information were statistically different for 5 of the 8 SIS domains and for the SIS-16. In all 5 of these domains, the proxies rated the patients as more impaired than the patients rated themselves. However, the biases between patient and proxy means scores were low (-0.1 to 0.4). The strength of the agreement between proxy and patient reporting ranged from moderate to excellent (ICCs ranged from 0.50 to 0.83). The best agreements were for domains that represented observable physical behaviors, and the worst agreements were on more subjective domains (eg, memory and thinking, communication, emotion, and strength).
|
Factors That Affect Patient-Proxy Differences
In patients with more severe stroke (Rankin Scale 4 to 5), the differences in patient-proxy reporting for ADL/IADL and SIS-16 were significantly larger than in individuals with less severe stroke. Stroke severity did not affect patient-proxy differences for the other SIS domains (Table 6). We used 17 regression models with interaction for each SIS domain to assess multiple patient and proxy characteristics (eg, age, sex, education, cognitive function, mental health, relationships, and time spent with the patients), which may affect patient-proxy differences. The results of these analyses did not systematically identify any patient or proxy factors that affected the observed differences.
|
Validity of Patient and Proxy Ratings
The relationships between patient and proxy SIS domains and some relevant standardized measures are reported in Table 7. The correlations between the Barthel Index and ADL/IADL domains and mobility domains of the SIS were very strong for both patient and proxy reporting. Agreement between Lawton IADL and SIS ADL/IADL domains was also excellent. The correlations between both proxy and patient reporting of memory functions and the Folstein MMSE were moderate (0.37 and 0.42, respectively). The mean scores for both patient and proxy reporting on all SIS domains and SIS-16 are depicted in Table 8. Both patient and proxy scores were significantly different across Rankin categories of stroke severity (0/1, 2, 3, and 4/5). The differences across Rankin categories were large for the physical domains but smaller for the memory, emotion, and communication domains.
|
|
| Discussion |
|---|
|
|
|---|
Agreement between stroke survivors and their proxies has been examined in 5 previous studies with the use of 6 different self-report measures. In a small study (n=38) of interviewer-administered assessments of chronic stroke survivors, Segal and Schall5 reported that proxy agreement was excellent (ICC=0.91) for the physical domain of the Functional Independence Measure but lower for the social-cognitive domain (ICC=0.61). Proxy agreement on IADL, as measured by the Frenchay Activities Index, was excellent (ICC=0.81) but was very poor for the SF-36 (average ICC=0.32; range, 0.67 for physical function to 0.15 for emotional role function).21 In a subsequent study of 25 of the subjects enrolled in the previous study, Segal and Schall5 compared proxy-patient agreement for telephone administration of the Functional Independence Measure. These subjects were assessed 18 months after stroke. Paralleling the results found for the in-person administration of the Functional Independence Measure at 6 months after stroke, the patient-proxy agreement was 0.91 for the physical domain and was lower for the social-cognitive domain (0.52).
Sneeuw et al22 compared proxy-patient responses for a group of stroke survivors on the Sickness Impact Profile. ICCs were high (0.85) for the physical domains and moderate for the psychosocial domains (0.61). Systematic differences were observed between patient and proxy responses. Proxies rated patients as having more impairments than the patients rated themselves. The tendency of proxies to rate patients as more impaired was most pronounced in the more severely affected patients, but the bias was small. The proxy Sickness Impact Profile scores were significantly different across Rankin categories, which supports the validity of the proxy scores.22 Dorman et al23 compared proxy and patient responses for a group of stroke survivors on the EuroQol. They reported that proxy-patient agreements were best for self-care and physical function yet not much better than chance for psychosocial domains. Patients reported that their health states were better than the assessment of the proxies. Assessment of proxy-patient agreement on a telephone-administered community integration measure, the CHART, revealed that in a sample of 177 stroke patients the correlations were moderate to excellent for most domains but were lower for estimates of social integrations (ICC range, 0.44 to 0.72).24 The proxies were systematically more likely to rate the patients as more impaired, but the differences were small. The proxy-patient agreements for stroke survivors were similar to those for other disability groups.24
The results of the current comparison of patient and proxy reporting on the SIS are consistent with previously reported studies in the elderly and in individuals with stroke.1924 Proxy and patient reports on the SIS domains are significantly different, and proxies report the patients to be more impaired than the patients report themselves. However, the magnitude of the differences (bias) is small and not clinically meaningful. The average patient-proxy differences for SIS-16 range from -1.5 to 6.3 (Table 5); however, the differences across Rankin severity levels for both patient and proxy reporting are large (Table 8). For example, the differences in scores for the SIS-16 physical domain across adjacent Rankin categories range from 11.9 to 25.5. The agreement between patient and proxy reporting is best for observable domains of ADL/IADL, mobility, hand function, and SIS-16. Agreements decrease when subjective judgment is required to assess strength, memory, emotion, communications, and social participation. The agreements for the subjective domains of the SIS are better than those reported for the subjective domains of the SF-365 and the EuroQol23 and are equally as good as patient-proxy agreement on the social-cognitive domain of the Functional Independence Measure.5
Similar to the findings of Sneeuw et al22 for the Sickness Impact Profile, a factor that affects patient-proxy agreement is stroke severity. In the most severely affected patients, the differences between patient-proxy reporting were greatest in the ADL/IADL and SIS-16 physical domains. However, the maximum average difference between patient-proxy reports (10.7 points), which occurred in the ADL/IADL domain, was less than half of the difference (22.8 points) between Rankin levels 3 and 4/5 for that domain.
Both patient and proxy ratings on the SIS are valid. Patient and proxy ratings are strongly correlated with clinician-administered standardized measures (Table 7). The weakest correlation was between the memory domain and the Folstein MMSE. This weak correlation may be due to 2 factors: (1) the low variance in Folstein MMSE scores and self-report memory deficits and (2) the fact that only 2 of the items of the Folstein MMSE capture memory. The between-level differences in Rankin Scale severity are significant for all domains of the SIS. As expected, given the emphasis of the Rankin Scale on physical function, the between-level differences were larger for the physical domains than for the memory, communication, and emotion domains.
Conclusions
The evaluation of proxy-patient agreement on the SIS suggests that proxies may provide valid information for assessment of stroke outcomes. This study provides further evidence that there are acceptable levels of proxy and patient agreement for observable physical behaviors and that the bias observed is not clinically meaningful. Clinicians and researchers should exercise more caution when proxy responses are used to assess the more subjective domains of emotion, memory, and communication. However, our estimates of agreement on the more subjective domains may be conservative because of the lower variability in scores in these domains in our sample.
The results of this study have implications for designing stroke outcome studies. It is appropriate to use proxy assessment; however, the researcher must be aware of the biases introduced with proxy responders. Proxies will report more limitations in function. Therefore, if an intervention improves outcomes and patients are more likely to be able to respond in the treatment group, the magnitude of the effect of the intervention may be exaggerated compared with the control group. This difference, however, may not affect the direction of the treatment effect.
| Acknowledgments |
|---|
Received April 2, 2002; revision received May 28, 2002; accepted June 11, 2002.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
B. T. Volpe, D. Lynch, A. Rykman-Berland, M. Ferraro, M. Galgano, N. Hogan, and H. I. Krebs Intensive Sensorimotor Arm Training Mediated by Therapist or Robot Improves Hemiparesis in Patients With Chronic Stroke Neurorehabil Neural Repair, June 1, 2008; 22(3): 305 - 310. [Abstract] [PDF] |
||||
![]() |
K. Hilari, S. Owen, and S. J. Farrelly Proxy and self-report agreement on the Stroke and Aphasia Quality of Life Scale-39 J. Neurol. Neurosurg. Psychiatry, October 1, 2007; 78(10): 1072 - 1075. [Abstract] [Full Text] [PDF] |
||||
![]() |
T. Ewert and G. Stucki Validity of the SS-QOL in Germany and in Survivors of Hemorrhagic or Ischemic Stroke Neurorehabil Neural Repair, March 1, 2007; 21(2): 161 - 168. [Abstract] [PDF] |
||||
![]() |
T. L. Eadie, K. M. Yorkston, E. R. Klasner, B. J. Dudgeon, J. C. Deitz, C. R. Baylor, R. M. Miller, and D. Amtmann Measuring communicative participation: a review of self-report instruments in speech-language pathology. Am J Speech Lang Pathol, November 1, 2006; 15(4): 307 - 320. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. S. Williams, T. Bakas, E. Brizendine, L. Plue, W. Tu, H. Hendrie, and K. Kroenke How Valid Are Family Proxy Assessments of Stroke Patients' Health-Related Quality of Life? Stroke, August 1, 2006; 37(8): 2081 - 2085. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. Kollen, G. Kwakkel, and E. Lindeman Longitudinal robustness of variables predicting independent gait following severe middle cerebral artery stroke: a prospective cohort study Clinical Rehabilitation, March 1, 2006; 20(3): 262 - 268. [Abstract] [PDF] |
||||
![]() |
M. K. Kapral, J. Fang, M. D. Hill, F. Silver, J. Richards, C. Jaigobin, A. M. Cheung, and for the Investigators of the Registry of the Canad Sex Differences in Stroke Care and Outcomes: Results From the Registry of the Canadian Stroke Network Stroke, April 1, 2005; 36(4): 809 - 814. [Abstract] [Full Text] [PDF] |
||||
![]() |
S.-M. Lai, S. Perera, P. W. Duncan, and R. Bode Physical and Social Functioning After Stroke: Comparison of the Stroke Impact Scale and Short Form-36 Stroke, February 1, 2003; 34(2): 488 - 493. [Abstract] [Full Text] [PDF] |
||||
![]() |
P.W. Duncan, S.M. Lai, R.K. Bode, S. Perera, and J. DeRosa Stroke Impact Scale-16: A brief assessment of physical function Neurology, January 28, 2003; 60(2): 291 - 296. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2002 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |