| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2003;34:1741.)
© 2003 American Heart Association, Inc.
Original Contributions |
From the School of Occupational Therapy, College of Medicine, National Taiwan University, Taipei, Taiwan (I.-P.H., C.-L.H.); School of Physical Therapy, College of Medical Technology, Chun-Shan Medical University, Taichung, Taiwan (C.-H.W.); and Department of Psychology, DePaul University, Chicago, Ill (C.-F.S.).
Correspondence to Ching-Lin Hsieh, School of Occupational Therapy, College of Medicine, National Taiwan University, No 7, Chung-Shan S Rd, Taipei 100, Taiwan, ROC. E-mail mike26{at}ha.mc.ntu.edu.tw
| Abstract |
|---|
|
|
|---|
Methods The validity and responsiveness of the 3 mobility measures were prospectively examined by monitoring 57 stroke patients with the measures and the Barthel Index at 14, 30, 90, and 180 days after stroke onset. Two individual raters used the 3 measures to evaluate a different sample of 40 patients on 2 separate occasions to determine the interrater reliability.
Results The Spearman
between STREAM and MRMI was
BORDER="0">0.92; the intraclass correlation coefficient (ICC, a measure of agreement) between them was
0.89, indicating high concurrent validity of both measures. RMI showed a moderate to high relationship and agreement with STREAM and MRMI (
0.78, ICC
0.5). Responsiveness of the 3 measures was high before 90 days after stroke onset (standardized response mean
0.83) and low at 90 to 180 days after stroke onset (0.2
standardized response mean
0.4). The score changes of the 3 measures at each stage were significant (P
0.05), except for RMI and MRMI at 90 to 180 days after stroke onset. The interrater agreement of the 3 measures was high (ICC
BORDER="0">0.92).
Conclusions All 3 measures examined showed acceptable levels of reliability, validity, and responsiveness in stroke patients. The psychometric characteristics of STREAM were slightly superior to those of the other 2 measures among our patients. We prefer and recommend STREAM for measuring mobility disability in stroke patients.
Key Words: cerebrovascular accident disability evaluation psychometrics
| Introduction |
|---|
|
|
|---|
The Rivermead Mobility Index (RMI), one of the few clinical mobility measures specifically designed for stroke patients, is the only mobility measure endorsed by the US Agency for Health Care Policy and Research.4 RMI has been used in several studies to examine treatment effect on mobility.5 According to Lennon and Hastings,6 the responsiveness of RMI is poor because its items are scored on a dichotomous (yes/no) basis. Therefore, the modified RMI (MRMI)2 was developed to increase the responsiveness of the measure by extending the scoring level to 6 points. However, the psychometric properties of the MRMI have yet to be systematically explored. The Mobility Subscale of the Stroke Rehabilitation Assessment of Movement Measure (STREAM)7 that assesses mobility after stroke is a more recently developed measure. STREAM is simple to administer and is reliable and valid in stroke patients.7,8 Although the 3 measures appear to be accepted by both clinicians and researchers to measure mobility disability after stroke, to the best of our knowledge, no empirical data exist to determine which measure is most psychometrically sound.
Comparison of the psychometric properties of clinical mobility measures can provide useful guidelines for both clinicians and researchers to determine an objective and scientific measure.9 The purpose of this study was to compare the reliability, validity, and responsiveness of RMI, MRMI, and STREAM in stroke patients.
| Methods |
|---|
|
|
|---|
Procedures
The study protocol consisted of 2 parts. The first part was a validity and responsiveness study. The 3 mobility measures and the Barthel Activities of Daily Living Index (BI)10 were administered to patients at 14, 30, 90, and 180 days after stroke onset. The BI was used as the external criterion to examine convergent and predictive validity. Initial stroke severity was ascertained with the Canadian Neurological Scale11 applied retrospectively to medical records. Degrees of responsiveness of the mobility measures were calculated from the changes occurring between 14 to 30, 30 to 90, 90 to 180, 14 to 90, and 14 to 180 days after stroke onset.
When necessary, patients were allowed to rest during the testing protocol, which lasted
1 hour. All of the above assessments were made by occupational therapist A, according to previously published standardized methods of administration.2,1114
The second part of the protocol was an interrater reliability study. Forty stroke patients at a rehabilitation unit participated in this part of the study. The 3 mobility measures were administered separately by 2 occupational therapists (A and B). To minimize the effects of possible recovery, assessments were administered within a 24-hour period according to a counterbalanced sequence. The therapists were blinded to the results of each others assessments during the study period.
Instruments
RMI, which covers a range of hierarchical activities from turning over in bed to running, comprises 14 questions and 1 direct observation.13 Each patients mobility performance is rated primarily by interviewing the patients and/or their primary caregiver. The highest score, 15, indicates the highest mobility status. Although previous studies3,13,15 found that RMI had good psychometric properties in stroke patients, sample sizes in 2 of these studies were modest (
23),13,15 limiting generalization of their results.
MRMI has 8 test items: turning over, changing from lying to sitting, maintaining sitting balance, going from sitting to standing, standing, transferring, walking indoors, and climbing stairs. Scores of the MRMI range from 0 to 40. One main characteristic of the MRMI is that patients are scored by direct observation of their performance on the items. Lennon and Johnson2 have proposed that the psychometric characteristics of the MRMI be examined further.
STREAM7 measures mobility after stroke by direct observation. It contains 10 four-point items: rolling, bridging, going from supine to sitting, changing from sitting to standing, standing, placing affected foot onto first step, stepping backward, stepping to affected side, walking 10 m, and walking down stairs. Scores of STREAM range from 0 to 30. Although the reliability and validity of STREAM are high in stroke patients,8,12 its responsiveness has not been reported.
The BI is a weighted scale of 10 items of basic activities of daily living (ADL).10 The highest score, 100, indicates that the patient is fully independent in physical function; the lowest score, 0, represents a totally dependent, bedridden state. The reliability, validity, and responsiveness of the BI are well established in stroke patients.16,17
Stroke severity at admission was determined by the Canadian Neurological Scale as described by Goldstein and Chilukuri.11 The score ranges from 0 to 11.5. This instrument has been shown to be valid and reliable in assessing stroke severity.11,18
Statistical Analyses
A mobility measure should be able to reflect the whole range of mobility disability after stroke. We calculated the floor and ceiling effects, representing the percentage of subjects achieving the lowest and highest scores possible, respectively. Floor and ceiling effects exceeding 20% of sample size are considered to be significant,19 indicating that the measure can represent only a limited range of mobility disability.
Concurrent validity is usually established by demonstrating a high correlation or agreement between the measure and a gold standard.16 Because each of the 3 measures used has a different score range, the scores from each measure were converted to a range of 0 to 100.16 The relationship and agreement between the 3 mobility measures at 4 time points were examined by use of the Spearman correlation coefficient (
) and the intraclass correlation coefficient (ICC), respectively.
The convergent validity of the mobility measures was assessed by examining the relationships between the total scores of the mobility measures and those of the BI at all 4 time points after stroke using Spearmans
. The predictive validity of the mobility measures was assessed by examining the associations between results of the mobility measures at 3 time points (14, 30, and 90 days after stroke onset) and those of the BI at 180 days after stroke onset using the Spearman
.
Responsiveness was examined with the standardized response mean (SRM), 1 type of effect size. SRM was calculated by dividing the mean change scores by the SD of the change score in the same subjects. An effect size >0.8 is usually considered large; 0.5 to 0.8, moderate; and 0.2 to 0.5, small.20 Wilcoxon matched-pairs signed-rank tests were performed to determine the statistical significance of the change scores.
The interrater agreement on individual items of the mobility measures was analyzed with the quadratic weighted
statistic. The interrater agreement of the total score of the mobility measures was analyzed with the ICC statistic. The fixed effect of ICC model 321 was used to compute the ICC value for interrater reliability. Both weighted
and ICC values
0.80 indicate very good agreement; 0.60 to 0.79, good agreement; 0.40 to 0.59, moderate agreement; 0.20 to 0.39, fair agreement; and 0 to 0.2, poor agreement.18
| Results |
|---|
|
|
|---|
95). Characteristics of the study sample are presented in Table 1.
|
The interquartile score range of the RMI at baseline was quite limited (Table 1). Furthermore, Table 2 shows that, except for RMI, none of the mobility measures exhibited significant floor or ceiling effects at the 4 time points after stroke. These results indicate that MRMI and STREAM demonstrated acceptable distribution from the acute stage up to 180 days after stroke onset.
|
Table 3 shows that the correlation (
0.92) and agreement (ICC
0.89) between STREAM and MRMI were high, indicating that both measures had high concurrent validity. RMI showed moderate to high concurrent validity (
0.78, ICC
0.50) when evaluated against STREAM and MRMI. Table 4 shows that the 3 mobility measures had high convergent validity (
0.72) and acceptable predictive validity (
0.5).
|
|
The 3 mobility measures were highly responsive in detecting changes before 90 days after stroke onset (14 to 30 days, SRM
BORDER="0">1.14; 30 to 90 days, SRM
0.83; Table 5). At 90 to 180 days after stroke onset, the levels of responsiveness of these measures, as expected, were low (0.2
SRM
0.4). The changes shown by the 3 measures at each stage were all significant (P<0.05), except for those shown by RMI and MRMI at 90 to 180 days after stroke onset (P>0.14).
|
Forty patients participated in the interrater reliability study. This sample consisted of 19 men and 21 women with a mean age of 63 years (SD, 10.2 years). The medians of the weighted
statistic for each item of RMI, MRMI, and STREAM were 0.71 (range, 0.37 to 0.94), 0.72 (range, 0.47 to 0.9), and 0.81 (range, 0.55 to 0.89), respectively, indicating generally acceptable interrater agreement on the item level. Four RMI items, 2 MRMI items, and 1 STREAM item had fair to moderate agreement (0.37

0.6). The ICCs for the total scores of RMI, MRMI, and STREAM were 0.92 (95% confidence interval [CI], 0.84 to 0.96), 0.95 (95% CI, 0.90 to 0.97), and 0.97 (95% CI, 0.95 to 0.99), respectively, indicating excellent total score agreement.
| Discussion |
|---|
|
|
|---|
The score distributions of the measures of the study sample should not exhibit severe ceiling or floor effects. We found that all 3 mobility measures demonstrated acceptable distributions from the acute stage up to 180 days after stroke onset, except for RMI, which at 14 days after stroke onset showed a limited score range and a notable floor effect. These results indicate that the RMI might not adequately characterize patients mobility functions in the early stages of stroke, especially for patients with severe disabilities.
The validity of the 3 mobility measures has been reported in previous studies.2,3,8,12,13 However, given the notable heterogeneity of stroke effects, comparison with previous results was difficult because they were not examined in the same group of patients. The validity of the 3 measures was first compared in a cohort of patients in this study. We found that STREAM and MRMI had high concurrent, convergent, and predictive validity, whereas RMI was not highly correlated with either STREAM or MRMI. It is of note that the RMI score was retrieved mainly via interview, whereas the STREAM and MRMI scores were based on direct observation of a patients performance. Patients may overstate their functional abilities.22 Our results suggest that among our patients, STREAM and MRMI were more valid measures of mobility after stroke than RMI. However, the differences in validity between the 3 measures may not be statistically significant.
Responsiveness is important for any measurement tool designed to measure change over time.23 Nonetheless, the responsiveness of most mobility measures has yet to be examined. The 3 mobility measures were highly responsive in detecting changes before 90 days after stroke onset. All changes in the 3 measures at each stage were significant, except for the RMI and MRMI at 90 to 180 days after stroke onset. According to the present results, STREAM was slightly more responsive than the other 2 measures. Responsiveness of the 3 measures was, as expected, low at later stages of recovery (90 to 180 days after stroke onset). One possible explanation is that patients improvements in mobility had reached a plateau after 90 days after stroke onset. Moreover, balance, motor, and ADL functions showed only minor improvement after 90 days after stroke onset.24,25 Another possible explanation is that these 3 mobility measures lack items sensitive enough to detect change >90 days after stroke onset.
Interestingly, MRMI was no more responsive than RMI, despite the fact that MRMI, with more scoring levels, was revised from RMI to make it more responsive.2 Some recent studies have demonstrated that an increase in the number of items or grading levels does not necessarily improve the responsiveness or difference detection between patient groups of mobility measures,26 balance measures,24 and ADL measures.16,27,28 These results support the argument that selection of a measurement tool should be based on empirical evidence, not on clinical opinion.27
Interrater agreement on individual items of mobility measures has rarely been examined. The interrater agreement of STREAM was high for individual items and the total scores. Although the total score interrater agreement of the RMI and MRMI was high, at least 2 items of both measures demonstrated only fair to moderate agreement between raters. These findings indicate that the interrater reliability of STREAM is slightly higher than that of RMI and MRMI.
Some mobility measures were not selected for comparison in this study. For example, the gait speed test (eg, 10-m walking speed test and 6-minute walking distance) is commonly used to measure mobility after stroke in both clinical and research settings. However, the gait speed test is not relevant for all patients with stroke. Mobility, by nature, is complex and multifactorial, whereas the gait speed test simply reflects 1 unique and specific dimension of mobility. Furthermore, the speed test cannot be used for the patients without the ability to walk. On the other hand, the 3 mobility measures used in this study measure patients performance on some tasks that reflect the multifactorial nature of mobility. Furthermore, the 3 mobility measures examined in this study are feasible for assessing most stroke patients, including those with very poor mobility.
A limitation of the present study is that the intrarater reliability of the measures was not examined. We found a high interrater reliability of the measures. Therefore, the intrarater reliability of the 3 measures might not be an issue of great concern. Another limitation is that the sample size in this study was not large enough to further analyze the data according to type or severity of stroke. Because the type of stroke or level of severity could affect the results of these measures, further studies with larger sample sizes are necessary to analyze these effects on the psychometric characteristics of the measures. Furthermore, the psychometric properties of STREAM appeared to be slightly better than those of the other 2 measures (eg, RMI showed a notable floor effect in the early stages of stroke; the score changes of RMI and MRMI at 90 to 180 days after stroke onset were not significant). However, the psychometric differences among the 3 measures may not be statistically significant.
In summary, the 3 mobility measures examined demonstrated acceptable levels of reliability, validity, and responsiveness among our stroke patients. The psychometric characteristics of STREAM were slightly superior to those of RMI and MRMI. We prefer and recommend STREAM for assessing mobility disability after stroke in both clinical and research settings.
| Acknowledgments |
|---|
Received October 29, 2002; revision received January 17, 2003; accepted January 28, 2003.
| References |
|---|
|
|
|---|
2. Lennon S, Johnson L. The modified Rivermead Mobility Index: validity and reliability. Disabil Rehabil. 2000; 22: 833839.[CrossRef][Medline] [Order article via Infotrieve]
3. Hsieh CL, Hsueh IP, Mao HF. Validity and responsiveness of the Rivermead Mobility Index in stroke patients. Scand J Rehabil Med. 2000; 32: 140142.[CrossRef][Medline] [Order article via Infotrieve]
4. Gresham G, Duncan P, Stason W. Post-Stroke Rehabilitation: Assessment, Referral, and Patient Management: Quick Reference Guide Number 16. Rockville, Md: US Agency for Health Care Policy and Research. AHCPR publication No. 95-0663, 1995.
5. Forlander DA, Bohannon RW. Rivermead Mobility Index: a brief review of research to date. Clin Rehabil. 1999; 13: 97100.
6. Lennon S, Hastings M. Key physiotherapy indicators for quality of stroke care. Physiotherapy. 1996; 82: 655664.[CrossRef]
7. Daley K, Mayo N, Danys I, Cabot R, Wood-Dauphinee S. The Stroke Rehabilitation Assessment of Movement (STREAM): refining and validating the content. Physiother Can. 1997; 49: 269278.
8. Wang CH, Hsieh CL, Dai MH, Chen CH, Lai YF. Inter-rater reliability and validity of the Stroke Rehabilitation Assessment of Movement (STREAM) instrument. J Rehabil Med. 2002; 34: 2024.[CrossRef][Medline] [Order article via Infotrieve]
9. Spilg EG, Martin BJ, Mitchell SL, Aitchison TC. A comparison of mobility assessments in a geriatric day hospital. Clin Rehabil. 2001; 15: 296300.
10. Mahoney F, Barthel D. Functional evaluation: the Barthel Index. Md State Med J. 1965; 14: 6165.[Medline] [Order article via Infotrieve]
11. Goldstein LB, Chilukuri V. Retrospective assessment of initial stroke severity with the Canadian Neurological Scale. Stroke. 1997; 28: 11811184.
12. Daley K, Mayo N, Wood-Dauphinee S. Reliability of scores on the Stroke Rehabilitation Assessment of Movement (STREAM) measure. Phys Ther. 1999; 79: 819.
13. Collen FM, Wade DT, Robb GF, Bradshaw CM. The Rivermead Mobility Index: a further development of the Rivermead motor assessment. Int Disabil Stud. 1991; 13: 5054.[Medline] [Order article via Infotrieve]
14. Wade DT, Collin C. The Barthel ADL Index: a standard measure of physical disability? Int Disabil Stud. 1988; 10: 6467.[Medline] [Order article via Infotrieve]
15. Green J, Forster A, Young J. A test-retest reliability study of the Barthel Index, the Rivermead Mobility Index, the Nottingham Extended Activities of Daily Living Scale and the Frenchay Activities Index in Stroke Patients. Disabil Rehabil. 2001; 23: 670676.[CrossRef][Medline] [Order article via Infotrieve]
16. Hsueh IP, Lin JH, Jeng JS, Hsieh CL. Comparison of the psychometric characteristics of the Functional Independence Measure, 5 item Barthel Index, and 10 item Barthel Index in patients with stroke. J Neurol Neurosurg Psychiatry. 2002; 73: 188190.
17. Hsueh IP, Lee MM, Hsieh CL. Psychometric characteristics of the Barthel Activities of Daily Living Index in stroke patients. J Formos Med Assoc. 2001; 100: 526532.[Medline] [Order article via Infotrieve]
18. Bushnell CD, Johnston DC, Goldstein LB. Retrospective assessment of initial stroke severity: comparison of the NIH Stroke Scale and the Canadian Neurological Scale. Stroke. 2001; 32: 656660.
19. Holmes WC, Shea JA. Performance of a new, HIV/AIDS-targeted quality of life (HAT-QoL) instrument in asymptomatic seropositive individuals. Qual Life Res. 1997; 6: 561571.[CrossRef][Medline] [Order article via Infotrieve]
20. Cohen J. Statistical Power Analysis for the Behavioral Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
21. Shrout P, Fleiss J. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979; 86: 420428.[CrossRef][Medline] [Order article via Infotrieve]
22. Dorman PJ, Waddell F, Slattery J, Dennis M, Sandercock P. Are proxy assessments of health status after stroke with the EuroQol questionnaire feasible, accurate, and unbiased? Stroke. 1997; 28: 18831887.
23. Guyatt G, Walter S, Norman G. Measuring change over time: assessing the usefulness of evaluative instruments. J Chronic Dis. 1987; 40: 171178.[CrossRef][Medline] [Order article via Infotrieve]
24. Mao HF, Hsueh IP, Tang PF, Sheu CF, Hsieh CL. Analysis and comparison of the psychometric properties of three balance measures for stroke patients. Stroke. 2002; 33: 10221027.
25. Jorgensen HS, Nakayama H, Raaschou HO, Vive-Larsen J, Stoier M, Olsen TS. Outcome and time course of recovery in stroke, II: time course of recovery: the Copenhagen Stroke Study. Arch Phys Med Rehabil. 1995; 76: 406412.[CrossRef][Medline] [Order article via Infotrieve]
26. Rossier P, Wade DT. Validity and reliability comparison of 4 mobility measures in patients presenting with neurologic impairment. Arch Phys Med Rehabil. 2001; 82: 913.[CrossRef][Medline] [Order article via Infotrieve]
27. Hobart JC, Thompson AJ. The five item Barthel Index. J Neurol Neurosurg Psychiatry. 2001; 71: 225230.
28. Wallace D, Duncan PW, Lai SM. Comparison of the responsiveness of the Barthel Index and the Motor Component of the Functional Independence Measure in stroke: the impact of using different methods for measuring responsiveness. J Clin Epidemiol. 2002; 55: 922928.[CrossRef][Medline] [Order article via Infotrieve]
This article has been cited by other articles:
![]() |
H.-M. Chen, C.-L. Hsieh, Sing Kai Lo, L.-J. Liaw, S.-M. Chen, and J.-H. Lin The Test-Retest Reliability of 2 Mobility Performance Tests in Patients With Chronic Stroke Neurorehabil Neural Repair, July 1, 2007; 21(4): 347 - 352. [Abstract] [PDF] |
||||
![]() |
I-P. Hsueh, W.-C. Wang, C.-H. Wang, C.-F. Sheu, S.-K. Lo, J.-H. Lin, and C.-L. Hsieh A Simplified Stroke Rehabilitation Assessment of Movement Instrument Physical Therapy, July 1, 2006; 86(7): 936 - 943. [Abstract] [Full Text] [PDF] |
||||
![]() |
L.-J. Liaw, C.-L. Hsieh, S.-K. Lo, S. Lee, M.-H. Huang, and J.-H. Lin Psychometric properties of the modified Emory Functional Ambulation Profile in stroke patients Clinical Rehabilitation, May 1, 2006; 20(5): 429 - 437. [Abstract] [PDF] |
||||
![]() |
I. G.L. van de Port, G. Kwakkel, I. van Wijk, and E. Lindeman Susceptibility to Deterioration of Mobility Long-Term After Stroke: A Prospective Cohort Study Stroke, January 1, 2006; 37(1): 167 - 171. [Abstract] [Full Text] [PDF] |
||||
![]() |
Q. P. Tang, Q. D. Yang, Y. H. Wu, G. Q. Wang, Z. L. Huang, Z. J. Liu, X. S. Huang, L. Zhou, P. M. Yang, and Z. Y. Fan Effects of Problem-Oriented Willed-Movement Therapy on Motor Abilities for People With Poststroke Cognitive Deficits Physical Therapy, October 1, 2005; 85(10): 1020 - 1033. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2003 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |