Determining the Modified Rankin Score After Stroke by Postal and Telephone Questionnaires
Background and Purpose—The modified Rankin Scale (mRS) is the most common outcome measure in large randomized controlled trials in stroke. We tested 2 postal mRS questionnaires and a telephone questionnaire to determine completion rates and intermodality agreement.
Methods—We sent postal questionnaires containing 2 versions of the mRS to surviving stroke patients. One version, tick box, involved the patient/proxy ticking 1 of the 5 descriptions equating to mRS scores; the other, the simplified modified Rankin questionnaire (smRSq), included 5 questions with yes/no responses from which the mRS is derived. We performed a semistructured telephone interview to consenting respondents, blinded to postal responses, to assign an mRS. We compared the mRS obtained by these different methods.
Results—We sent questionnaires to 343 of 356 surviving patients (96%) and received 225 responses (66%). The mRS could not be derived in 27 respondents (12%) and 10 respondents (4%) on the tick box and smRSq, respectively (difference in proportion, 8% [95% CI, 3–13]. One hundred three of 190 respondents (54%) to the postal questionnaire agreed on the tick box versus smRSq version (κ=0.44 [0.38, 0.50]). Agreements between the tick box versus telephone and smRSq versus telephone were 57% (ie, 87/152, κ=0.47 [0.40, 0.55], and 64% (ie, 104/161, κ=0.55 [0.47, 0.62], respectively.
Conclusions—In large studies where face-to-face assessment of mRS is impractical, a postal smRSq with telephone follow-up to nonresponders will achieve higher levels of follow-up than will the tick box version and also good levels of intermodality agreement with least risk of bias.
The modified Rankin Scale (mRS)1 remains the most commonly used primary outcome in large stroke trials and is potentially useful in assessing outcome to audit stroke services.2 Many large trials have assessed mRS by telephone interview and postal questionnaire. Postal questionnaires are easily standardized and remove the professional observer and thus 1 source of variation in outcome measurement. However, there is an inevitable nonresponse rate, even with repeat mailing, so they are usually used in combination with telephone questionnaires for nonresponders. It is important to minimize any differences in mRS obtained from the 2 methods; depending on whether a treatment influences patient outcome and whether the patient responds to the postal questions, any systematic differences between postal and telephone mRS could introduce a bias that could increase or decrease the effect size observed. We have not identified any published studies concerning the feasibility of, and agreement between, postal and telephone methods of assessment. We aimed to: estimate the response rate to a single mailing, compare completion rates between 2 postal versions, and compare the mRS obtained with each version with that obtained by a semistructured telephone interview.
We identified all stroke patients admitted to 2 hospitals in Edinburgh, UK. We sent a postal questionnaire to survivors with a UK postal address. This included the tick box mRS used in most ongoing trials (where patients are asked to choose the mRS descriptor that best describes their current state) and the simplified modified Rankin scale questionnaire (smRSq), which provides high levels of agreement between face to face assessors (78%) and between face to face and telephone assessment (82%).3,4 In patients agreeing to be telephoned, an author delivered a semistructured telephone interview based on the Rankin Focused Assessment (RFA),5 blinded to the results of the postal questionnaire.
Between July 1, 2010, and Dec 31, 2010, 461 stroke patients were admitted. We sent postal questionnaires to 343 patients (96%) of 356 surviving patients (86%), and 225 patients (66%) returned a questionnaire. Those who responded and those who agreed to be phoned tended to be younger (and consequently were men), had less previous disability, and had fewer communication and motor problems and thus shorter lengths of hospital stay than those who did not; however, none of these differences were statistically significant.
Twenty-seven patients (12%) did not complete the tick box mRs appropriately, of whom 7 patients left it blank, and 20 ticked 2 or more boxes. Fifteen patients (7%) did not answer all the smRSq questions, but we could derive an mRS from the published algorithm4 in all but 10 patients (4%; difference in proportions, 8% [95% CI, 3–13]).
Tables 1, 2, and 3 show the responses and agreement between the 2 postal versions and between the tick box and smRSq and the telephone mRS. The agreement with telephone mRS was higher for the smRSq than for the tick box, but this was not statistically significant (difference in proportional agreement, 7% [95% CI, 3–18]).
Overall, patients were categorized as more disabled on the tick box than on the telephone mRS, whereas there was less systematic difference between those derived from the smRSq and the telephone interview.
To establish whether a strategy of using both postal questionnaires might increase the proportional agreement with telephone mRS, we carried out additional analyses. Of the 171 patients who were telephoned, the tick box and smRSq agreed in 82 patients (48%). In 69 (84%; 95% CI, 76–92) of these 82 patients, the telephone mRS matched the postal mRS. The agreement between telephone and postal questionnaire could be enhanced (compared with the smRSq version alone) by accepting only those postal mRS where the 2 versions of the postal mRS matched (difference of proportional agreement, 20% [95% CI, 9–30]).
About two thirds of patients responded to a single postal questionnaire comprising 2 versions of the mRS. There were fewer uninterpretable responses for the smRSq than for the tick box version. Although a repeat posting is likely to increase this proportion of postal responders in a trial, an additional method of follow-up is needed; the most practical is a telephone questionnaire. An mRS derived from the smRSq delivered by postal questionnaire is more often in agreement with 1 assigned during a semistructured telephone questionnaire than with a simple tick box postal mRS. If the postal mRS is only used where both postal versions agree, then agreement with a telephone assessment is 20% higher; however, this is at the expense of having to telephone more than half of all patients, which would greatly increase the resources required in a trial.
Our study has several limitations. We did not test construct or concurrent validity of the postal or telephone mRS, we did not carry out any face-to-face interviews to determine how the mRS based on postal or telephone follow-up relate to those, nor did we assess the repeatability (or test-retest reliability) of the postal mRS. Although we included larger numbers than did most previous studies of mRS reliability, we had too few numbers to indicate reliably whether differences in agreement between measures, and the patients completing them, were statistically significant. Last, we only tested our postal and telephone follow-up in patients or proxies who could speak English.
Despite its widespread use in large randomized trials and observational studies, we have not identified any previous studies that have evaluated the tick box postal mRS. The smRSq can be delivered by telephone, and mRS based on a telephone administration has excellent agreement with a face-to-face assessment.4 Based on our results, we recommend that studies in which face-to-face interviews are impractical might use a combination of a postal smRSq and a telephone follow-up for nonresponders. This combination will result in the best postal completion rate and roughly equivalent scores with both methods; this minimizes the chance of bias resulting from nonresponse to postal questionnaire. We have not used the telephone smRSq in this study, but it seems likely that agreement between postal and telephone follow-up would be enhanced by doing so.
- Received September 22, 2011.
- Accepted October 17, 2011.
- © 2012 American Heart Association, Inc.
- van Swieten JC,
- Koudstaal PJ,
- Visser MC,
- Schouten HJ,
- van Gijn J
- Bruno A,
- Shah N,
- Lin C,
- Close B,
- Hess DC,
- Davis K,
- et al
- Bruno A,
- Akinwuntan AE,
- Lin C,
- Close B,
- Davis K,
- Baute V,
- et al
- Saver JL,
- Filip B,
- Hamilton S,
- Yanes A,
- Craig S,
- Cho M,
- et al