Response to Letter by Quinn et al
We appreciate the comments by Dr Quinn et al concerning our report on the validity and reliability of the modified Rankin Scale (mRS).1 We agree with the authors in advocating caution in interpreting reliability results for the mRS, given the small number of tests that have been conducted and the limitations of traditional reliability testing. We also agree on emphasizing the importance of accurately classifying clinical end points for randomized clinical trials. This issue is especially relevant to the mRS because it is comprised of a small number of categories for which misclassification can have a substantial impact on statistical power.
Reliability is a complex concept that attempts to ascertain the degree to which an assessment or classification is affected by nonrandom sources of error on the part of raters.2 As the authors point out, multiple sources of error variance (eg, nationality, training of the rater) may contribute to misclassification of health levels. Unfortunately, commonly applied tests of reliability may aggregate those sources into single statistics which do not permit a detailed understanding of the performance of an instrument. The feasibility of using more comprehensive tests, such as Rasch analysis or generalizability theory, is unclear in the absence of uniformly applied structured interviews (comprised of items posed to the patient/caregiver). To this point, video training is a more efficient and practical approach to improving mRS reliability.
Banks J, Marotta CA. Outcomes validity and reliability of the modified Rankin scale: implications for stroke clinical trials. A literature review and synthesis. Stroke. 2007; 38: 1091–1096.
McDowell I, Newell C. Measuring Health: A Guide to Rating Scales and Questionnaires (2nd edition). New York: Oxford University Press; 1996.