This Article Full Text (PDF) All Versions of this Article: 34/2/377    most recent 01.STR.0000055766.99908.58v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Newcommon, N. J. Articles by Bone, I. Search for Related Content PubMed PubMed Citation Articles by Newcommon, N. J. Articles by Bone, I.

(Stroke. 2003;34:377.)
© 2003 American Heart Association, Inc.

Letters to the Editor

Improving the Assessment of Outcomes in Stroke: Use of a Structured Interview to Assign Grades on the Modified Rankin Scale

Nancy J. Newcommon, RN, MN; Teri L. Green, MScHRM; Eryka Haley, RN; Timothy Cooke, BSc Michael D. Hill, MD, FRCPC

Calgary Stroke Program, Calgary Health Region, University of Calgary, Alberta, Canada

To the Editor:

We read with interest the article by Wilson et al,¹ describing the use of a structured interview for the modified Rankin Scale (mRS). We attempted a similar exercise to validate the assessment of the mRS over the telephone by nonphysician interviewers using an analogously structured interview.

Thirty-four patients with a discharge diagnosis of stroke participated in the study 3 to 6 months from the day of admission. All patients were seen in person first, where they provided informed consent, by two trained personnel (one stroke neurologist or one of three stroke nurses). Each patient was interviewed consecutively by two interviewers in the Stroke Prevention Clinic or, for badly disabled patients, at a current place of residence (long-term care facility, nursing home, or rehabilitation facility) and the mRS assigned. Each face-to-face interview was followed by a telephone interview conducted by trained interviewers following a structured questionnaire within 5 days of the original interview. No proxy interviews were used. Each person rated the patient on the mRS, blind to the other raters’ score. Overall, three observers scored each patient’s outcome.

Two telephone interviewers were not medical professionals (RN or MD) but had 5 or more years of experience with telephone interviewing, had completed a course in telephone interviewing, and were considered experienced. Questions were scripted and deviations from the script were not permitted. The study was approved by the local Ethics Review Board.

Agreement, assessed using unweighted statistics with 95% confidence limits, was excellent between the two observers in the stroke clinic (=0.72 [0.55 to 0.89]), but less impressive between either clinical observer and the telephone interviewer (=0.38 [0.21 to 0.55] and =0.30 [0.13 to 0.47]).² The telephone interviewer was most likely to agree with the in-clinic observer at the extremes of the scale (mRS=0 or mRS4). In the mid-range where the scale is commonly dichotomized (0 to 2 versus 3 to 5 and 0 to 1 versus 2 to 5), the telephone interviewer correctly rated the patient only 50% of the time.

Predictably, a structured interview for the mRS improves reliability between observers. However, our experience suggests that telephone assessment of the mRS may result in substantial variability in scoring. It remains important for physicians and/or trained nurses to score patients in person rather than relying on telephone interviewers.

References

1. Wilson JTL, Hareendran A, Grant M, Baird T, Schulz UGR, Muir KW, Bone I. Improving the assessment of outcomes in stroke: use of a structured interview to assign grades on the modified Rankin Scale. Stroke. 2002; 33: 2243–2246.[Abstract/Free Full Text]
2. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977; 33: 671–679.[CrossRef]

J.T.L. Wilson, PhD

Department of Psychology, University of Stirling, Stirling, UK

A. Hareendran, PhD

Outcomes Research Pfizer Ltd Sandwich, UK

T. Baird, MD; K.W. Muir, MD I. Bone, MD

Department of Neurology Institute of Neurological Sciences Glasgow, UK

Response

Telephone interviews are a popular method of assessing outcome after stroke because of their low cost and simplicity. Studies have indicated that information concerning activities of daily living after stroke can be reliably obtained by telephone.^1,2 In contrast, Hill and colleagues found low interobserver agreement for the modified Rankin Scale (mRS) when using a structured questionnaire.

The report sounds an appropriate cautionary note and serves as a reminder that it has not been fully established what contributes to the success or failure of telephone interviews. For example, it is not clear whether it is better to adhere to a script or allow rephrasing of questions when there appear to be communication problems. Telephone interviews tend to elicit shorter, less informative responses than face-to-face contact, and this may contribute to disagreement between interviewers. Use of skilled interviewers, as here, should help to minimize these problems. However, in addition to standardizing the questions, we specified the criteria that are used to interpret the responses.³ The midrange of the mRS is particularly problematic in this respect, because the categories are not well defined and therefore open to rater interpretation. Differences in interpretation are more likely when interviewers come from different professional backgrounds, and this may also contribute to the low agreement found by Hill et al. Previous work shows that differences in observers may be more important than differences in modality.¹

We have not yet specifically tested the modified Rankin Scale structured interview (mRS-SI) in a telephone interview, but we hope that the significantly improved interrater reliability of the mRS-SI would be reproduced in this setting. By specifying the definition of each level of the mRS, the mRS-SI removes a large part of the subjectivity that blights distinction between shades of disability. Further work has confirmed that the benefits of the mRS-SI are more striking when raters from varied backgrounds are involved.

References

1. Candelise L, Pinardi G, Aritzu E, Musicco M. Telephone interview for stroke outcome assessment. Cerebrovasc Dis. 1994; 4: 341–343.
2. Korner-Bitensky N, Wood-Dauphinee S, Siemiatycki J, Shapiro S, Becker R. Health-related information postdischarge: telephone versus face-to-face interviewing. Arch Phys Med Rehabil. 1994; 75: 1287–1296.[Medline] [Order article via Infotrieve]
3. Wilson JTL, Hareendran A, Grant M, Baird T, Schulz UGR, Muir KW, Bone I. Improving the assessment of outcomes in stroke: use of a structured interview to assign grades on the modified Rankin Scale. Stroke. 2002; 33: 2243–2246.[Abstract/Free Full Text]

This article has been cited by other articles:

				T. Pfefferkorn, T. E. Mayer, C. Opherk, N. Peters, A. Straube, H.-W. Pfister, M. Holtmannspotter, S. Muller-Schunk, M. Wiesmann, and M. Dichgans Staged Escalation Therapy in Acute Basilar Artery Occlusion: Intravenous Thrombolysis and On-Demand Consecutive Endovascular Mechanical Thrombectomy: Preliminary Experience in 16 Patients Stroke, May 1, 2008; 39(5): 1496 - 1500. [Abstract] [Full Text] [PDF]

				J. L. Banks and C. A. Marotta Outcomes Validity and Reliability of the Modified Rankin Scale: Implications for Stroke Clinical Trials: A Literature Review and Synthesis Stroke, March 1, 2007; 38(3): 1091 - 1096. [Abstract] [Full Text] [PDF]

				J. G. Merino, S. U. Lattimore, and S. Warach Telephone Assessment of Stroke Outcome Is Reliable Stroke, February 1, 2005; 36(2): 232 - 233. [Full Text] [PDF]

This Article Full Text (PDF) All Versions of this Article: 34/2/377    most recent 01.STR.0000055766.99908.58v1 Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Request Permissions Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Newcommon, N. J. Articles by Bone, I. Search for Related Content PubMed PubMed Citation Articles by Newcommon, N. J. Articles by Bone, I.