(Stroke. 1999;30:2141-2145.)
© 1999 American Heart Association, Inc.
Original Contributions |
From the Departments of Neurology (S.S., M.M.G., L.H.S.), Dermatology (J.C.K.), and Medicine (Y.C.), Massachusetts General Hospital, and Center for Telemedicine, Partners Healthcare Inc (S.S., J.C.K., L.H.S.), Boston, Mass.
Correspondence to Lee H. Schwamm, MD, Department of Neurology, VBK 915, Massachusetts General Hospital, 55 Fruit St, Boston, MA 02114. E-mail lschwamm{at}partners.org
| Abstract |
|---|
|
|
|---|
MethodsOne bedside and 1 remote NIHSS score were independently obtained on each of 20 patients with ischemic stroke. The bedside examination was performed by a stroke neurologist at the patient's bedside. The remote examination was performed by a second stroke neurologist through an interactive high-speed audio-video link, assisted by a nurse at the patient's bedside. Kappa coefficients were calculated for concordance between bedside and remote scores.
ResultsRemote assessments took slightly longer than bedside
assessments (mean 9.70 versus 6.55 minutes, P<0.001).
NIHSS scores ranged from 1 through 24. Based on weighted
coefficients, 4 items (orientation, motor arm, motor leg, and neglect)
displayed excellent agreement, 6 items (language, dysarthria,
sensation, visual fields, facial palsy, and gaze) displayed good
agreement, and 2 items (commands and ataxia) displayed poor agreement.
Total NIHSS scores obtained by bedside and remote methods were strongly
correlated (r=0.97, P<0.001).
ConclusionsThe NIH Stroke Scale remains a swift and reliable clinical instrument when used over interactive video. Application of this technology can bring stroke expertise to the bedside, regardless of patient location.
Key Words: reliability stroke assessment stroke, ischemic telemedicine
| Introduction |
|---|
|
|
|---|
Because measurement of the degree of neurological deficit is a critical first step in acute stroke evaluation, telemedicine in this setting requires audio-video data transmission with enough speed and clarity to ensure adequate neurological evaluation. The National Institutes of Health Stroke Scale (NIHSS) is a validated, 13-item examination tool for measuring stroke deficit with established interrater reliability.4 5 6 7 8 It contains most of the neurological elements needed for clinical decision making and has been used as an entry criterion in trials of acute stroke therapy. To assess the feasibility of remote stroke evaluations, we tested the hypothesis that interrater reliability for the NIHSS would extend to a telemedicine paradigm.
| Subjects and Methods |
|---|
|
|
|---|
The telemedicine link is shown in schematic form in Figure 1
. All examinations were performed with
patients on stretchers, in front of a VTEL SmartStation TC 1000 (VTEL
Corporation) and viewed remotely on a Pentium-based desktop
personal computer with a proprietary Codec card (VTEL Corporation) and
21-inch color monitor (Mitsubishi Electric Corporation) set at 800x600
pixel resolution. The connection platform included a user interface,
Windows 95based conference control interface software, a PCI Codec
card, an ISA TRI-BRI Promptus I-Mux card, and an integrated PTZ Sony
camera (all from VTEL Corporation). An external microphone anchored by
tape to the patient's pillow captured audio information. Data were
transmitted over 3 parallel integrated services digital network (ISDN)
lines at speeds up to 384 kilobits per second. The video transmission
used standard compression algorithms with full CIF and was viewed at 30
frames per second. Two preset camera angles were used in all remote
examinations: a zoom close-up for assessing facial weakness and eye
movements and a wide angle for all other items (Figure 2
). To minimize the need for shifting
camera views, wide-angle items were performed first in the standard
sequence,5 followed by assessment of facial weakness and
eye movements with close-up zoom. For every patient, both the bedside
and the remote neurologists scored each item of the NIHSS and
recorded the duration of each examination. For the language and
dysarthria items, standard full-size NIHSS materials were used (cookie
jar picture, naming sheet, word list, and sentence list). After
completion of the NIHSS examinations, information on infarct location
(categorized by vascular territory) and infarct size (categorized as
small, medium, and large based on involvement of <one third, <two
thirds, or >two thirds of the affected vascular territory,
respectively) was collected for each patient.
|
|
Interrater agreement between bedside and remote scores was measured by
computing weighted
statistics for each item of the NIHSS and by
calculating the interrater correlation coefficient for the total NIHSS
scores. In contrast to the unweighted
statistic, the weighted
method assigns weights to disagreements based on the magnitude of
discrepancy9 and may be a more accurate measure for a
discrete ordinal scale such as the NIHSS. Published standards for
interpretation of the
statistic were used (values >0.75, excellent
agreement beyond chance; values between 0.40 and 0.75, fair to good
agreement beyond chance; values <0.40, poor agreement beyond
chance).9 Mean examination times for bedside versus remote
assessments were compared using a 2-tailed Student
t-test.
| Results |
|---|
|
|
|---|
|
The NIHSS scores obtained ranged from 1 through 24. Bedside and remote examiners did not differ on any patient by >3 points. There was a strong linear correlation between total bedside and total remote scores (interrater correlation coefficient=0.97, P<0.001).
Based on weighted
scores, 4 NIHSS items (orientation, motor arm,
motor leg, and neglect) displayed excellent agreement beyond chance, 6
items (language, dysarthria, sensation, visual fields, facial palsy,
and gaze) displayed good agreement beyond chance, and 3 items (level of
arousal, commands, and ataxia) displayed poor to no agreement beyond
chance. Table 2
compares
coefficients for each scale item in
our study with those reported from 2 other studies4 5 of
NIHSS interrater reliability.
|
| Discussion |
|---|
|
|
|---|
The current study demonstrates that it is feasible to perform remote
neurological assessments with widely available and relatively
inexpensive (under $10 000 and decreasing) telemedicine technology.
The majority of studies attempting to validate telemedicine systems
have focused on asynchronous processes, such as review of radiological
and dermatologic images and of pathological specimens.13
The NIHSS is a clinical tool with demonstrated interrater reliability
that is frequently used to evaluate acute stroke patients and assess
their outcomes after therapy.6 14 15 When we compared
itemized NIHSS scores obtained by bedside examination with scores for
the same patients obtained remotely through a telemedicine link, the
degree of interrater agreement on the various scale items
paralleled the interrater agreement previously documented for
bedside-bedside comparisons. Previous studies have used the difference
of
4 points on the NIHSS to reflect a clinically significant change
beyond interrater variability.1 8 16 It is worth noting
that the remote and bedside examiners in the current study did not
differ in the total scores on any patient by >3 points (Table 1
). As seen in Table 2
, the
coefficients obtained in
our study compare favorably with the corresponding values reported by
Goldstein et al4 and Brott et al,5 who
studied the interrater agreement of bedside-administered NIHSS in 20
and 24 patients, respectively. The comparison suggests that certain
NIHSS items (including dysarthria, ataxia, gaze, and facial palsy) have
consistently inferior interrater agreement while
other items (including orientation, motor arm, motor leg and language)
have consistently superior interrater agreement. Our
telemedicine-based study found fair interrater agreement for best gaze
and facial palsy, suggesting that the technology provided adequate
spatial and temporal resolution for close-up evaluation of the face and
eyes (Figure 2
). One of the items in our study (neglect) had
greater interrater agreement compared with earlier reports. We also
found that 1 of the items (performance of verbal commands) had
poor interrater agreement compared with the results of Goldstein et
al4 and Brott et al.5 However, there was no
consistent pattern to the disagreement. The different pattern
of interrater agreement in our study may result from the natural
variability to be expected in studies of interrater reliability or it
may reflect differences inherent in the telemedicine paradigm. The
latter possibility suggests that detailed scoring and administration
guidelines may have to be adapted for an NIH "telestroke"
scale.
In its present form, the NIHSS is amenable to administration in a telemedicine environment. The modest increase in duration of telemedicine-administered NIHSS (average of 3.15 minutes compared with bedside administration) would not be expected to have a significant impact on clinical care. Proper testing of visual fields, sensation, and neglect requires assistance at the bedside. Additionally, while items such as language, ataxia, and performance of verbal commands are testable without assistance, their administration is facilitated by a bedside assistant. The bedside assistant need not have neurological expertise, because the specialist can expertly direct the assistant through interactive video. The adequacy of a nonspecialist assistant in this model suggests that remote administration of the NIHSS may be implemented in diverse settings, including patients in the emergency medical services system or prehospital arena. This may be important for tertiary care centers serving widely distributed, low-density rural populations. The use of training videotapes to demonstrate the performance of the NIHSS over a telemedicine link may enhance the reliability of "telestroke" assessments.7
To avoid interruption of the time-sensitive nature of acute stroke evaluation with an unproven modality (ie, telemedicine-administered neurological examination), we deliberately excluded patients in the acute phase. Recruitment of clinically stable patients also minimized examination variability attributable to clinical fluctuations in the acute setting. Because of the subacute nature of our test bed, the current data must be considered preliminary in determining their potential impact on actual clinical decision making. Our future research aims to reproduce the results of this study in acute and hyperacute stroke patients under real emergency room conditions. It is also important to note that while telemedicine-enabled neurological assessment may expedite stroke-related decision making, it cannot and should not be thought of as a substitute for the comprehensive clinical evaluation of the acute stroke patient, including thorough medical and cardiac evaluations.
| Acknowledgments |
|---|
Received April 9, 1999; revision received June 17, 1999; accepted July 13, 1999.
| References |
|---|
|
|
|---|
2. Furlan AJ, Higashida R, Wechsler L, Schulz G, PROACT II Investigators. PROACT II: Recombinant prourokinase (r-ProUK) in acute cerebral thrombo-embolism. In: Program of the 24th American Heart Association International Conference on Stroke and Cerebral Circulation; February 46, 1999; Nashville, Tenn. Abstract 15.
3.
Perednia DA, Allen A. Telemedicine technology and
clinical applications. JAMA. 1995;273:483488.
4.
Goldstein LB, Bertels C, Davis JN. Interrater
reliability of the NIH Stroke Scale. Arch Neurol. 1989;46:660662.
5.
Brott T, Adams HP, Olinger CP, Marler JR, Barsan WG,
Biller J, Spilker J, Holleran R, Eberle R, Hertzberg V, Rorick M,
Moomaw CJ, Walker M. Measurements of acute cerebral infarction: a
clinical examination scale. Stroke. 1989;20:864870.
6. Spilker J, Kongable G, Barch C, Braimah J, Brattina P, Daley S, Donnarumma R, Rapp K, Sailor S, NINDS rt-PA Stroke Study Group. Using the NIH Stroke Scale to assess stroke patients. J Neurosci Nurs. 1997;29:384392.[Medline] [Order article via Infotrieve]
7. Lyden P, Brott T, Tilley B, Welch KMA, Mascha EJ, Levine S, Haley EC, Grotta J, Marler J, NINDS rt-PA Stroke Study Group. Improved reliability of the NIH Stroke Scale using video training. Stroke. 1994;25:22202226.[Abstract]
8. Wityk R, Pessin MS, Kaplan RF, Caplan LR. Serial assessment of acute stroke using the NIH Stroke Scale. Stroke. 1994;25:362365.[Abstract]
9. Fleiss JL. Statistical Methods for Rates and Proportions. New York, NY: John Wiley & Sons Inc; 1981;212236.
10.
Grigsby J, Sanders JH. Telemedicine: where it is and
where it's going. Ann Intern Med. 1998;129:123127.
11.
Levine SR, Gorman M. "Telestroke": the application
of telemedicine for stroke. Stroke. 1999;30:464469.
12.
DeCorato DR, Kagetsu NJ, Ablow RC. Off-hours
interpretation of radiologic images of patients admitted to the
emergency department: efficacy of teleradiology. AJR Am J
Roentgenol. 1995;165:12931296.
13. Taylor P. A survey of research in telemedicine, 2: telemedicine services. J Telemed Telecare. 1998;4:6371.[Medline] [Order article via Infotrieve]
14.
Heinemann AW, Harvey RL, McGuire JR, Ingberman D,
Lovell L, Semik P, Roth EJ. Measurement properties of the NIH Stroke
Scale during acute rehabilitation. Stroke. 1997;28:11741180.
15.
Lai S-M, Duncan P, Keighley J. Prediction of functional
outcome after stroke: comparison of the Orpington Prognostic Scale and
the NIH Stroke Scale. Stroke. 1998;29:18381842.
16.
Clark WM, Warach SJ, Pettigrew LC, Gammans RE,
Sabounjian LA, Citicoline Stroke Study Group. A randomized
dose-response trial of citicoline in acute ischemic stroke
patients. Neurology. 1997;49:671678.
This article has been cited by other articles:
![]() |
A. Pedragosa, J. Alvarez-Sabin, C. A Molina, C. Sanclemente, M C. Martin, F. Alonso, and M. Ribo Impact of a telemedicine system on acute stroke care in a community hospital J Telemed Telecare, July 1, 2009; 15(5): 260 - 263. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. H. Schwamm, R. G. Holloway, P. Amarenco, H. J. Audebert, T. Bakas, N. R. Chumbler, R. Handschu, E. C. Jauch, W. A. Knight IV, S. R. Levine, et al. A Review of the Evidence for the Use of Telemedicine Within Stroke Systems of Care: A Scientific Statement From the American Heart Association/American Stroke Association Stroke, July 1, 2009; 40(7): 2616 - 2634. [Abstract] [Full Text] [PDF] |
||||
![]() |
L. H. Schwamm, H. J. Audebert, P. Amarenco, N. R. Chumbler, M. R. Frankel, M. G. George, P. B. Gorelick, K. B. Horton, M. Kaste, D. T. Lackland, et al. Recommendations for the Implementation of Telemedicine Within Stroke Systems of Care: A Policy Statement From the American Heart Association Stroke, July 1, 2009; 40(7): 2635 - 2660. [Full Text] [PDF] |
||||
![]() |
D.-K. Kim, S. K Yoo, I.-C. Park, M. Choa, K. Y Bae, Y.-D. Kim, and J.-H. Heo A mobile telemedicine system for remote consultation in cases of acute stroke J Telemed Telecare, March 1, 2009; 15(2): 102 - 107. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. M. Demaerschalk, M. L. Miley, T.-E. J. Kiernan, B. J. Bobrow, D. A. Corday, K. E. Wellik, M. I. Aguilar, T. J. Ingall, D. W. Dodick, K. Brazdys, et al. Stroke Telemedicine Mayo Clin. Proc., January 1, 2009; 84(1): 53 - 64. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. L Kane, C. T Bever, M. Ehrmantraut, A. Forte, W. J Culpepper, and M. T Wallin Teleneurology in patients with multiple sclerosis: EDSS ratings derived remotely and from hands-on examination J Telemed Telecare, June 1, 2008; 14(4): 190 - 194. [Abstract] [Full Text] [PDF] |
||||
![]() |
J. E. Acker III, A. M. Pancioli, T. J. Crocco, M. K. Eckstein, E. C. Jauch, H. Larrabee, N. M. Meltzer, W. C. Mergendahl, J. W. Munn, S. M. Prentiss, et al. Implementation Strategies for Emergency Medical Services Within Stroke Systems of Care: A Policy Statement From the American Heart Association/ American Stroke Association Expert Panel on Emergency Medical Services Systems and the Stroke Council Stroke, November 1, 2007; 38(11): 3097 - 3115. [Full Text] [PDF] |
||||
![]() |
S. Schwab, B. Vatankhah, C. Kukla, M. Hauchwitz, U. Bogdahn, A. Furst, H. J. Audebert, M. Horn, and On behalf of the TEMPiS Group Long-term outcome after thrombolysis in telemedical stroke care Neurology, August 28, 2007; 69(9): 898 - 903. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. P. Adams Jr, G. del Zoppo, M. J. Alberts, D. L. Bhatt, L. Brass, A. Furlan, R. L. Grubb, R. T. Higashida, E. C. Jauch, C. Kidwell, et al. Guidelines for the Early Management of Adults With Ischemic Stroke: A Guideline From the American Heart Association/American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Atherosclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: The American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists. Circulation, May 22, 2007; 115(20): e478 - e534. [Abstract] [Full Text] [PDF] |
||||
![]() |
H. P. Adams Jr, G. del Zoppo, M. J. Alberts, D. L. Bhatt, L. Brass, A. Furlan, R. L. Grubb, R. T. Higashida, E. C. Jauch, C. Kidwell, et al. Guidelines for the Early Management of Adults With Ischemic Stroke: A Guideline From the American Heart Association/ American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Atherosclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: The American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists Stroke, May 1, 2007; 38(5): 1655 - 1711. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. L. Sacco, R. Adams, G. Albers, M. J. Alberts, O. Benavente, K. Furie, L. B. Goldstein, P. Gorelick, J. Halperin, R. Harbaugh, et al. Guidelines for Prevention of Stroke in Patients With Ischemic Stroke or Transient Ischemic Attack: A Statement for Healthcare Professionals From the American Heart Association/American Stroke Association Council on Stroke: Co-Sponsored by the Council on Cardiovascular Radiology and Intervention: The American Academy of Neurology affirms the value of this guideline. Circulation, March 14, 2006; 113(10): e409 - e449. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. L. Sacco, R. Adams, G. Albers, M. J. Alberts, O. Benavente, K. Furie, L. B. Goldstein, P. Gorelick, J. Halperin, R. Harbaugh, et al. Guidelines for Prevention of Stroke in Patients With Ischemic Stroke or Transient Ischemic Attack: A Statement for Healthcare Professionals From the American Heart Association/American Stroke Association Council on Stroke: Co-Sponsored by the Council on Cardiovascular Radiology and Intervention: The American Academy of Neurology affirms the value of this guideline. Stroke, February 1, 2006; 37(2): 577 - 617. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Mikulik, A. V. Alexandrov, M. Ribo, Z. Garami, N. A. Porche, E. Fulep, J. C. Grotta, A. W. Wojner-Alexandrov, and J. Y. Choi Telemedicine-Guided Carotid and Transcranial Ultrasound: A Pilot Feasibility Study Stroke, January 1, 2006; 37(1): 229 - 230. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. C. Meyer, P. D. Lyden, L. Al-Khoury, Y. Cheng, R. Raman, R. Fellman, J. Beer, R. Rao, and J. A. Zivin Prospective reliability of the STRokE DOC Wireless/Site Independent Telemedicine System Neurology, March 22, 2005; 64(6): 1058 - 1060. [Abstract] [Full Text] [PDF] |
||||
![]() |
Task Force Members, L. H. Schwamm, A. Pancioli, J. E. Acker III, L. B. Goldstein, R. D. Zorowitz, T. J. Shephard, P. Moyer, M. Gorman, S. C. Johnston, et al. Recommendations for the Establishment of Stroke Systems of Care: Recommendations From the American Stroke Association's Task Force on the Development of Stroke Systems Stroke, March 1, 2005; 36(3): 690 - 703. [Full Text] [PDF] |
||||
![]() |
L. H. Schwamm, A. Pancioli, J. E. Acker III, L. B. Goldstein, R. D. Zorowitz, T. J. Shephard, P. Moyer, M. Gorman, S. C. Johnston, P. W. Duncan, et al. Recommendations for the Establishment of Stroke Systems of Care: Recommendations From the American Stroke Association's Task Force on the Development of Stroke Systems Circulation, March 1, 2005; 111(8): 1078 - 1091. [Full Text] [PDF] |
||||
![]() |
H. J. Audebert, C. Kukla, S. Clarmann von Claranau, J. Kuhn, B. Vatankhah, J. Schenkel, G. W. Ickenstein, R. L. Haberl, M. Horn, and on behalf of the TEMPiS Group Telemedicine for Safe and Extended Use of Thrombolysis in Stroke: The Telemedic Pilot Project for Integrative Stroke Care (TEMPiS) in Bavaria Stroke, February 1, 2005; 36(2): 287 - 291. [Abstract] [Full Text] [PDF] |
||||
![]() |
G. Saposnik, B. Young, B. Silver, S. Di Legge, F. Webster, V. Beletsky, V. Jain, Y. Nilanont, and V. Hachinski Lack of Improvement in Patients With Acute Stroke After Treatment With Thrombolytic Therapy: Predictors and Association With Outcome JAMA, October 20, 2004; 292(15): 1839 - 1844. [Abstract] [Full Text] [PDF] |
||||
![]() |
R. Handschu, R. Littmann, U. Reulbach, C. Gaul, J. G. Heckmann, B. Neundorfer, and M. Scibor Telemedicine in Emergency Evaluation of Acute Stroke: Interrater Agreement in Remote Video Examination With a Novel Multimedia System Stroke, December 1, 2003; 34(12): 2842 - 2846. [Abstract] [Full Text] [PDF] |
||||
![]() |
A. Wiborg and B. Widder Teleneurology to Improve Stroke Care in Rural Areas: The Telemedicine in Stroke in Swabia (TESS) Project Stroke, December 1, 2003; 34(12): 2951 - 2956. [Abstract] [Full Text] [PDF] |
||||
![]() |
O. Crome and M. Bahr Editorial Comment--Remote Evaluation of Acute Ischemic Stroke: A Reliable Tool to Extend Tissue Plasminogen Activator Use to Community and Rural Stroke Patients? Stroke, October 1, 2003; 34 (10): e191 - e192. [Full Text] [PDF] |
||||
![]() |
S. Wang, S. B. Lee, C. Pardue, D. Ramsingh, J. Waller, H. Gross, F. T. Nichols III, D. C. Hess, and R. J. Adams Remote Evaluation of Acute Ischemic Stroke: Reliability of National Institutes of Health Stroke Scale via Telestroke Stroke, October 1, 2003; 34 (10): e188 - e191. [Abstract] [Full Text] [PDF] |
||||
![]() |
M. P. LaMonte, M. N. Bahouth, P. Hu, M. Y. Pathan, K. L. Yarbrough, R. Gunawardane, P. Crarey, and W. Page Telemedicine for Acute Stroke: Triumphs and Pitfalls Stroke, March 1, 2003; 34(3): 725 - 728. [Abstract] [Full Text] [PDF] |
||||
![]() |
B. C. Meyer, T. M. Hemmen, C. M. Jackson, and P. D. Lyden Modified National Institutes of Health Stroke Scale for Use in Stroke Clinical Trials: Prospective Reliability and Validity Stroke, May 1, 2002; 33(5): 1261 - 1266. [Abstract] [Full Text] [PDF] |
||||
![]() |
P. W. Duncan, H. S. Jorgensen, and D. T. Wade Outcome Measures in Acute Stroke Trials : A Systematic Review and Some Recommendations to Improve Practice Stroke, June 1, 2000; 31(6): 1429 - 1438. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 1999 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |