Prehospital Unassisted Assessment of Stroke Severity Using Telemedicine
A Feasibility Study
Background and Purpose—We evaluated the feasibility and the reliability of remote stroke severity quantification in the prehospital setting using the Unassisted TeleStroke Scale (UTSS) via a telestroke ambulance system and a fourth-generation mobile network.
Methods—The technical feasibility and the reliability of the UTSS were studied in healthy volunteers mimicking 41 stroke syndromes during ambulance transportation.
Results—Except for 1 issue, high-quality telestroke assessment was feasible in all scenarios. The mean examination time for the UTSS was 3.1 minutes (SD, 0.4). The UTSS showed excellent intrarater and interrater variability (ρ=0.98 and 0.97; P<0.001), as well as excellent internal consistency and rater agreement. Adequate concurrent validity can be derived from the strong correlation between the UTSS and the National Institutes of Health Stroke Scale (ρ=0.90; P<0.001).
Conclusions—Remote assessment of stroke severity in fast-moving ambulances using a system dedicated to prehospital telemedicine, 4G technology, and the UTSS is feasible and reliable.
Ambulance-based teleconsultation is feasible,1 but the experience with prehospital telemedicine for stroke (telestroke) is limited. Standardized evaluation of stroke severity is pivotal for adequate treatment decision making, but remote quantification of stroke severity remains challenging because current stroke scales require trained professional support at the patient’s bedside.2–4 We recently developed a scale for assessing stroke severity through telemedicine without assistance from a third party (Unassisted TeleStroke Scale [UTSS]).5 This study aims to evaluate the feasibility and the reliability of remote stroke severity quantification in the prehospital setting using the UTSS via a telestroke ambulance system and a fourth-generation mobile network.
Feasibility of Prehospital Telestroke Application
We equipped 2 routine ambulances with a prototype for real-time bidirectional interactive audiovideoconferencing over the Internet. The system architecture consisted of commercially available hardware and a Web-based telemedicine platform. The data were transmitted to a multimedia server unit over a mobile ultrabroadband connection with theoretical transfer speed up to 150 megabits per second (mbps; Figure). Data privacy was secured by password-protected login, role-based access control, and hypertext transfer protocol secure encryption. The bandwidth of the data connection and the total amount of data transfer were measured during the telestroke sessions. Any issue with hardware, software, or connectivity was recorded.
Prehospital Assessment of the UTSS
Two volunteers (both females, aged 27 and 53 years) were transported in an ambulance while being secured to a stretcher according to rules for patient transportation. They randomly simulated 41 stroke syndromes, based on the admission National Institutes of Health Stroke Scale (NIHSS) scores from the first consecutive 41 patients with suspicion of acute stroke admitted to the Stroke Unit of the Universitair Ziekenhuis Brussel starting from January 1, 2012 (Table).
Two examiners were trained in the use of the UTSS and were randomly assigned to separately assess stroke severity via the telestroke system. The UTSS is a patient-friendly scale that has shown to be reliable and valid for assessment of stroke severity through telemedicine, without the need for bedside assistance by a trained healthcare professional.5 Application of a multilingual software tool allowed standardized telestroke assessments, which were randomized to be performed in Dutch (n=12), French (n=15), or English (n=14). To mimic real-life transportations, we tested the potential impeding effects of activating the ambulance sirens and the application of an oxygen mask to the actor’s face, each in 5 scenarios. All examinations were performed live, and instant video screen recording was performed at the examiner’s side, allowing delayed reassessment by both examiners for evaluation of intrarater and interrater variability. This study was approved by the Medical Ethics Committee of the Universitair Ziekenhuis Brussel.
Intrarater and interrater variability of the UTSS were tested by Spearman ρ correlation, and its internal consistency was assessed by Cronbach α.6 The agreement among raters was summarized using the intraclass correlation coefficient and weighted κ statistics.6 Concurrent validity of the UTSS was tested by Spearman ρ correlation with the NIHSS and with linear regression analysis.6 Statistical computations were performed with the SPSS software package version 17.0 (SPSS Inc, Chicago, IL) and SAS version 9 (SAS Institute Inc, Cary, NC).
In 5 scenarios (12.2%), we experienced a short freezing of the video (<5 seconds), and the audio transmission was suboptimal in 4 cases (9.8%; eg, slight voice echo or system-induced noise). In 2 cases (4.9%), there was a transient light overexposure attributable to very bright sunlight. These events had no effect on the examination duration, nor did they relevantly influence the examination process or quality. There was a signal loss during 1 session attributable to disconnection of the 4G USB dongle caused by vibrations in the ambulance. Except for this instance, no telestroke session was stopped or interrupted for technical reasons.
The bandwidth was sufficient and stable during all telemedicine sessions. The mean download speed per session was 11.0 mbps (SD, 0.8), and the maximal download speed yielded 20.9 mbps. The mean bidirectional data transfer per telestroke session was 696 megabit (SD, 61.6).
Prehospital Assessment of the UTSS
A screen capture from the examiner’s computer during prehospital telestroke UTSS assessment is shown in Movie I in the online-only Data Supplement. The mean examination time for the UTSS was 3.1 minutes (SD, 0.4), and the median score was 3 (interquartile range, 1–10). Spearman ρ values for intrarater and interrater variability were 0.98 and 0.97, respectively (P<0.001 for both). The UTSS showed strong internal consistency (Cronbach α, 0.91; 95% confidence interval, 0.86–0.94) and high agreement among raters: the intraclass correlation coefficient for single measures was 0.98 (95% confidence interval, 0.96–0.99; P<0.001), and the weighted κ statistic yielded 0.89 (95% confidence interval, 0.84–0.94; P<0.001). Calculation of κ statistics for the individual items of the UTSS showed excellent interobserver agreement (κ statistic, 0.75–1.0) for 12 items (75%) and moderate agreement (κ statistic, 0.4–0.75) for 3 items (19%). The item Spatial attention and left/right orientation had poor κ statistics (0.01; P=0.71). Application of ambulance sirens or oxygen mask did not relevantly influence the telestroke experience or assessment of stroke severity. The UTSS score correlated strongly with the NIHSS score (Spearman ρ, 0.90; P<0.001). Based on linear regression analysis, the NIHSS score can be predicted from the UTSS score as follows: 2.054+1.53×UTSS score (P<0.001).
Feasibility studies on prehospital quantification of stroke severity through telemedicine are limited to 2 projects using the NIHSS and relying on mobile technology with limited bandwidth and unstable connectivity.2–4 This completely precluded real-time conferencing in TeleBat2,3 and yielded frequent inadequate assessments in a recent study using 3G technology.4 The need for trained professional support at the patient’s bedside for assessment of the NIHSS is a major drawback in the prehospital setting, which can be overcome by the UTSS. Prehospital scoring of the UTSS through a dedicated multilingual software tool showed to be reliable and fast. Only the item Spatial attention and left/right orientation had poor κ statistics, which may be attributable to the difficulty with mimicking this symptom and the low number of scenarios (n=4) containing this feature.
This study is the first to use mobile ultrabroadband Internet access for ambulance-based telestroke, and our study population is the largest reported so far in validating stroke severity assessment through telestroke in the prehospital setting. The use of experienced stroke clinicians mimicking scenarios based on patients with real stroke is another strength of this study because it guarantees adequate representation of the heterogeneous stroke population encountered in clinical practice. Involvement of patients at this stage was not possible for safety reasons, but the encouraging results from this feasibility study form the basis for a prospective prehospital stroke trial in the near future. It should be taken into account that the 4G technology was used in an ideal situation. Technical issues concerning stability and availability cannot be excluded in future real-life application, which will necessitate passing through different cell tower ranges and competition with other users. The latter concern, however, may be solved by medical prioritization.
In conclusion, remote assessment of stroke severity in fast moving ambulances using a system dedicated to prehospital telemedicine, 4G technology, and the UTSS is feasible and reliable.
We thank Door Lauwaert for his organizational support and Belgacom for the highly valued collaboration.
Sources of Funding
This research was supported by the Brussels Institute for Research and Innovation, the King Baudouin Foundation, and the Scientific Fund Willy Gepts.
Drs De Smedt and Cambron are research assistants of the Fund for Scientific research Flanders (FWO). Dr Moens is a clinical investigator of The Research Foundation Flanders (FWO).
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.113.002079/-/DC1.
- Received May 31, 2013.
- Accepted July 2, 2013.
- © 2013 American Heart Association, Inc.
- Liman TG,
- Winter B,
- Waldschmidt C,
- Zerbe N,
- Hufnagl P,
- Audebert HJ,
- et al
- Van Hooff RJ,
- De Smedt A,
- De Raedt S,
- Moens M,
- Marien P,
- Pacquier P,
- et al
- Lyden PD,
- Lau GT