Development and Validation of a Dispatcher Identification Algorithm for Stroke Emergencies
Background and Purpose—Recent innovations such as CT installation in ambulances may lead to earlier start of stroke-specific treatments. However, such technically complex mobile facilities require effective methods of correctly identifying patients before deployment. We aimed to develop and validate a new dispatcher identification algorithm for stroke emergencies.
Methods—Dispatcher identification algorithm for stroke emergencies was informed by systematic qualitative analysis of the content of emergency calls to ambulance dispatchers for patients with stroke or transient ischemic attack (N=117) and other neurological (N=39) and nonneurological (N=51) diseases (Part A). After training of dispatchers, sensitivity and predictive values were determined prospectively in patients admitted to Charité hospitals by using the discharge diagnosis as reference standard (Part B).
Results—Part A: Dysphasic/dysarthric symptoms (33%), unilateral symptoms (22%) and explicitly stated suspicion of stroke (47%) were typically identified in patients with stroke but infrequently in nonstroke cases (all <10%). Convulsive symptoms (41%) were frequent in other neurological diseases but not strokes (3%). Pain (26%) and breathlessness (31%) were often expressed in nonneurological emergencies (6% and 7% in strokes). Part B: Between October 15 and December 16, 2010, 5774 patients were admitted by ambulance with 246 coded with final stroke diagnoses. Sensitivity of dispatcher identification algorithm for stroke emergencies for detecting stroke was 53.3% and positive predictive value was 47.8% for stroke and 59.1% for stroke and transient ischemic attack. Of all 275 patients with stroke dispatcher codes, 215 (78.5%) were confirmed with neurological diagnosis.
Conclusions—Using dispatcher identification algorithm for stroke emergencies, more than half of all patients with stroke admitted by ambulance were correctly identified by dispatchers. Most false-positive stroke codes had other neurological diagnoses.
Effective acute stroke therapies such as intravenous thrombolysis and stroke unit treatment have been established over the last decades.1 However, these treatment options are not available in all hospitals1 and the effect of thrombolysis on functional outcome is time-dependent.2,3
Early recognition of patients with acute stroke in the prehospital setting is helpful in triaging patients and increasing admission rates of patients with stroke to specialized facilities. Prehospital stroke scores have been developed to assist in identifying stroke and show acceptable sensitivity (between 66% and 91%) and positive predictive values (78% and 90%).4–7 These scores were tested with medical professionals in primary care, emergency medical services (EMS), and emergency department physicians.7
Recent developments such as the installation of a CT scanner and point-of-care laboratory in EMS ambulances8,9 promise a complete diagnostic stroke workup required for thrombolytic treatment before transport to the hospital. The possibility of sending such special ambulances to stroke emergencies might enable new prehospital stroke treatments to be evaluated, for example, ambulance-based neuroprotective trials.10
However, efficient use of such expensive and complex mobile facilities in clinical trial protocols and service delivery requires an effective method of identifying patients with stroke during the emergency call before deploying the EMS. No “gold standard” method exists and the benefits of existing algorithms are debated in the literature.11
We therefore aimed to develop and test the feasibility and accuracy of a new stroke identification interview algorithm in the German language at the Berlin Dispatch Center in preparation of the evaluation of a Stroke Emergency Mobile Unit equipped with a mobile CT scanner and all necessary point-of-care laboratory tests.
Materials and Methods
Prehospital Stroke Care in Berlin
Berlin is the largest city in Germany with approximately 3.5 million inhabitants. Prehospital emergency care is centrally organized by the Berlin Fire Brigade, running both the Berlin Dispatch Center and the EMS. The emergency call attendance is managed in 4 shifts (each staffed by 9 dispatchers). In case of escalating emergencies, the dispatch service capacity can be augmented with other trained employees of the Fire Brigade. Approximately 1 million emergency telephone calls are recorded annually in the dispatch center and EMS is activated almost 250 000 times per year.12 All emergency calls are temporarily recorded for evaluation and training purposes. The EMS activation is currently based on a computerized system but during the study period, emergency calls were still performed using unstructured interviews by trained paramedics. This allowed the evaluation of a wide variety of interview styles and in particular of information spontaneously given by callers.
According to routinely gathered statistics of the Berlin Department of Health, 12 811 patients with stroke or transient ischemic attack (TIA) were treated in acute care hospitals in 2009.13 Approximately 2600 of these patients were treated at the 3 campuses of the Charité University hospitals.
The study was conducted in 2 parts: Part A—a retrospective review of emergency calls that used qualitative methods to identify the contents of calls. This would allow us to confirm the face validity of the algorithm and ensure that key items and areas that might assist dispatchers in discriminating between stroke and nonstroke patients were included.
Part B—in the second part, this new algorithm was tested prospectively on patients admitted by EMS to the Charité's hospitals.
Retrospective Analysis of Emergency Calls and Development of the New Interview Algorithm
To identify typical information given in emergency telephone calls of stroke and nonstroke patients, we took a purposive sample of consecutive recorded calls from patients who were subsequently admitted to 1 of the 3 Charité Departments of Neurology and the Accident & Emergency Department of the Charité Campus Benjamin Franklin between December 2009 and September 2010. Only data from patients admitted to Charité hospitals were analyzed in the present evaluation.
We collected consecutive EMS protocols aiming to analyze at least 40 acute patients' emergency calls for each of the following categories: stroke patients treated with or without thrombolytic treatment and other acute neurological diseases and nonneurological diseases.
Emergency calls of cases with collected EMS protocols were transcribed and deidentified but retained the source of the text as “caller” or “dispatcher.” The transcribed communications were then analyzed by 2 independent raters (one medical student [S.K.] and 1 neurologist [M.E.]) blinded to the final diagnosis using a semistructured evaluation form to identify themes or patterns mentioned in dispatch conversations (using semantic analysis). In case of disagreement, another neurologist (H.J.A.) served as a referee. Finally, we performed a quantitative analysis of word frequencies (using only the callers' data) using NVIVO 8 software (QRS International). After excluding redundant words without specific meaning, words were grouped according to the type of information given in emergency calls. Results were compared between stroke-related and nonstroke groups and to the existing literature. We used the quantitative analysis of words to identify words (“discriminatory terms”) that are frequently used in 1 disease category but rarely in others and might therefore be used to distinguish in unclear cases.
Based on these results, we created a new algorithm, which was further refined in discussion with a small group of 5 dispatchers in Berlin. We aimed to optimize the use of information regarding stroke identification but to keep the interview algorithm practical and of short duration. In contrast to the existing computer-based “Advanced Medical Priority Dispatch System” (AMPDS), the stroke code can be activated with only 1 typical stroke symptom mentioned by the caller and without asking questions for other stroke symptoms. Similar to the AMPDS, a mentioned suspicion of stroke is used to trigger the stroke-specific questions.
For the planned use of the algorithm to identify suitable patients for specialized prehospital stroke care, suspected stroke was categorized as “onset within 4 hours of emergency call,” “onset >4 hours,” and “unknown onset.”
Implementation of the New Interview Algorithm
Stroke identification was not part of the established dispatch interview system in Berlin. This system did not use standardized, disease-specific algorithms but was focused on recognition of patients with life-threatening events (eg, cardiac arrest or coma) and trauma.
Dispatchers were trained in the use of the new algorithm. The 1-hour training comprised a presentation of the preceding analysis of dispatcher interviews. Typical and atypical stroke signs were explained to the dispatchers. We also informed the dispatchers about symptoms less typical for stroke but still indicating a possible stroke case when occurring with sudden onset. These symptoms included unilateral sensory loss, unilateral ataxia, coma, nystagmus, rotatory vertigo, and double vision. Dispatchers were also informed how to use the discriminatory terms as “supporting information” in equivocal calls.
The new stroke identification algorithm was prospectively tested.
The Berlin Dispatch Center provided a list of all patients admitted to 1 of the Charité hospitals who had either a (suspected) stroke diagnosis at the dispatcher level or a stroke diagnosis by ambulance staff. Neither the dispatchers nor the administrators of the Dispatch Center data bank were informed about the in-hospital diagnosis of the patients.
Definition of Reference Standard
The admission and hospital discharge diagnoses were recorded as the International Classification of Diseases, 10th Revision code for each patient. The hospital discharge diagnosis was taken as the final diagnosis (reference standard).
Stroke management is centered in the Departments of Neurology at all 3 Charité campuses. Stroke diagnosis is based on cerebral imaging (in all cases) and neurological workup (or at least neurological review in patients treated in other departments). International Classification of Diseases, 10th Revision codes I61.x, I63.x, and I64.x were summarized as acute stroke diagnoses.
All patients with stroke diagnoses at admission or discharge were included in the data set provided by the Medical Records department of the Charité and information regarding age, type of admission (by EMS or others), and thrombolytic treatment was collected. Patients not admitted by EMS or the relatives of these patients were asked whether the dispatch center was contacted using the emergency telephone code. Accuracy, sensitivity for stroke or TIA recognition as well as positive predictive values were calculated in patients admitted by ambulance by comparing matched prehospital and hospital diagnoses.
The diagnostic accuracy of stroke diagnoses was also compared between dispatchers using the new algorithm and paramedics working on ambulances. The “emergency assistants” working in the Berlin EMS had received training on recognition of stroke symptoms as part of their professional education. The Berlin Stroke Units had organized many “acute stroke management” courses for paramedics over the previous years, particularly during the “Berlin Against Stroke” campaign. However, we have no reliable information on how many paramedics in our study participated in these training sessions.
The study was approved by the Ethics Committee of the Charité Universitätsmedizin Berlin. The dataflow was approved by the Berlin Commissioner for Data Protection and the institutional representatives for data protection.
Statistical calculations were performed with SPSS-18 software.
In the retrospective evaluation of 207 consecutive emergency calls, we analyzed 117 stroke calls including 42 patients with ischemic stroke who received intravenous thrombolysis, 75 patients with stroke or TIA treated without thrombolysis (51 ischemic strokes, 17 hemorrhagic strokes, 7 TIAs), and 90 nonstroke calls, 39 patients with nonstroke neurological diseases and 51 patients with nonneurological emergencies.
The results of the semantic analysis are shown in Table 1. Limb weakness and speech problems were the most frequent symptoms in patients with stroke but rarely mentioned in nonstroke patients. Spontaneous mention of stroke was very common in the stroke groups. Typical signs expressed in nonstroke neurological casualties were unconsciousness and seizures, whereas pain and breathlessness were frequent clinical information given in nonneurological casualties.
Falls or atypical movement disorders as well as reported sudden onset of symptoms were frequent in all neurological and nonneurological categories and are therefore deemed to be suitable for screening for stroke symptoms but not for discriminating between stroke and nonstroke patients.
These observations were confirmed in the software based word count analysis (online-only Supplemental Table A; http://stroke.ahajournals.org).
On the basis of these results, we established the new stroke identification interview algorithm allowing dispatchers to assign the stroke code when only 1 typical stroke symptom (with sudden onset) is reported. The Face–Arm–Speech-Test (FAST)7 was introduced in a modified version (Figure 1) to identify patients with stroke on the basis of the initial description of their signs and symptoms.
The new stroke identification algorithm was tested prospectively from October 16 to December 16, 2010. The patient selection is shown in a flow diagram (Figure 2). Of all 38 172 patients with EMS transport, 5774 patients were admitted to Charité hospitals. Stroke was coded as dispatcher diagnosis at the Berlin Dispatch Center in 2009 cases with 284 patients transported to Charité hospitals. Ten of these admissions could not be confirmed in the Charité data bank either because of patient refusal of admission or mismatch of identification data. Of all 274 patients with stroke diagnoses at the dispatcher level, 160 (58.4%) were categorized by the algorithm as: stroke within 4 hours, 48 (17.5%), beyond 4 hours, or 66 (24.1%) of unknown onset.
During the evaluation period, a total of 394 patients received inpatient stroke treatment in the 3 Charité hospitals according to discharge diagnoses (mean age, 71.8 years; SD, 14 years; 53% female). EMS transported 246 patients and 148 were brought to the hospital by private transport. Intracerebral hemorrhage was diagnosed in 56 patients and ischemic stroke in 338 patients. According to the information given by patients or relatives, all patients were admitted by ambulance if the Berlin Dispatch Center was called. The algorithm accurately identified stroke in 131 of the 246 cases. Positive predictive values of dispatcher diagnoses were 47.8% for stroke and 59.1% for stroke or TIA. These and other test parameters are described in Table 2.
The 10 most frequent diagnoses of patients admitted with a dispatcher stroke code are shown in the online-only Supplemental Table B. Two hundred fifteen patients (78.5%) were confirmed with a neurological diagnosis.
Sensitivity was slightly higher for ambulance compared with dispatcher diagnosis with 143 true-positive cases (58.1%). Positive predictive values for diagnosis by ambulance staff (based on normal diagnostic training within the professional education) were 51.7% for stroke and 63.4% for stroke or TIA.
Appropriateness of the Algorithm in Identifying Patients Suitable for Thrombolytic Treatment
Intravenous thrombolysis was applied to 45 (22%) of all 203 patients with ischemic stroke admitted by EMS and in 18 of the 60 patients with ischemic stroke (30%) with a correct dispatcher stroke diagnosis within 4 hours of onset.
The prospectively developed stroke identification interview algorithm allowed correct identification of more than half of all patients with stroke admitted by EMS. Most of the patients with suspected stroke at the dispatcher level had a neurological disease (79%) with 59% stroke or TIA. Thus, selective dispatch of stroke-specific EMS vehicles seems feasible and the concept of evaluating more specific stroke treatment in the prehospital setting is encouraged by these results. However, many patients with stroke were not recognized with the current algorithm demanding both improvement of sensitivity and ongoing provision of conventional prehospital stroke management.
Information given in emergency calls for patients with stroke has been analyzed in several studies.14–18 However, our qualitative study is unique in that it compared calls of stroke and nonstroke patients. Similar to our evaluation, Handschu et al and Porteous et al described limb weakness and speech problems as frequently reported symptoms in patients with stroke.14,15,17 Almost identical rates of callers using the word “stroke” without prompting were determined by Porteous et al and Rosamond et al (51% and 45%, respectively, versus 49% in our sample)14,17 They also mentioned the frequent report of impaired consciousness (or altered mental status) and decreased ability to stand or walk in patients with stroke, but this information was not provided in nonstroke patients.
Diagnostic accuracy of stroke diagnosis at the dispatcher level on the basis of the computer-based AMPDS has been evaluated in a number of more recent studies.16,18–20 Using the AMPDS, dispatchers allocate a specific code for EMS activation. The stroke-specific questions of the original version of the AMPDS were not prospectively developed and results of accuracy tests vary widely in published literature with positive predictive values between 4220 and 49%19 and sensitivity between 3114,17 and 83%.20 The high sensitivity of 83% reported by Ramanujam et al was calculated in a retrospective study of stroke cases entered into a stroke registry with unknown completeness rate. Because recognition rates have been deemed unsatisfactory, the original stroke questions (Card 28 of the Medical Priority Dispatch System) have been replaced by questions/commands of the Cincinnati Prehospital Stroke Scale and accuracy of dispatch stroke diagnosis is currently being determined.21
Our stroke identification algorithm has advantages but also disadvantages compared with the AMPDS. With the preceding analysis of dispatch interviews for stroke and nonstroke casualties, we were able to identify stroke-specific and discriminatory information (such as pain or breathlessness) in a systematic manner. The collaborative approach in designing the interview algorithm helped us take into account the preferences and experiences expressed by the dispatchers. We kept the interview algorithm as “lean” as possible, allowing dispatchers to assign the stroke code when only 1 typical stroke symptom is described. The item “asking the patient to repeat a saying” (as used in the AMPDS) was substituted with asking them to give their address, because this seemed more appropriate in emergency calls.
It is likely that the noncomputer-based interview algorithm is associated with less frequent use of the stroke identification tool, but we were not able to measure the compliance of the dispatchers. The evaluation of a convenience sample of dispatch interviews in false-negative cases revealed that the algorithm was not used in most of these calls. A re-evaluation of the algorithm is therefore planned after implementation of the computer-based system.
Our study has limitations. First, the stroke recognition algorithm was tested directly after termination of a citywide stroke awareness campaign called “Berlin Against Stroke” started in May 2010 and finished in October 2010. Hence, the population of Berlin may have been more alert to stroke symptoms and neurological deficits may have been better described than under previous circumstances. Second, according to the German data protection legislation, we were allowed to validate the accuracy of the stroke identification algorithms only in patients admitted to Charité hospitals. Because these 3 hospitals serve as comprehensive stroke centers, we cannot exclude that the sensitivity results are biased by the prehospital triage rules, which require admission of patients with suspected stroke to stroke units. However, the percentage of patients with a dispatch stroke code admitted to Charité hospitals (of all patients with a dispatch stroke code) was similar compared with the proportion of all patients admitted by EMS to Charité hospitals (14.1% versus 15.1%).
Symptoms typical for posterior circulation strokes such as coma, nystagmus, rotatory vertigo, double vision, or alternating sides of paresis are not included in the new interview algorithm. Although we explained to the dispatchers that these symptoms are also suspicious for stroke, particularly when occurring with sudden onset, recognition of these signs and symptoms is probably poor. Diagnostic accuracy may therefore be worse for posterior than for anterior circulation strokes.
Although there is still potential for further improvement of the stroke recognition rate, the implementation of the new stroke identification algorithm suggests that evaluation and service delivery of more specific prehospital care in patients with neurological deficits is feasible. Thus, the vision of “saving the brain” by starting thrombolytic treatment in the prehospital setting has become more realistic.
Sources of Funding
The study was funded by the Berlin Technology Foundation (including cofunding by the European Union via the EFRE program) within the Stroke Emergency Mobile Unit (STEMO) project, including part-time employment of PK.
H.J.A. reports consultancy honoraria by Lundbeck Pharma and Bayer Vital as well as speaker honoraria by Takeda Pharma, Boehringer Ingelheim, Lundbeck, Bayer Vital, UCB Pharma, and Sanofi-Syntelabo. J.S. reports speaker honoraria by Takeda Pharma, Boehringer Ingelheim, Pfizer, UCB Pharma, and Sanofi-Syntelabo. A.M.B. reports a board membership and consultancy at International Academy for Emergency Dispatch. In addition, M.E., M.R., J.S., U.M., P.U.H., and H.J.A. are employed in the Center for Stroke Research Berlin (CSB) funded by the German Federal Ministry for Education and Research (BMBF). P.U.H. received speaker honoraria by the German Stroke Foundation. I.W. received support from the Herman Oppenheim Scholarship at the Charité Universitätsmedizin Berlin and the UK National Institute for Health Research (NIHR) Comprehensive Biomedical Research Centre award to Guy's & St Thomas' NHS Foundation Trust in partnership with King's College London and King's College Hospital NHS Foundation Trust, London, UK.
We thank all involved dispatchers of the Berlin Dispatch Center for their crucial contributions to this study.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.111.634980/-/DC1.
- Received August 3, 2011.
- Revision received November 2, 2011.
- Accepted November 7, 2011.
- © 2012 American Heart Association, Inc.
- Leys D,
- Ringelstein EB,
- Kaste M,
- Hacke W
- Saver JL
- Harbison J,
- Hossain O,
- Jenkinson D,
- Davis J,
- Louw SJ,
- Ford GA
- Fassbender K,
- Walter S,
- Liu Y,
- Muehlhauser F,
- Ragoschke A,
- Kuehl S,
- et al
- Saver JL,
- Kidwell C,
- Eckstein M,
- Starkman S
- Clawson JJ,
- Olola CH,
- Scott G
- Brigade BF
Berlin SfUuG. Case numbers of selected hospital admission in Berlin (year 2009). 2010 [cited March 14, 2011]. Available at: www.berlin.de/imperia/md/content/sen-gesundheit/krankenhauswesen/dokumente/transparenzdaten2009_neu_internet.pdf?start&ts=1297325610&file=transparenzdaten2009_neu_internet.pdf.
- Handschu R,
- Poppe R,
- Rauss J,
- Neundorfer B,
- Erbguth F
- Buck BH,
- Starkman S,
- Eckstein M,
- Kidwell CS,
- Haines J,
- Huang R,
- et al
- Deakin CD,
- Alasaad M,
- King P,
- Thompson F