Diagnostic Accuracy of Stroke Referrals From Primary Care, Emergency Room Physicians, and Ambulance Staff Using the Face Arm Speech Test
Background and Purpose— Timely referral of appropriate patients to acute stroke units is necessary for effective provision of skilled care. We compared the characteristics of referrals with suspected stroke to an academic acute stroke unit via 3 primary referral routes: ambulance paramedics using a rapid ambulance protocol and stroke recognition instrument, the Face Arm Speech Test; primary care doctors (PCDs); and emergency room (ER) referrals.
Methods— Patient characteristics, final diagnosis, and admission delay were recorded in all suspected acute stroke referrals in a 6-month period.
Results— Four hundred eighty-seven patients (356 strokes/transient ischemic attacks) were admitted by the 3 routes: 178 by ambulance, 216 by PCDs, and 93 through the ER. The proportion of nonstrokes admitted by each route was similar (23%, 29%, and 29%, respectively). Ambulance paramedics’ stroke diagnosis was correct in 144 of 183 (79%) stroke patients who initially presented to them. Thirty-nine of 66 strokes/transient ischemic attacks referred via ER were taken there following initial ambulance assessment. Compared with PCDs, paramedics referred more total anterior circulation (39% versus 14%, P<0.0001) and fewer lacunar strokes (14% versus 31%, P<0.001) and admitted more patients (46% versus 12%, P<0.01) within 3 hours of symptom onset. The most common nonstroke conditions were seizures, infections and confusion, cardiovascular collapse, and cerebral tumors. Paramedics admitted more patients with seizures.
Conclusions— Misdiagnosis of stroke is common in the ER and by PCDs. Paramedics using the Face Arm Speech Test achieved high levels of detection and diagnostic accuracy of stroke.
Reducing delays from onset of stroke symptoms to admission is necessary if access to organized stroke care and initiation of thrombolysis and other treatments are to be expedited. Accurate recognition of stroke by ambulance or emergency medical (EMS) services, emergency room (ER) departments, and primary care doctors (PCDs) is necessary to ensure rapid transfer of patients to acute stroke units (ACUs). Few current data exist on the accuracy of the diagnosis of stroke by these professional groups. Prehospital ambulance paramedics are in a unique position to reduce delays in presentation and treatment in acute stroke. We have previously shown high diagnostic accuracy of stroke by ambulance staff using a rapid ambulance protocol.1 In the present study we compared the characteristics and accuracy of diagnosis of stroke by ambulance staff using a rapid ambulance protocol incorporating the Newcastle Face Arm Speech Test (FAST) assessment, PCDs, and ER doctors.
Subjects and Methods
Acute Stroke Service and Referral Routes
The Freeman Hospital Stroke Services admit >90% of strokes referred to secondary care in the city of Newcastle on Tyne (population ≈300 000). As Freeman Hospital has no ER on site, patients admitted with suspected strokes can arrive by 3 routes:
(1) Ambulances provide direct transfer from the community, on the basis of a diagnosis by the paramedics who use a rapid ambulance protocol, which incorporates a stroke identification instrument, the Face Arm Speech Test (FAST) (Figure 1)
(2) Primary care doctors refer patients to the ASU by telephone through a centralized admissions office (Hospital Direct).
(3) Emergency room personnel handle referral of patients who present themselves to the ER, or are brought there by ambulance, where stroke has not been diagnosed by ambulance staff or where a diagnosis of stroke has been made but the Glasgow Coma Scale (GCS) score is ≤7 or head injury is suspected. Following assessment and resuscitation, stroke patients are transferred to the ASU.
All patients referred to the ASU by these routes have had a provisional diagnosis of stroke made by a doctor or ambulance crew member.
Rapid Ambulance Protocol
The rapid ambulance protocol was established in 1997 as a mechanism to expedite the rapid transfer of patients to the ASU, ie, to bypass the ER sited at a separate hospital in Newcastle. This protocol applies when patients contact the ambulance service directly (ie, they bypass their primary care doctor by phoning “999”). Ambulance staff who assess a patient as having a suspected acute stroke undertake the FAST assessment and, if this is positive, contact personnel at the ambulance control center, who notify the Freeman Hospital admissions unit of the impending admission of a suspected stroke patient. A stroke nurse is then contacted to undertake the initial patient assessment and liaise with medical staff. Experience with the protocol following the first year of implementation has previously been reported.1
Development of FAST
Other studies have demonstrated variable diagnostic accuracy of stroke by paramedics and emergency medical technicians with positive predictive values between 64% and 77%.2–4 In 1 prospective study, a 4-hour educational stroke seminar was associated with an increase in sensitivity in stroke identification by paramedics from 61% to 91% (P=0.01).5 These observations underpinned the development and implementation of a stroke recognition instrument.
The FAST was developed in 1998 as a stroke identification instrument by a group of stroke physicians, ambulance personnel, and an emergency room physician and was designed to be an integral part of a training package for UK ambulance staff. The FAST was derived following a review of existing North American stroke identification instruments, created to expedite administration of intravenous tissue plasminogen activator to patients within 3 hours of acute stroke symptom onset. The instruments at this time with most evidence of validity were the Cincinnati Prehospital Stroke Scale (CPSS)6 and the Los Angeles Prehospital Stroke Screen (LAPSS).7
The CPSS was derived from a simplification of the 15-item National Institutes of Health Stroke Scale (NIHSS)8 and evaluates the presence or absence of facial palsy, asymmetric arm weakness, and speech abnormalities in potential stroke patients. Speech is tested by asking the patient to repeat the sentence, “The sky is blue in Cincinnati,” and abnormality is reported if the patient slurs words, says the wrong words, or is unable to speak. The CPSS was evaluated in 171 patients selected from an emergency department and inpatient neurology service who had a final diagnosis of stroke, or a “stroke-mimicking” or other neurological condition. The CPSS was performed by physicians certified in the use of the NIHSS and watched and scored by prehospital personnel who had received 10 minutes of education on the use of the scale. There was good correlation between the physicians and prehospital staff for total score (intraclass correlation coefficient, rI,=0.92; 95% CI, 0.89 to 0.93). The sensitivity was 88% for identification of patients with anterior circulation strokes.
The LAPSS is a longer instrument consisting of 4 history items, a blood glucose measurement, and 3 examination items designed to detect unilateral motor weakness (facial droop, hand grip, and arm strength). The LAPSS was prospectively studied in 1298 unselected patients transported by 19 paramedic crews to UCLA Medical Center. The crews underwent a 60-minute training session, and LAPSS forms were completed in 206 patients. Sensitivity was 91% (95% CI, 76% to 98%), specificity 97% (95% CI, 93% to 99%), positive predictive value 86% (95% CI, 70% to 95%), and negative predictive value 97% (95% CI, 84% to 99%).
In the FAST development process, emphasis was placed on producing a simple test that would complement existing assessments used by UK paramedics, such as the Glasgow Coma Scale. The FAST contains 3 key elements (facial weakness, arm weakness, and speech disturbance) from the CPSS (Figure 1) but avoids the need to repeat a sentence as a measure of speech, instead using assessment of language ability by the paramedic during normal conversation with the patient. Similar to the CPSS and LAPSS, the FAST was designed for assessment of a seated subject, and we therefore omitted assessment of leg weakness. None of these instruments detects visual field defects or disorders of perception, balance, and coordination, thus they are all relatively insensitive to lesions in the posterior cerebral circulation. It was felt, however, that increasing the complexity of the FAST would lengthen paramedic assessment time and could increase the proportion of false-positive diagnoses without greatly increasing the sensitivity of the instrument. The 3 assessments contained within the FAST were incorporated into the standard ambulance report form used for recording all ambulance contacts. A training package for ambulance personnel was produced, comprising lecture notes, slide presentation, handout, and multiple choice questionnaire, and was subsequently received by most Newcastle ambulance staff in 1998-1999 and in the training of newly recruited staff.
All patients referred to the Freeman Hospital Stroke Service were prospectively studied over a 6-month period (February 1 to July 31, 2000). Medical notes including results of CT brain imaging, ambulance admission sheets, and emergency room admission records were examined. Stroke was defined according to WHO criteria of “a focal or global neurological deficit with symptoms lasting for 24 hours or resulting in death before 24 hours which was considered to be due to a vascular cause after investigation.” Transient ischemic attacks (TIAs) were defined as clinical syndromes characterized by an acute loss of focal cerebral or monocular function with symptoms lasting less than 24 hours and thought to be caused by inadequate blood supply as a result of thrombosis or embolism. Cases of subarachnoid hemorrhage were considered as nonstroke cases for the purpose of this study. Patients’ age, gender, mode of referral, Oxford Community Stroke Project (OCSP) Classification,9 and time of admission were recorded. Details of time of onset were also recorded where available. The medical notes of patients with a nonstroke diagnosis were independently reviewed by 2 assessors, who were stroke physicians in training (J.H., O.H.). Where a difference existed between the researchers’ diagnosis, a third evaluator (G.A.F.) adjudicated on the diagnosis, in discussion with the 2 assessors. For stroke patients who were taken to the ER by ambulance (contrary to protocol), the ambulance record and FAST scales were examined.
Data were recorded on a secure computerized database program on a personal computer (Access 97, Microsoft Inc) within the terms of the United Kingdom Data Protection Act. Mean ages were compared using unpaired t tests and 1-way ANOVA. Between-group proportions were compared using chi-square analyses. Statistical analyses were performed using a proprietary computerized statistics package (SPSS 9.0).
The study group comprised 487 subjects, representing 92% of total referrals to the ASU, presented via the 3 routes during the study (Table 1, Figure 2). The remaining 8% of referrals were inpatient strokes, interhospital transfers, or patients who self-presented to the stroke service. Fifty-two percent of referrals were female. There was a nonsignificant trend for patients referred via the ER to be younger. Four hundred eighty-seven patients with confirmed stroke all had CT brain imaging except in 11 cases (5 died within 24 hours of admission, 2 with minor stroke were discharged within 24 hours, 4 were excluded for other reasons). Twenty-nine of 62 TIA patients underwent CT brain imaging during admission; 71 of 131 nonstroke patients underwent CT brain imaging.
The most common route of referral was by PCDs (44%), followed by ambulance staff (37%) and ER doctors (19%). Forty-six (70%) of the 66 patients referred by ER doctors had been taken to the ER by ambulance staff, contrary to protocol except in 7 patients with a GCS score of <7 or evidence of head injury. The positive predictive value (PPV) and 95% CIs for suspected stroke patients brought to the ASU by ambulance staff were 78% (72% to 84%; 137+7/178+7), the denominator being the 178 suspected strokes brought to the hospital plus the 7 stroke patients with head injury or low GCS scores appropriately not diverted and taken to the ER, assuming all patients with stroke/TIA with GCS score <7 or evidence of head trauma taken to the ER were referred to the ASU. The PPVs and 95% CIs for ER and PCDs were both 71% (66/93, 95% CI 64% to 78% and 153/216, 95% CI 65 to 77%, respectively) (Figure 2). It is not possible to calculate an accurate diagnostic sensitivity in any referring group because non-referrals to the ASU were not reviewed. However, a stroke/TIA detection rate (diagnostic sensitivity) can be estimated for ambulance staff, if it is assumed all stroke/TIAs who were taken by ambulance to ER were referred to the ASU. This suggests an upper estimate of diagnostic sensitivity of 79% (137+7/183), the denominator being the number of confirmed stroke/TIA patients admitted who had initial contact with ambulance staff.
Ambulance staff referred more total anterior circulation infarcts than did PCDs or ER doctors (P<0.01). A higher proportion of lacunar strokes were admitted by PCDs (P<0.05) than by the ER or ambulance protocol. Ambulance staff recognized 3 of 7 posterior circulation strokes referred to them (43%). However, these 7 patients represent only 24% of the posterior circulation strokes referred to the stroke service in the course of the study. A higher proportion of stroke referrals were admitted by ambulance staff within 1, 2, and 3 hours of stroke onset compared with PCDs and ER doctors (P<0.001) (Table 2).
Of the 66 patients with stroke/TIA referred via the ER, 20 were self-presentations and 7 were via the rapid ambulance protocol but had a low GCS score or a head injury. Of the remaining 39 patients, ambulance admission sheets were available for review in the notes for 20. Only 5 of these had the FAST completed. On reviewing the notes of patients with uncompleted and unavailable FAST scores, it was found that on presenting to the ER, 9 of the 15 with no completed scores and 9 of the 19 with unavailable scores had neurological deficits documented by the admitting ER doctor, which would likely have allowed a diagnosis of stroke using the FAST.
One hundred thirty-one patients without confirmed acute stroke or TIA were referred to the stroke service during the course of the study. There was no significant difference among the proportion of nonstroke patients referred by ambulance paramedics, ER physicians, and general practitioners (Table 3). The most common nonstroke diagnoses admitted by each of the 3 groups were seizures by the rapid ambulance protocol, infection or delirium by general practitioners, and seizures or delirium by the ER. Fifty two percent of misdiagnosed patients presented with other “neurological” diagnoses (eg, peripheral neuropathy, migraine). The remainder presented with primarily “non-neurological” were conditions usually managed by internal physicians or geriatricians. Eighteen patients (14%) who were misdiagnosed as having acute strokes had residual neurological deficits from previous strokes.
Ambulance technicians and paramedics using the FAST achieved high levels of PPV in redirecting patients to the acute stroke unit. Although ambulance paramedic diagnosis of stroke was as accurate as that of primary care or ER doctors, the strokes they admitted tended to be more severe and likely were easier to diagnose. As demonstrated in other studies, ambulance personnel admitted a significantly greater proportion of stroke patients within 3 hours compared with general practitioner referrals. This pattern is clearly in the best interests of patients who may be eligible for thrombolytic therapy.
Although all patients with a false-positive diagnosis of stroke in Newcastle are referred to the Freeman Stroke Service, there is no means of accurately estimating the number of false-negative diagnoses, so we are unable to accurately determine the sensitivity and specificity of the accuracy of diagnosis of stroke by the 3 groups. Patients with undiagnosed strokes may be referred to another hospital or managed at home, as occurs in an estimated 20% patients in the United Kingdom. Our data suggest a good level of diagnostic sensitivity by ambulance staff (>75%) if it is assumed that all stroke/TIA cases not redirected to the ASU were subsequently referred to the ASU by the ER team. Although TIA cases may have been discharged home from the ER, it is unlikely that significant numbers of patients with stroke were discharged home or admitted to the other acute medical hospital in Newcastle (<10% acute strokes in Newcastle). Allowing for non-referral of cases by ER would still suggest a diagnostic accuracy of at least 60% for acute stroke. The average age of our subjects, 72 years, is similar to that found in community-based epidemiology stroke studies,9–11 but the proportion of posterior circulation strokes is less than found in the Oxfordshire Community Stroke Project (10% versus 24%).12 This suggests patients with posterior circulation infarcts may be less likely to be referred to the acute stroke service. We suspect some patients with posterior circulation strokes are misdiagnosed with vestibular disorders or labyrinthitis in the community and not referred immediately to our service.
Few studies have examined diagnostic accuracy by non-neurologists within the first few hours of presentation, ie, clinical evaluation of a “brain attack.” Norris and Hachinski reported a 13% incorrect diagnosis in 821 consecutive patients admitted over a 3-year period to an acute stroke unit.13 However, the diagnosis of stroke had been “confirmed” by a neurology resident prior to admission to the stroke unit, and the accuracy of initial assessment by ER physicians was not reported. Libman et al reported a misdiagnosis rate of 19% in 411 patients initially diagnosed to have stroke in the ER14 with postictal states, systemic infections, tumors, and toxic-metabolic disturbances accounting for 60% stroke mimics. Our findings are similar except for more cases of syncope and functional causes and fewer toxic-metabolic disturbances. The Freeman Hospital Stroke Service differs from many UK centers in that patients referred for assessment have all had a diagnosis of stroke made by another clinician or ambulance paramedic prior to referral. This provided an opportunity to compare the rate of misdiagnosis of referrals by various routes.
Only one quarter of stroke patients with available ambulance sheets referred to the ER against protocol had the FAST completed. This indicates that in most of these cases a diagnosis of stroke had not been considered by ambulance staff, suggesting opportunities to improve stroke recognition skills by further training or incorporation of additional assessments into the FAST. In 2 of the 5 patients with stroke who had the FAST completed but were taken to the ER, an alternative diagnosis had been suggested by nursing home staff (epilepsy, urinary tract infection) that may have precipitated the misreferral.
The nonstroke referrals to the unit generally reflect published differential diagnoses of stroke. It is well recognized that mass lesions may mimic stroke. Like Norris and Hachinski’s series from the 1980s, seizures (39% compared with 21% in our series) and acute confusional states were the most common nonstroke diagnoses.13 Similar to our experience, one third of patients with seizure had residual neurological deficits from a previous stroke, and many patients with acute confusional states had preexisting dementia. Fifteen percent of nonstroke referrals were patients with cardiovascular collapse referred as suspected TIAs. The misdiagnosis of TIA in patients with syncope was noted in the Oxfordshire Community Stroke Project.12 Another common reason for misreferral (8%) was a group of chronic progressive neurodegenerative disorders, usually dementia or Parkinson’s disease, referred as stroke. The circumstances in which these patients were often admitted were when carers were no longer able to cope and a doctor unfamiliar with the patient’s previous condition referred the patient with an apparent acute deterioration. The characteristics of misdiagnosed cases differed among the 3 referring groups, which have implications in service planning and development of appropriate stroke recognition instruments and skills. Ambulances admitted more patients with seizures and general practitioners referred a greater variety of diagnoses with fewer “acute” neurological disorders. The number of patients with seizures brought by ambulance staff to the ASU was low, given that ambulance staff are more frequently called to patients with seizure than to stroke, and postictal neurological deficits are often present. This suggests that ambulance staff may be using a history of seizures as part of their initial screening for stroke. We suspect that, as with any medical assessment, the “context” of the patient is key in deciding whether to formally evaluate a patient with the FAST. Our study has assessed the diagnostic accuracy of ambulance staff using the FAST when they suspect a patient may have stroke, and as such has not examined the diagnostic utility of the FAST instrument against ambulance staff assessment without use of the instrument. Because other ambulance staff might make different judgments of the context in which to use the FAST, it cannot be assumed that similar diagnostic sensitivity and specificity would be obtained by use of the instrument in a different health care setting. It would have been impractical for ambulance staff in our setting to undertake the FAST assessment in all patient contacts. Further studies are required to understand how ambulance paramedics use the FAST and other stroke diagnostic instruments and the additional diagnostic benefit provided by these instruments.
Half of the nonstroke referrals had conditions, such as sepsis or cardiovascular collapse, that would usually be managed by specialists other than neurologists in the United Kingdom. This emphasizes the importance of having both neurological and general internal medical input and expertise when managing patients with suspected stroke. Although diffusion-weighted MRI can play a valuable role in the initial diagnosis of stroke, in the majority of cases admitted to our unit, CT and MRI were not significant factors in making a diagnosis of stroke or nonstroke.15 Despite advances in imaging, we consider Allen’s statement from 1984 following the widespread introduction of CT imaging in stroke that “despite the substantial advances in neuroimaging which have been made over the last decade, the initial clinical assessment of a patient with acute stroke remains important” still applies today.16
Our study is unique in comparing stroke referrals by ambulance paramedics using a rapid assessment scale against referrals by other routes. As in previous studies, we found that patients referred directly by ambulance staff tended to have more severe strokes, particularly anterior circulation strokes,17,18 which may account for the trend toward a higher diagnostic accuracy than found with general practitioners and ER doctors. Anterior circulation cerebral infarcts may gain more benefit from thrombolysis and future neuroprotective therapies. As such, this is the most important group to obtain rapid access to an ASU. However, development of paramedic identification instruments to improve the identification of posterior circulation symptoms such as double vision, ataxia, and visual disturbance should be considered.
Our study demonstrates continuing difficulties in the initial diagnosis of stroke by ER and primary care doctors, ie, the medical staff who first assess most patients with suspected stroke. The development of clinical stroke recognition instruments could help in improving initial diagnostic accuracy. If thrombolysis and future acute interventions are to be effectively delivered to patients with acute stroke, early accurate recognition of acute stroke by these groups is necessary to ensure appropriate referrals to acute stroke teams.
The diagnostic accuracy for ambulance staff found in our study is comparable to but slightly less than that reported by Los Angeles paramedics using the LAPSS, where 34 of 36 “target” stroke patients were recognized by ambulance staff.7 Whether this difference is due to differences between the study populations in ambulance paramedic skills, instrument utility, or a lower prevalence of true acute stroke/TIA cases in the population presenting to both paramedic groups, a factor known to impact on the calculation of PPV, is unclear and requires further study. In fact, if at all possible, sensitivity would be a better indicator of the performance of an assessment in this situation, given that sensitivity addresses the more pertinent question, “Of all cases of acute stroke, what proportion would be correctly identified by this instrument?”
Many studies have shown that direct referral to ambulance services is associated with more rapid admission to secondary care. The development of rapid transfer protocols and diagnostic tools for ambulance staff may facilitate reducing delays that occur once patients arrive at the ER. We have found that the diagnostic accuracy of ambulance staff using a stroke recognition instrument is comparable to that of emergency room and family physicians, albeit in a stroke population with different characteristics. Our findings suggest that with appropriate training, ambulance paramedics can correctly identify a high proportion of patients with stroke who directly present to paramedical emergency services and facilitate rapid referral to organized acute stroke care.
Dr Joseph Harbison was supported by a National Health Service Research Northern & Yorkshire Research and Development Clinical Fellowship. We thank Janssen-Cilag, UK for financial support in developing the FAST assessment and paramedic training package. We thank Mr Bas Sen and colleagues in the Accident and Emergency department, Newcastle General Hospital, for assistance with data collection. Membership of the FAST development group comprised Prof Gary Ford (Freeman Hospital Stroke Service); Dr Damian Jenkinson (Stroke Service, Royal Bournemouth and Christchurch Hospitals); Dr Ed Glucksman (Accident & Emergency Department, Kings College Hospital, London); Dr Tom Quinn (West Midlands Thrombolysis Project); David Hodge (Northumbria Ambulance Service); Peter Cuthbertson (Northumbria Ambulance Service); Lee Barnett (Northumbria Ambulance Service); John Glasspool and Catherine Owen (Janssen-Cilag, UK); and Mark O’Connor and Bernie Rochford (Caldwell Gardiner Communications).
- Received April 9, 2002.
- Revision received June 20, 2002.
- Accepted July 29, 2002.
Kothari R, Barsan W, Brott T, Broderick J, Ashbrock S. Frequency and accuracy of prehospital diagnosis of acute stroke. Stroke. 1995; 26: 937–941.
Kidwell CS, Starkman S, Eckstein M, Weems K, Saver J. Identifying stroke in the field: prospective validation of the Los Angeles Prehospital Stroke Screen (LAPSS). Stroke. 2000; 31: 71–76.
Brott T, Adams HP, Olinger CP. Measurements of acute cerebral infarction: a clinical examination scale. Stroke. 1989; 20: 864–870.
Carolei A, Marini C, Di Napoli M, Di Gianfilippo G, Santalucia P, Baldassarre M, De Matteis G, di Orio F. High stroke incidence in the prospective community-based L’Aquila registry 1994–1998: first year’s results. Stroke. 1997; 28: 2500–2506.
Petty GW, Brown RD Jr, Whisnant JP, Sicks JD, O’Fallon WM, Wiebers DO. Ischemic stroke subtypes: a population-based study of incidence and risk factors. Stroke. 1999; 30: 2513–2516.
Warlow CP, Dennis MS, Van Gijn J, Hankey GJ, Sandercock PAG, Bamford JM, Wardlaw J. Stroke: A Practical Guide to Management. 1st edition. Oxford, UK. Blackwell Science Ltd; 1996: 55.
Allen CMC. Differential diagnosis of acute stroke: a review. J R Soc Med. 1984; 77: 878–881.
Lacy CR, Suh DC, Bueno M, Kostis JB. Delay in presentation and evaluation for acute stroke: stroke time registry for Outcomes Knowledge and Epidemiology (S.T.R.O.K.E.). Stroke. 2001; 32: 63–69.
Wester P, Radberg J, Lundgren B, Peltonen M. Factors associated with delayed admission to hospital and in-hospital delays in acute stroke and TIA. Stroke. 1999; 30: 40–48.