Published online before print December 12, 2008, doi: 10.1161/STROKEAHA.107.510370
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2009;40:555.)
© 2009 American Heart Association, Inc.
Original Contributions |
The Toronto Bedside Swallowing Screening Test (TOR-BSST)
Development and Validation of a Dysphagia Screening Tool for Patients With Stroke
From the Department of Speech-Language Pathology (R.M.) and the Department of Psychiatry (D.L.S.), University of Toronto; Health Care and Outcomes Research (R.M.), Toronto Western Research Institute; the Stroke Program (F.S.), the Department of Speech-Language Pathology (G.N.), and the Department of Gastroenterology (N.E.D.), University Health Network; Department of Physical Medicine and Rehabilitation (R.T.), St. Joseph’s Health Care London; Neurology Rehabilitation Program (M.B.), Toronto Rehabilitation Institute; Kunin-Lunenfeld Applied Research Unit (D.L.S.), Baycrest; Ontario, Canada.
Correspondence to Rosemary Martino, PhD, Department of Speech-Language Pathology, University of Toronto, 160-500 University Ave, Toronto, Ontario, Canada M5G 1V7. E-mail rosemary.martino{at}utoronto.ca
 |
Abstract |
|---|
 Top Abstract IntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Background and Purpose— Dysphagia occurs in 55% of all acute stroke patients. Early identification of dysphagia from screening can lead to earlier treatments and thereby reduce complications. We designed and validated a new bedside dysphagia screening tool—the Toronto Bedside Swallowing Screening Test (TOR-BSST) for stroke survivors in acute and rehabilitative settings.
Methods— The TOR-BSST initially contained 5 items with proven high predictive ability for dysphagia. Trained screeners administer and score the TOR-BSST in less than 10 minutes. Trained nurses from 2 acute and 2 rehabilitation facilities administered the TOR-BSST to consecutively admitted stroke inpatients. A positive screen identified patients at risk for dysphagia. Blinded repeat screenings were conducted within 24 hours. Test-retest reliability was established with the first 50 administrations at an ICC=0.92 (CI, 0.85 to 0.96). Items were eliminated if they contributed 
5% to the total score and were judged clinically impractical. 20% of all enrolled patients were randomly allocated to gold standard videofluoroscopic assessment of swallowing and findings rated independently by 4 blinded experts. Adequate validity was set at sensitivity 
90% and negative predictive value 90%.
Results— 311 stroke inpatients were enrolled; 103 acute and 208 rehabilitation. Screening was positive in 59.2% acute and 38.5% rehabilitation patients. The pharyngeal sensation item did not meet inclusion criteria and was eliminated. The TOR-BSST demonstrated excellent validity with sensitivity at 91.3% (CI, 71.9 to 98.7), and negative predictive values at 93.3% in acute and 89.5% in rehabilitation settings.
Conclusion— The TOR-BSST is a simple accurate tool to identify stroke patients with dysphagia regardless of severity and setting.
Key Words: dysphagia • stroke • screening • assessment scale • validation
|
Introduction |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Dysphagia presents in approximately 55% of all acute stroke patients admitted to hospital.1 The presence of dysphagia can itself cause serious consequences. Patients with dysphagia, regardless of severity or presence of aspiration, are 3 times more likely to develop pneumonia than stroke patients without dysphagia.1 The risk of pneumonia is even higher, approximately 11 times more likely, for those patients with severe dysphagia marked by the presence of aspiration. Similarly, mortality is significantly higher for stroke patients with dysphagia, especially in the first 90 days after stroke.2 At later stages of recovery, stroke survivors continue to be at high risk for dysphagia, and the consequence of undetected dysphagia can lead to serious comorbidities.3,4 Fortunately, there is emerging evidence that early detection of dysphagia reduces not only pulmonary complications, but also reduces length of hospital stay and overall healthcare costs for acute patients.5,6
Stroke guidelines are stressing early dysphagia detection using validated screening and assessment tools. In Canada,7,8 the United States,9 the United Kingdom,10 and Australia,11 stroke guidelines require that a trained clinician screen individuals admitted with stroke or suspicion of stroke for dysphagia as soon as they are alert and able. A standardized tool must be used. Those patients with a positive dysphagia screen result should be kept "nil by mouth" (NPO) and followed with a full assessment of swallowing within 24 hours. The premise is that earlier detection allows for earlier treatment which not only shortens the stroke recovery period, but also reduces the overall rehabilitation costs.6
To date, the Burke Dysphagia Screening Test developed by DePippo and colleagues12 has been the instrument most commonly used to screen for dysphagia in clinical settings.5 Unfortunately, this tool was developed in only the rehabilitation setting with unblinded raters and no reported reliability—both necessary psychometric standards for proper tool validation.13 The purpose of the present study was to design and assess the reliability and validity of the TOR-BSST (the Toronto Bedside Swallowing Screening Test) as a screening instrument to identify dysphagia in stroke survivors across the continuum of care. The TOR-BSST was developed to include only items with potentially high predictive value6 to be simple, and to be readily administered by trained personnel who are not necessarily dysphagia experts. For the TOR-BSST to establish high clinical utility it must have high sensitivity (ability to detect those with dysphagia) and high negative predictive value (ability to rule out those without dysphagia) in both the acute and rehabilitation setting.14 Our study was designed to assess these factors.
|
Materials and Methods |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Development
The items of the TOR-BSST were derived from an extensive systematic review by Martino and colleagues.6,15 This study was the first to systematically target item generation for a screening tool using the best available evidence, and the first to identify the combination of items included in the TOR-BSST. In this review, 49 individual clinical tests were identified but only 2 were found to have potential in accurately predicting dysphagia: dysphonia or coughing during the 50-mL water test by Kidd and colleagues16 and impaired pharyngeal sensation.16,17 Although the predictive ability for these individual tests was assessed with only aspiration on videofluoroscopy, their good predictive value is supported by good study design, including operational definitions for administration and interpretation.6 Two other clinical tests, impaired tongue movement and general dysphonia, showed promise with predictive ability but results were not consistent across different studies. The DePippo 3-oz water test was not included in the TOR-BSST because it was not as accurate a predictor as the Kidd water test with likelihood ratios of 2.5 and 5.0, respectively.6 The remaining tests, including weak voluntary cough without oral intake and gag reflex, were also not included because they did not rate as good predictors of dysphagia because of low likelihood ratios, wide 95% confidence intervals, or inconsistent results from more than one study. Only the 4 clinical tests with highest likelihood ratios were carried forward as screening items, namely the Kidd water swallow test, pharyngeal sensation, tongue movement, and general dysphonia. General dysphonia was split into 2 items, "voice before" and "voice after," in an attempt to secure screener attention to voice quality throughout screening. Therefore, initially a total of 5 items were selected to form the TOR-BSST. Our hypothesis was that in combination these 5 items would have the best predictive ability for increased dysphagia risk.
The response grid and layout were developed by a multi-professional group (3 speech-language pathologists, 2 gastroenterologists, 1 nurse, and 1 neurologist) that works full-time with the acute stroke population and that represents potential users and interpreters of the new screening tool. A pass/fail response was assigned to each item so that failure on any item constitutes a positive screen result and therefore an increased risk for dysphagia. The new form was pilot tested by speech-language pathologists with newly admitted acute stroke patients. Their responses indicated high ratings for ease in administration, scoring, and interpretation. The entire TOR-BSST could be administered, scored, and placed on the medical chart in approximately 10 minutes. Because administration continues only until the first TOR-BSST item is failed, patients with severe dysphagia typically fail an early item thereby reducing administration to less than 10 minutes. The TOR-BSST is intended for use by any health care professional trained in the clinical assessment of poststroke patients. It is designed so that it can be administered to stroke patients across all settings, including acute, rehabilitative, and chronic facilities. The primary purpose of the TOR-BSST is to predict dysphagia defined by aspiration or any physiological abnormality on videofluoroscopy, which is the accepted gold standard. More information about the TOR-BSST is available at http://swallowinglab.uhnres.utoronto.ca/torbsst.html.
Education
A 4-hour didactic session was developed to train screeners on administration and interpretation of the TOR-BSST using digitized real-life examples of 5 stroke patients. Training also included review of basic anatomy and physiology of swallowing as well as strategies to use when administering the TOR-BSST to patients with receptive or expressive aphasia. In addition, screeners were taught how to determine whether patients meet the criteria for dysphagia screening. According to published guidelines,7,8 patients should only be screened for dysphagia if they are alert, can be supported to sit upright, and can follow simple instruction. Patients without these 3 requirements are not to be screened because they are assumed to have dysphagia. Instead screeners are instructed to refer them directly for a swallowing assessment. Didactic training was followed by individual training where nurses independently administered the TOR-BSST to stroke patients. Didactic and individual training was facilitated by a speech-language pathologist with expertise in assessment and management of dysphagia poststroke.
Subjects
Dysphagia Screeners
All nurses working on participating stroke units were eligible to be screeners once they successfully completed the didactic training and 2 consecutive supervised TOR-BSST administrations. Selection of nurse participation was negotiated with hospital administrators to ensure availability of at least 1 trained TOR-BSST dysphagia screener across all shifts.
Stroke Patients
All consecutive patients newly admitted to hospital with the confirmed diagnosis of a brain stem stroke or cerebellar stroke and all other stroke patients with a National Institutes of Health Stroke Scale (NIH-SS) score greater than or equal to 4 were eligible and approached for participation. Stroke was confirmed from a physician’s clinical note or from CT scan or MRI findings. Nonbrain stem and noncerebellar stroke patients with low NIH-SS scores were considered by our stroke neurologist to have no swallowing difficulties; therefore, these patients were noted but excluded. Also excluded were patients with current respiratory compromise, a nonoral feeding regime, or a history of one of more of the following: nonstroke neurological disorder, surgery to the head or neck, a history of previous oropharyngeal dysphagia, dementia, or decreased level of consciousness. Patients were recruited from inpatient stroke units at 2 acute and 2 rehabilitation tertiary care hospitals in southern Ontario.
Procedure
As soon as possible after hospital admission, patients were identified, consented, and enrolled by a research assistant. The presence of aphasia did not exclude patients from participation. For those patients with communication impairment, the research coordinator obtained consent in 1 of 2 ways: (1) directly from the patient using the methods of Supported Conversation for Adults with Aphasia (SCA)18; or (2) from the substitute decision maker in cases where SCA was not possible. After enrolment, 1 in 5 were randomized to receive the videofluoroscopic assessment of swallowing (VFS), the reference standard. The randomization list was generated a priori using SPSS software and kept off site with the central study coordinator. We used a 2-step randomization process to prevent advance knowledge of a patient’s allocation. Each enrolled patient was assigned a study number, which was then matched to a randomization grid on a separate page. Regardless of VFS randomization, all patients were administered the TOR-BSST screen by 2 trained nurses. Nurses were blinded to each other’s screening results, patient medical information, and VFS randomization. As a safety check, 1 in 5 patients whose screen results were negative and all patients whose screen results were positive were referred for a clinical swallowing assessment by a speech-language pathologist. Patients who could not be administered the TOR-BSST were noted and excluded from analysis. All repeat TOR-BSST screenings and swallowing assessments were to be administered within 24 hours of each other. Patient demographic and stroke information was collected immediately after enrollment by the research assistant from review of the medical chart.
This research was conducted with approval of the institutional ethics review boards at each site and registered with www.clinicaltrials.gov (ID NCT00141752). All participants, or substitute decision-makers, gave informed consent.
Reliability
Interrater reliability for administration of the TOR-BSST tool by trained nurse screeners was established with the first 50 patient screenings. The overall agreement between screeners was excellent with a total score intraclass correlation coefficient (ICC)=0.92 (95% CI: 0.85 to 0.96) thereby achieving published standards for tools measuring individual status.19
Item Reduction
Items were eliminated if they singly contributed 5% to the total test score and were judged clinically impractical. Clinically impractical items were identified from independent judgements from 3 or more (of 5) speech-language pathologists with 5 years dysphagia experience.
Criterion Reference Testing
Dysphagia was defined to be any swallow-associated abnormal physiology in the upper aerodigestive tract, including aspiration, during intake of liquid or solid boluses. The presence of dysphagia was judged from the findings of videofluoroscopic swallowing (VFS) test. Despite being the accepted gold standard, reliability among experts remains low.20,21 Therefore we used 4 speech-language pathologists, each with more than 5 years experience with dysphagia and stroke, to make independent and blinded judgements of VFS findings using 3 measures: the Penetration Aspiration Scale (PAS),22 the MASA dysphagia subscore, and the MASA aspiration subscore.23 Latent class modeling was used first to assess reliability among experts and second to combine the 3 judgements from each of the 4 experts to determine the presence or absence of dysphagia on VFS for each patient. We report these statistical methods elsewhere.24
Data Analysis
For dysphagia, patient, and stroke information, univariate statistics were used to describe the data including frequencies, percentages, range, and mean. Accuracy of the TOR-BSST was tested using sensitivity, specificity, likelihood ratios, and predictive values for overall TOR-BSST score and for acute and rehabilitation patient subgroups. Because of the perceived increased risk to patients from radiation exposure, our hospital ethic boards mandated that we randomize to videofluoroscopic assessment according to standard clinical practice. A recent survey reported that approximately 20% of all patients with dysphagia were referred to videofluoroscopy.25 We calculated that a sample size of 60 provided sufficient power for validation statistics. Uncertainty was quantified with 95% confidence intervals for every estimate. A total sample of 300 patients allowed validity testing with a subset of 60 patients (20% of the total) with power at 80% and significance at 5%.
|
Results |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
Participants
Fifty-five nurses were trained, 28 from acute sites and 27 from rehabilitations sites. Of these, 41 (75%) were certified as registered nurses and 14 (25%) as registered practical nurses. A total of 2717 patients were screened for study eligibility between October 28, 2002 and May 31, 2006 (Figure). In an effort to adhere to published practice guidelines,8 the initial study protocol required patients to be screened with the TOR-BSST and assessed using the gold standard for dysphagia within 24 hours of admission to hospital. By study year 3, this criterion was eliminated as study monitors showed a large loss (n=133) of otherwise eligible patients. The 24-hour window was sufficient to identify patients and administer the TOR-BSST but was not practical to abide by research protocols requiring that patients be allowed up to 24 hours to consider study conditions before they consented to be study patients. Overall, 311 patients were assessed for dysphagia with similar demographic and stroke characteristics between sites. (Table 1) The mean time from last seen normal to TOR-BSST screening and VFS assessment was 6.1 day in acute sites and 31.6 days in rehabilitation sites. The mean time from hospital admission to screening was 4.6 days in acute sites and 4.9 days in rehabilitation sites with more than half of these patients screened in less than 3.1 day after admission. The proportion of positive first screenings was 59.2% in acute sites and 38.5% in rehabilitation sites.
|
|
Item Reduction
To identify items for exclusion, positive TOR-BSST results across the original 5 items were compared with the total score (Table 2) and were judged for their practical application. Items #2 (tongue movements) and #4 (water swallows) contributed 8% and 25%, respectively to the total positive score and therefore met our inclusion criteria. Items #1 (voice before), #3 (pharyngeal sensation), and #5 (voice after) each contributed less than 5% to the total positive score. Of these, item #3 was not endorsed by expert clinicians because they reported great difficulty on the part of screeners to differentiate pharyngeal sensation from a gag reflex. Item #3 was therefore judged impractical and did not meet clinical inclusion criteria. Items #1 and #5 were unanimously endorsed by clinicians to be clinically practical and therefore satisfied our inclusion criteria. In summary, of the original 5 items only item #3 (pharyngeal sensation) met our exclusion criteria and therefore was eliminated. The remaining 4 items constitute the final version of the TOR-BSST: voice before, tongue movement, water swallow, and voice after.
|
Validation
Gold standard VFS assessments were conducted on a random subset (n=68; 21.9%) of all enrolled patients.24 Characteristics of patients randomly selected to receive VFS assessment were the same as all other patients in the study. (Table 3) Nine patients were eliminated from accuracy statistics because their first screen and VFS assessments were administered more than 24 hours apart. Of the remaining patients assessed with VFS, 36 (61.0%) were judged by experts to have no dysphagia and 23 (39.0%) to have dysphagia. Separate accuracy statistics were calculated for all patients, the subset of acute patients, and the subset of rehabilitation patients. (Table 4) Predictive values are dependent on prevalence and thus were calculated separately for acute and rehabilitation patients. A good screening test needs to have a high sensitivity and high negative predictive value.14 The TOR-BSST meets these criteria with an overall sensitivity of 91.3% (71.9 to 98.7) across all settings and negative predictive values of 93.3% in acute patients and 89.5% in rehabilitation patients.
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
The TOR-BSST screening tool is a clinical tool that is simple, sensitive, and predictive of dysphagia in patients with stroke in both acute and rehabilitation settings. It fits with published standards for screening tools to have high sensitivity and high negative predictive value.14 This fit is especially important for diseases such as dysphagia where the risk of being undetected can lead to serious consequences of pneumonia,1 malnutrition26 and even death.2 Of particular importance, there is confidence that stroke patients with a negative TOR-BSST result will not have dysphagia, and that the TOR-BSST will miss few if any patients with dysphagia. The necessity for more extensive and specific assessments, and their associated costs in time and money, can be avoided in those patients with negative findings. Therefore, the TOR-BSST functions as a reliable and valid method by which to triage only patients with positive screen results to a more expensive and complicated full assessment by a speech-language pathologist dysphagia expert.
The main purpose of a screening test is to identify as many cases as possible before clinical signs occur.13,27 Hence, it is desirable that the sensitivity be high. For the TOR-BSST, this sensitivity is of prime importance because not identifying dysphagia can lead to the serious consequences of pneumonia5,6 and mortality.2 Because with any diagnostic test there is a trade-off with specificity, the result is an increased number of false-positives. However, because these can be identified later by an experienced speech-language pathologist professional before any intervention is started, it is preferable to increase the sensitivity at the first stage, to avoid an unacceptable number of false-negatives. Such is the case with the TOR-BSST.
At the time this study was initiated, there was only emerging evidence from a systematic review that screening for dysphagia in stroke patients has health benefits.6 Specifically, there was evidence across published studies for the reduction of both pneumonia and death events in patients who were screened for dysphagia, versus those who were not. This review also identified screening to be associated with a reduction in total patient charges by 14.6%. Shortly thereafter, a systematic review of the quality of existing tools reported few available dysphagia screening tools and of those published none were developed with proper study methodology.28 In 1999, the Agency for Health Care Policy and Research declared a need to develop such a tool.29 The Burke Dysphagia Screening tool had been developed and made available in 199412 but did not meet the published standards. Specifically, it was conducted without rater blinding and no assessment for reliability. It was limited to a rehabilitation setting and included a 3-oz water swallow test shown to be less effective than the Kidd 50-mL water swallow test included in the TOR-BSST.6 More recently, another swallowing screening tool was published—the Gugging Swallowing Screen.30 This tool was validated for use with nurse screeners with only 30 acute patients and assessed for its accuracy to detect only aspiration. This is in direct contrast to the common view that dysphagia is any abnormal physiology of the oropharyngeal swallow regardless of aspiration.31 In fact, we now have evidence that dysphagia, regardless of the presence of aspiration, increases the likelihood of pneumonia by three fold.1
It is important to mention that the TOR-BSST was not assessed for accuracy with patients who were tube-fed or had comorbidities such as reduced level of alertness or pulmonary disease. From a clinical perspective, patients who are tube-fed have previously been identified to have dysphagia and thus would not require screening. From a research perspective, the presence of the feeding tube would have biased the screeners’ ratings. Likewise because swallowing requires a basic level of alertness, patients with reduced alertness are clinically considered to have dysphagia and hence do not require screening. Patients with pneumonia were excluded on an ethical basis, that is, to reduce any further risk of aspiration in a pulmonary system that was already seriously compromised. Finally, only patients with stroke were included in this study. Validation of the TOR-BSST with other neurological etiologies is now under separate research. It is important to also note that this study focused on assessing the psychometric properties of the TOR-BSST. Previous pilot work had already established its feasibility for utilization in patients immediately after hospital admission within both acute and rehab settings.
Despite the absence of a completely satisfactory screening tool, further evidence for the benefit of screening has been garnered from more recent prospective multi-site survey research.5 A lower rate of pneumonia was identified in sites with a formal dysphagia screening program, regardless of screening method, compared to sites with no screening. Although these reports of screening benefit were a result of screening tools that were not standardized, it is logical to assume that the benefits would be even greater with the use of a dysphagia screening tool that was systematically developed and tested for stability and accuracy. The added practical and economic value of a screening test that rules out dysphagia is obvious. To date and to accomplish these ends, the TOR-BSST is the only tool developed according to published standards13,32 and with a large stroke population across the continuum of care and with proven psychometric properties.
Summary
In an effort to standardize care for stroke patients across all settings, the Canadian Stroke Strategy7 and the U.S. Joint Commission33 have committed to ensure that every patient admitted to hospital with a suspicion of stroke be screened for dysphagia. The TOR-BSST offers an accurate method to identify stroke patients with dysphagia in the acute and rehabilitation setting with confidence that patients with a negative screen will not have dysphagia. Early positive identification of those stroke patients with dysphagia will allow for earlier referral to dysphagia experts for proper diagnosis and directed treatment, and a reduction in dysphagia-related complications such as pneumonia, malnutrition, and possibly even death.
|
Acknowledgments |
|---|
The authors thank the following for their invaluable contribution to the rigorous conduct of this study: Beatrix Reichardt, MHSc; Norine Foley, MSc; Marla Roth, MHSc; Jamie Bitenski, MSc; Rhonda McNicoll-Whiteman, RN; Sandra Black, MD; Martin O’Donnell, MD; and the speech-language pathology staff at each of the participating hospitals. We extend a special thank you to the staff at the University Health Network whose positive attitude fuelled this project with energy.
Sources of Funding
Financial support for this project was provided primarily by the Canadian Stroke Network (CSN) and also the Canadian Institute of Health Research in the form of a New Investigator Award in Aging to R.M. In addition, funding was provided by the Toronto Rehabilitation Institute. None of the funding sources had influence on interpretation of data or final conclusions.
Disclosures
None.
Received November 18, 2007; revision received July 8, 2008; accepted July 10, 2008.
|
References |
|---|
|
TopAbstractIntroductionMaterials and MethodsResultsDiscussionReferences |
|---|
1. Martino R, Foley N, Bhogal S, Diamant N, Speechley M, Teasell R. Dysphagia after stroke: Incidence, diagnosis, and pulmonary complications. Stroke. 2005; 36: 2756–2763.
2. Sharma JC, Fletcher S, Vassallo M, Ross I. What influences outcome of stroke–pyrexia or dysphagia? Int J Clin Pract. 2001; 55: 17–20.[Medline] [Order article via Infotrieve]
3. Kumlien S, Axelsson K. Stroke patients in nursing homes: eating, feeding, nutrition and related care. J Clin Nurs. 2002; 11: 498–509.[CrossRef][Medline] [Order article via Infotrieve]
4. Loeb M, McGeer A, McArthur M, Walter S, Simor AE. Risk factors for pneumonia and other lower respiratory tract infections in elderly residents of long-term care facilities. Arch Intern Med. 1999; 159: 2058–2064.
5. Hinchey JA, Shephard T, Furie K, Smith D, Wang D, Tonn S. Formal dysphagia screening protocols prevent pneumonia. Stroke. 2005; 36: 1972–1976.
6. Martino R, Pron G, Diamant NE. Screening for oropharyngeal dysphagia in stroke: Insufficient evidence for guidelines. Dysphagia. 2000; 15: 19–30.[Medline] [Order article via Infotrieve]
7. Canadian Stroke Strategy. Canadian best practice recommendations for stroke care. 2006; 112.
8. Heart and Stroke Foundation of Ontario. Improving recognition and management of dysphagia in acute stroke: A vision for Ontario. 2002.
9. Duncan PW, Zorowitz R, Bates B, Choi JY, Glasberg JJ, Graham GD, Katz RC, Lamberty K, Reker D. Management of adult stroke rehabilitation care: a clinical practice guideline. Stroke. 2005; 36: e100–e143.
10. Intercollegiate Stroke Working Party. National Clinical Guidelines for Stroke. London: Royal College of Physicians; 2004.
11. National Stroke Foundation. Clinical guidelines for acute stroke management. 2007.
12. DePippo KL, Holas MA, Reding MJ. The Burke dysphagia screening test: Validation of its use in patients with stroke. Arch Phys Med Rehab. 1994; 75: 1284–1286.[Medline] [Order article via Infotrieve]
13. Streiner DL. Diagnosing tests: using and misusing diagnostic and screening tests. J Pers Assess. 2003; 81: 209–219.[CrossRef][Medline] [Order article via Infotrieve]
14. Swets JA. The science of choosing the right decision threshold in high-stakes diagnostics. Am Psychol. 1992; 47: 522–532.[CrossRef][Medline] [Order article via Infotrieve]
15. Fernie RM, Pron G, Diamant NE. Screening for oropharyngeal dysphagia in stroke patients: Insufficient evidence for guidelines. Abstract. Dysphagia. 1997; 12: 115.
16. Kidd D, Lawson J, Nesbitt R, MacMahon J. Aspiration in acute stroke: A clinical study with videofluoroscopy. Q J Med. 1993; 86: 825–829.[Medline] [Order article via Infotrieve]
17. Horner J, Buoyer FG, Alberts MJ, Helms MJ. Dysphagia following brain-stem stroke. Arch Neurol. 1991; 48: 1170–1173.
18. Kagan A, Black SE, Duchan FJ, Simmons-Mackie N, Square P. Training volunteers as conversation partners using "Supported Conversation for Adults with Aphasia" (SCA): A controlled trial. J Speech Lang Hear Res. 2001; 44: 624–638.
19. Nunnally JC, Bernstein IH. Psychometric Theory. Toronto: McGraw-Hill, Inc.; 1994.
20. Scott A, Perry A, Bench J. A study of interrater reliability when using videofluoroscopy as an assessment of swallowing. Dysphagia. 1998; 13: 223–227.[CrossRef][Medline] [Order article via Infotrieve]
21. Stoeckli SJ, Huisman TA, Seifert B, Martin-Harris BJ. Interrater reliability of videofluoroscopic swallow evaluation. Dysphagia. 2003; 18: 53–57.[CrossRef][Medline] [Order article via Infotrieve]
22. Rosenbek JC, Robbins JA, Roecker EB, Coyle JL, Wood JL. A penetration-aspiration scale. Dysphagia. 1996; 11: 93–98.[CrossRef][Medline] [Order article via Infotrieve]
23. Mann G MASA: The Mann Assessment of Swallowing Ability. Clifton Park, NY: Singular; 2002.
24. Kowgier ME, Martino R, Tomlinson G Measuring agreement in videofluoroscopic ratings of swallowing impairment in stroke patients using Bayesian latent class models. 2007.
25. Martino R, Pron G, Diamant NE. Oropharyngeal dysphagia: surveying practice patterns of the speech-language pathologist. Dysphagia. 2004; 19: 165–176.[Medline] [Order article via Infotrieve]
26. Davalos A, Ricart W, Gonzalez-Huix F, Soler S, Marrugat J, Molins A, Suner R, Genis D. Effect of malnutrition after acute stroke on clinical outcome. Stroke. 1996; 27: 1028–1032.
27. Eddy DM. How to think about screening. In: Eddy DM, ed. Common Screening Tests. Philadelphia: American College of Physicians 1991; 1–21.
28. Perry L. Screening swallowing function of patients with acute stroke. Part one: Identification, implementation and initial evaluation of a screening tool for use by nurses. J Clin Nurs. 2001; 10: 463–473.[CrossRef][Medline] [Order article via Infotrieve]
29. Agency for Health Care Policy and Research. Diagnosis and treatment of swallowing disorders (dysphagia) in acute-care stroke patients. Evidence Report: Technology Assessment (Summary). 1999; 1–6.
30. Trapl M, Enderle P, Nowotny M, Teuschl Y, Matz K, Dachenhausen A, Brainin M. Dysphagia bedside screening for acute-stroke patients: the Gugging Swallowing Screen. Stroke. 2007; 38: 2948–2952.
31. Logemann JA. Dysphagia: Evaluation and treatment. Folia Phoniatr Logo. 1995; 47: 140–164.
32. Meyer GJ. Guidelines for reporting information in studies of diagnostic test accuracy: the STARD initiative. J Pers Assess. 2003; 81: 191–193.[CrossRef][Medline] [Order article via Infotrieve]
33. The Joint Commission. Disease-specific care certification program: stroke performance measurement implementation guide. 2007.
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||