doi: 10.1161/hs0102.101990
(Stroke. 2002;33:160.)
© 2002 American Heart Association, Inc.
Original Contributions |
Improving Delivery of Acute Stroke Therapy
The TLL Temple Foundation Stroke Project
From the Stroke Program, Department of Neurology (L.B.M., L.S., T.H.W., M.K., R.A.F., W.S.B., S.L.H., K.S., A.M.D., A.K., A.D., J.C.G.), and Department of Family Practice (J.G.), University of Texas Medical School, Houston; and the Departments of Epidemiology (L.B.M.), Biometry (W.C.), and Health Education and Health Behavior (L.K.B., J.G.), University of Texas School of Public Health, Houston.
Correspondence to Lewis B. Morgenstern, MD, Associate Professor of Neurology and Epidemiology, Department of Neurology, University of Texas, Houston, 6431 Fannin, Room 7.044, Houston, TX 77030. E-mail Lewis.Morgenstern{at}uth.tmc.edu
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background and Purpose— Only a small minority of acute stroke patients receive approved acute stroke therapy. We performed a community and professional behavioral intervention project to increase the proportion of stroke patients treated with approved acute stroke therapy.
Methods— This study used a quasi-experimental design. Intervention and comparison communities were compared at baseline and during educational intervention. The communities were based in 5 nonurban East Texas counties. The multilevel intervention worked with hospitals and community physicians while changing the stroke identification skills, outcome expectations, and social norms of community residents. The primary goal was to increase the proportion of patients treated with intravenous recombinant tissue plasminogen activator (rTPA) from 1% to 6% of all cerebrovascular events in the intervention community.
Results— We prospectively evaluated 1733 patients and validated 1189 cerebrovascular events. Intravenous rTPA treatment increased from 1.38% to 5.75% among all cerebrovascular event patients in the intervention community (P=0.01) compared with a change from 0.49% to 0.55% in the comparison community (P=1.00). Among the ischemic stroke patients, an increase from 2.21% to 8.65% was noted in the intervention community (P=0.02). The comparison group did not appreciably change (0.71% to 0.86%, P=1.00). Of eligible intravenous rTPA candidates, treatment increased in the intervention community from 14% to 52% (P=0.003) and was unchanged in the comparison community (7% to 6%, P=1.00).
Conclusions— An aggressive, multilevel stroke educational intervention program can increase delivery of acute stroke therapy. This may have important public health implications for reducing disability on a national level.
Key Words: education • emergency medical services • stroke, acute • thrombolytic therapy
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
In June 1996, the US Food and Drug Administration (FDA) approved intravenous recombinant tissue plasminogen activator (rTPA) for acute ischemic stroke. Treatment needs to begin within 3 hours of symptom onset. The approval was based largely on the 2 combined National Institute of Neurologic Disease and Stroke (NINDS) Acute Stroke Studies.1 Including the risk of intracranial hemorrhage, these studies suggested that the use of intravenous rTPA resulted in at least a 30% relative benefit in reducing disability from stroke. Consensus statements from the American Heart Association,2 American Academy of Neurology,3 and American College of Chest Physicians4 supporting the use of intravenous rTPA were published (grade A recommendations). Although these leading organizations published these statements supporting the use of intravenous rTPA for acute ischemic stroke, some individuals have expressed reservations in editorial format.5 Despite the published data, consensus statements, and guidelines, only a very small minority of acute stroke patients currently receive intravenous rTPA for acute ischemic stroke nationally.
Speculations regarding the cause of the low treatment rate include a long delay time for hospital presentation6–15 and reluctance among physicians to treat patients.16 The TLL Temple Foundation Stroke Project was organized to determine whether an aggressive, scientifically based behavioral intervention could increase the proportion of stroke patients treated with FDA-approved acute stroke therapy within a representative community.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
The study used a quasi-experimental comparison group design within 2 communities.17 The intervention was located in the community of the sponsoring agency. The comparison community was chosen to provide matched hospitals and similar nonurban demographic characteristics. Both communities were in East Texas, and they did not have overlapping media orbits. Each community was far enough from Houston that referral to Houston for acute stroke care was prohibitive without an initial stop at a local hospital. The intervention community contained 5 hospitals in Angelina, Nacogdoches, and Shelby counties in Texas. The comparison community contained 5 hospitals in Jefferson and Orange counties in Texas. In 1998, the intervention community comprised an estimated 160 833 residents (77% non-Hispanic white). The comparison community contained an estimated total of 332 676 residents (66% non-Hispanic white).18 A neurologist was on staff in 4 of 5 intervention hospitals and in all comparison hospitals. All 10 hospitals had 24-hour emergency departments and protocols for emergent head CT imaging. Both communities had full 911 emergency medical services (EMS).
This project had 2 prespecified phases. Phase I ran from February 1998 through October 1998; phase II ran from January 1999 through March 2000. In phase I, baseline data collection occurred in the intervention and comparison communities, and the intervention was developed. The community process relied on information from focus groups with stroke patients and caregivers to identify issues contributing to delay times for hospital presentation. A local community advisory board was formed in the intervention community to provide community-specific information relevant to the intervention. A random-digit dialed telephone survey (n=656) was conducted with adults in both intervention and comparison communities to assess factors related to intention to react rapidly to signs of stroke.
The data obtained during phase I were used in a systematic process of intervention development.19 The planning data suggested that simply providing knowledge of stroke symptoms and asking patients to call EMS for these symptoms was unlikely to be effective. Targeting outcome expectations, stroke recognition skills and self-efficacy, and perceived community norms and removing community-specific barriers were thought to be critical to reduce delay times and get patients to the hospital quickly after stroke symptom onset. Additionally, published information on factors influencing delay time for acute stroke presentation6–15 and suggested systems approaches20 were used for intervention planning.
The community intervention relied on role modeling of response to stroke symptoms in both mass and small media. Mass media strategies included billboard advertising, radio and television public service announcements, and news stories. "Small" media included brochures and posters (Figure 1). In addition, volunteers were trained in stroke recognition and reaction and in turn trained coworkers. Community figures were used as role models to show calling 911 immediately for stroke symptoms and to demonstrate that responding to stroke immediately can result in a better outcome (Figure 1). Community members were also encouraged to be assertive in asking the physician about TPA if taken to the emergency department for stroke symptoms, thereby providing a cue to act for healthcare providers who were initially reluctant.
|
Our group conducted 488 trainings to provide 634 subsequent trainers. This group spread the intervention message personally to 49 527 individuals in face-to-face meetings. More than 60 000 brochures on stroke were used, and >5000 posters were displayed in the intervention community. Large employers, hospitals, pharmacies, health clinics, and places of worship were targeted. Public service announcements were professionally created. Three local television statements ran the messages a total of 675 times. Radio public service announcements were broadcast 3376 times. There were 5 full-size billboards in the intervention community used for the intervention message.
The healthcare provider component of the intervention relied on systems change in hospitals, change of perceived norms in the medical community, and reinforcement of behavior change through both mass media (eg, highlighting successes in news stories) and small media (eg, providing newsletters to primary care providers, emergency physicians, and neurologists that provided feedback regarding stroke treatment accomplishments). Multidisciplinary teams in hospitals developed emergency department protocols, solved problems related to responsibility for care, and scheduled continuing medical education and mock "stroke codes" for both hospital and EMS staff.
Prospective data acquisition used both active and passive pursuit methodologies.21 All emergency department and hospital admission logs were first actively screened with published stroke screening terms.22 Passive surveillance techniques included recapture methods of stroke cases by use of discharge ICD-9 codes (430 through 438) listings obtained from each hospital at the end of the month. Trained abstractors blinded to the study hypothesis recorded demographic and clinical information into a case report form that had been extensively pilot tested in Houston and both communities. Quality assurance measures required frequent dual chart abstractions among 2 different abstractors and subsequent review by study investigators. Patient interviews began as soon as possible after hospital admission. Patients were asked to answer 3 general questions. If they were unable to answer because of aphasia, motor impairment, or other stroke manifestation, then the individuals most familiar with the patients’ daily activities were questioned. The interview was a formal structured interview, read from a script and recorded by the trained abstractors. Stroke cases were validated by fellowship-trained stroke neurologists using source documentation based on published criteria.23 Cases were defined as transient ischemic attack, ischemic stroke, intracerebral hemorrhage, or subarachnoid hemorrhage on the basis of clinical and radiological criteria.23,24 The stroke neurologists also reviewed the source documentation to determine whether the patient met the inclusion and exclusion criteria on the package insert for treatment with intravenous rTPA. The stroke neurologists were blinded to community for all assessments.
The primary prespecified outcome measure was the proportion of all cerebrovascular patients (ischemic stroke, intracranial hemorrhage, and transient ischemic attack patients) treated with intravenous rTPA. It was hypothesized from industry estimates that 1% of patients nationally received intravenous rTPA for acute ischemic stroke. The study was powered to detect a difference in the intervention community from an assumed baseline treatment rate of 1% to a postintervention treatment rate of 6%. Stroke incidence data were not available for the communities before the study. Incidence rates were approximated by extrapolating from county stroke mortality rates available from the Texas Department of Health and assuming a 30% stroke mortality rate. A sample size of 249 in the intervention group was needed in both phases to detect the desired difference with 80% power and 
=0.05. Before the study began, we decided to run phase I for 9 months and phase II for 15 months to meet sample size estimates and to give the intervention time to reach the population. The other primary outcome measures were the proportion of ischemic stroke patients treated with intravenous rTPA and the proportion of patients treated who met label eligibility criteria for intravenous rTPA for acute ischemic stroke. Although the study was powered to detect change in the intervention community only, a comparison community was also examined in parallel to determine whether change in the proportion of patients treated with intravenous rTPA for acute ischemic stroke had occurred.
Secondary outcome measures were (1) a change in the proportion of patients treated with intravenous rTPA in phase II compared with phase I in the intervention community relative to the comparison community; (2) a reduction in delay time from symptom onset to hospital arrival in the intervention community in phase II compared with phase I; (3) mention of consideration of the use of intravenous rTPA in physician notes in the hospital chart in the intervention community; (4) an increase in the proportion of stroke patients treated with intravenous rTPA with progressing month of the intervention; and (5) prespecified exploratory analyses that included a breakdown of eligible candidates into those treated, those not treated because of in-hospital delay, and those not treated for unknown reasons
Data collected in this study were entered into Microsoft Access. Measurements out of range were identified, checked with the hard copy, and corrected if needed. The verified data were then transferred into a SAS database, and analyses were performed with SAS, version 8. Descriptive statistics of demographic variables, risk factors, and clinical symptoms of stroke patients were calculated for each phase, separated by intervention and control groups. For dichotomous variables, Fisher’s exact test was applied to examine the homogeneity of the distribution between the intervention and control communities within each phase. For continuous variables, a 2-sample t test was used for the similar assessment.
To study the intervention effect on the proportion of intravenous rTPA treatment (a secondary outcome) and on the proportion of patients presenting within 2 hours of symptom onset, the Breslow-Day test for homogeneity of odds ratios was applied. The trend comparisons of delay time (treated as a continuous variable) over 2 phases were performed with general linear models. Because of the skewness of the delay time distribution, this outcome was first logarithmically transformed before the analyses were carried out. This study was approved by the University of Texas at Houston Committee for the Protection of Human Subjects and by each individual hospital. Subjects signed informed consent before being interviewed.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
During the study, 1733 patients were screened positive for a presenting symptom suggestive of stroke. Of these, 1189 were validated as having had a cerebrovascular event (ischemic stroke, intracranial hemorrhage, or transient ischemic attack). Table 1 presents the demographic and clinical information by study phase and community status. Consistently, 64% to 70% of the cerebrovascular events were ischemic stroke in both phases and in both communities. Table 2 presents the primary outcome measure of proportion of intravenous rTPA treatments among all cerebrovascular patients, ischemic stroke patients, and candidates eligible for treatment. Using any of the 3 denominators gives a significantly greater proportion of patients treated with intravenous rTPA in phase II compared with patients treated in phase I in the intervention community. No significant increase in treatment was noted in the comparison community.
|
|
The first secondary outcome measure compared the change in proportion of all cerebrovascular event patients treated from phase I to phase II in the intervention community with the change in proportion of patients treated in the comparison community. These proportions were not significantly different (P=0.30). Similar comparisons for ischemic stroke patients and TPA candidates also did not detect a significant change (P=0.35, P=0.11, respectively). This study would need considerably more subjects and probably multiple communities to demonstrate such a difference.
Table 3 presents delay time information for both phases and both communities. Delay time information was available in 68% of cases, similar to that found in other community efforts.25 Delay time to hospital presentation decreased in both communities over time. Breslow-Day tests for homogeneity of the odds ratios failed to demonstrate a significant relationship of the intervention in increasing the proportion of patients arriving within 2 hours of symptom onset. Similarly, a reduction in mean delay time could not be ascribed to the effects of intervention when the trends of delay over 2 phases were compared by use of general linear models.
|
The third secondary outcome measure was mention in the chart about candidacy for intravenous rTPA. This did not change between phases. In the intervention community, consideration of the use of intravenous rTPA was mentioned in the chart in 8.4% of cerebrovascular event patients in phase I and 7.6% in phase II. In the comparison community, it was 7.1% in phase I and 7.6% in phase II.
The fourth prespecified secondary outcome measure was determination of a potential time lag for the full effect of the intervention. We examined the trend for the proportion of ischemic stroke patients treated by month in each community. Figure 2 demonstrates this relationship. Logistic regression analysis did not demonstrate an increase in the proportion of ischemic stroke patients treated with intravenous rTPA over time of the intervention (P=0.91).
|
The results of how eligible candidates were handled by phase and community are shown in Figure 3. In phase II, eligible intravenous rTPA candidates in the intervention community experienced less hospital delay and less unexplained nontreatments. There was little change between the two phases in the comparison community. Protocol violations were found in 13% of patients receiving rTPA among phase II patients in the intervention community. Among the few treated patients in the comparison community and those treated in phase I in the intervention community, none of the patients were treated outside protocol specifications.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
This study provides evidence that aggressive professional and community education may increase the rate of use of FDA-approved acute stroke therapy. This increase was seen in a representative nonurban community. The comparison location did not see an increase in treatment during the same time interval. Although intravenous rTPA remains the only FDA-approved acute stroke therapy, animal investigations suggest that other thrombolytic or neuroprotective approaches are likely to have short time windows for efficacy and safety. Intra-arterial thrombolytic administration may play a role at select centers but will also require prompt hospital presentation and triage.25 Effective community and professional education and motivation for expeditious treatment are likely to be important for successful acute stroke treatment regardless of the agent administered.
The professional component of the intervention likely had the largest impact. Treatment of eligible candidates increased from 14% in phase I to 52% in phase II in the intervention community. The multilevel professional intervention used individual contact and explanation of acute stroke guidelines with emergency department physicians. This process allows adoption of therapeutic advances after the opportunity to review and interact with a peer health professional. This enhances knowledge, self-efficacy, and outcome expectations. This method has previously demonstrated a robust effect.26 Paramedics, physicians, and emergency department staff were allowed to practice acute stroke treatment during mock "stroke codes," which facilitated self-efficacy. Academic detailing was an important part of the intervention. The health educator and physician-investigators made regular trips to emergency departments and high-volume primary care physicians to remind them of acute stroke therapy.27 Messages were repeated through high-quality graphic presentations, and positive reinforcement was used to modify behavior.28
In the present study, delay time to hospital presentation decreased significantly in both intervention and comparison communities from phase I to phase II. We did not monitor background stroke awareness campaigns in either community. This may have been important to determine why delay time decreased in both communities. Both communities are served by the same American Heart Association affiliate, and none of the hospitals in either community advertised stroke services during the study period. This secular trend of reduced delay time may have resulted from general increased public awareness of the importance of time in acute stroke. The discrepancy of reduced delay time without increased treatment may imply a barrier in the comparison community. In poststudy discussions with community neurologists, no barriers that differed from the intervention community were readily apparent.
Examinations of the delay time data may indicate that the community intervention was unsuccessful because delay time reductions in the intervention community did not exceed those in the comparison community. However, the professional and community interventions were inextricably linked in this project. One feature of the community behavioral intervention was to encourage potential stroke patients to ask for TPA in the emergency department and to suggest that paramedics radio ahead with a potential "TPA stroke patient." The effectiveness of these efforts is difficult to measure in isolation, although several physicians reported that patients asked for specific treatment. Medical personnel (doctors, nurses, and paramedics) were introduced to colleagues and patients who had used and received the treatment successfully. Stories in the media and public service announcements were frequently discussed between patients and doctors. In this way, the community intervention diffused throughout the professional community and likely explains in part the improved treatment of eligible candidates by physicians.
Figure 3 shows that among the eligible candidates, the group that was not treated because of delay or for unknown reasons was shifted to treated patients after the intervention. For those who remained untreated, emergency department physicians told us that they remained concerned about treating patients without the support of accepting primary care physicians who would care for the patient in the immediate post-rTPA period. As the intervention progressed, we found that local primary care doctors were critical to supporting treatments in the emergency department. The number of eligible candidates was greater in phase II than phase I because phase I ran 9 months and phase II ran 15 months and because delay times were reduced in both communities. We cannot explain why the proportion of eligible candidates was lower in phase I in the comparison group, but the difference was not significant.
The results shown in Figure 2 were disappointing. We hoped that the effect of the intervention would increase over time. Instead, we found no trend. This may suggest that the intervention must continue for sustained benefits, but longer follow-up is needed to test this theory.
Early efforts to reduce delay time to hospital presentation for acute stroke used a long time window as a target. In North Carolina, investigators were able to increase the proportion of patients presenting within 24 hours from stroke symptom onset from 37% to 86% after community-wide education.29 An NIH consensus group, however, considers 2 hours to be the target upper limit time to hospital presentation to accomplish intravenous rTPA treatment within 3 hours of symptom onset.30 Indeed, recent evidence suggests that the effectiveness of intravenous rTPA in reducing disability is clearly time dependent. Patients treated within 1 hour of symptom onset are 
4 times more likely to recover back to independent function compared with those treated just before the 3-hour time window expires.31 Convincing physicians that stroke is a medical emergency may be difficult. A multicenter study reported that faced with a 3-hour time window, physicians are likely to treat just before this window expires rather than treat early-presenting patients immediately. "Door to needle time" can be quite short for those presenting late and quite long for those presenting early.32
Community interventions must begin with a target. Only a few percent of EMS calls are initiated by the patient.33 The target for the behavioral intervention then becomes the family and coworkers. Studies attempting to limit delay time for hospital presentation for acute myocardial infarction have not shown consistent results.34–40 The recently published Rapid Early Action for Coronary Treatment (REACT) study34 randomized 10 community pairs in the United States and, following a baseline phase, used a well-organized, thorough community intervention program to reduce prehospital delay. Although appropriate EMS use was increased, delay time to hospital presentation was not changed by the intervention.
The present study is limited by its nonrandomized design. The funding agency requested that the intervention be carried out in its community. A comparison community was sought by the investigators to be a geographic and demographic community match blinded to prehospital delay information or attitudes regarding rTPA among community physicians. The study data collection methods were hospital based because outcome measures were arrival time and treatment. The intervention, however, was aimed at the entire community. We matched hospitals by size. Although all 5 intervention community hospitals were used in this study, data from only 5 of the comparison community hospitals were needed to achieve a comparable sample size. Furthermore, the study did not examine EMS documentation to see whether the frequency or appropriateness of 911 usage was affected. For an individual subject, the study ended after a decision was made to treat or not treat the patient with intravenous rTPA. There is information on the safety and efficacy of intravenous rTPA.1,41–45 This study was not intended to examine outcome after intravenous rTPA but rather to determine whether the use of FDA-approved acute stroke therapy could be increased appropriately in a nonurban environment. Indeed, the rate of protocol violations in the intervention community in phase II was less than that previously reported.45 Nevertheless, the lack of outcome data prohibits a cost-effectiveness analysis. Allocation of resources to treat stroke may be better served by stroke prevention strategies, but this requires further study.
Effective acute stroke therapy goes above and beyond rTPA and includes preventing medical complications and other therapies as illustrated by the successes of stroke units.46 The present study is relevant to the 61 million US residents who reside in nonurban areas47; generalizing these results to urban populations may be inappropriate. An alternative view would suggest that if this program was successful in nonurban East Texas, it is likely to have an impact in areas with greater population densities that include closer distances to the hospital, more healthcare services, and expanded media opportunities. Future studies are needed to confirm these findings. Urban areas may benefit from replicating the professional intervention and parts of the community intervention of this project. In addition, future work should be performed in a multicenter, randomized design that incorporates monitoring of secular trends in both communities. Furthermore, cost analysis and outcomes and community and professional surveys that monitor the reasons why components of the intervention were or were not successful should be included.
|
Acknowledgments |
|---|
This study was funded by a grant from the TLL Temple Foundation.
Received August 16, 2001; revision received October 11, 2001; accepted October 25, 2001.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1. National Institute of Neurological Disorders and Stroke r-tPA Stroke Study Group. Tissue plasminogen activator for acute ischemic stroke. N Engl J Med. 1995; 333: 1581–1587.
2.
Adams HP, Brott TG, Furlan AJ, Gomez CR, Grotta J, Helgason CM, Kwiatkowski T, Lyden PD, Marler JR, Torner J, Feinberg W, Mayberg M, Thies W. Guidelines for thrombolytic therapy for acute stroke: a supplement for the guidelines for the management of patients with acute ischemic stroke: a statement for healthcare professionals from a special writing group of the Stroke Council, American Heart Association. Circulation. 1996; 94: 1167–1174.
3.
Report of the Quality Standards Subcommittee of the American Academy of Neurology. Practice advisory: thrombolytic therapy for acute ischemic stroke: summary statement. Neurology. 1996; 47: 835–839.
4.
Albers GW, Easton JD, Sacco RL, Teal P. Antithrombotic and thrombolytic therapy for ischemic stroke. Chest. 1998; 114 (suppl): 683S–698S.
5.
Caplan LR, Mohr JP, Kistler JP, Koroshetz W. Should thrombolytic therapy be the first-line treatment for acute ischemic stroke? Thrombolysis: not a panacea for ischemic stroke. N Engl J Med. 1997; 337: 1309–1310.
6.
Alberts MJ, Bertels C, Dawson DV. An analysis of time of presentation after stroke. JAMA. 1990; 263: 65–68.
7.
Feldmann E, Gordon N, Brooks JM, Brass LM, Fayad PB, Sawaya KL, Nazareno F, Levine SR. Factors associated with early presentation of acute stroke. Stroke. 1993; 24: 1805–1810.
8.
Menon SC, Pandey DK, Morgenstern LB. Critical factors determining access to acute stroke care. Neurology. 1998; 51: 427–432.
9. Barsan WG, Brott TG, Broderick JP, Haley EC, Levy DE, Marler JR. Time of hospital presentation in patients with acute stroke. Arch Intern Med. 1993; 153: 2258–2561.
10. Rosamond WD, Gorton RA, Hinn AR, Hohenhaus SM, Morris DL. Rapid response to stroke symptoms: the Delay in Accessing Stroke Healthcare (DASH) Study. Acad Emerg Med. 1998; 5: 45–51.[Medline] [Order article via Infotrieve]
11.
Schroeder EB, Rosamond WD, Morris DL, Evenson KR, Hinn AR. Determinants of use of emergency medical services in a population with stroke symptoms: the Second Delay in Accessing Stroke Healthcare (DASH II) Study. Stroke. 2000; 31: 2591–2596.
12.
Morris DL, Rosamond W, Madden K, Schultz C, Hamilton S. Prehospital and emergency department delays after acute stroke: the Genentech Stroke Presentation Survey. Stroke. 2000; 31: 2585–2590.
13.
Fogelholm R, Murros K, Rissanem A, Ilmavirta M. Factors delaying hospital admission after acute stroke. Stroke. 1996; 27: 398–400.
14.
Wester P, Radberg J, Lundgren B, Peltonen M. Factors associated with delayed admissions to hospital and in-hospital delays in acute stroke and TIA: a prospective, multicenter study. Stroke. 1999; 30: 40–48.
15.
Azzimondi G, Bassein L, Fiorani L, Nonino F, Montaguti U, Celin D, Re G, D’Alessandro R. Variables associated with hospital arrival time after stroke: effect of delay on the clinical efficiency of early treatment. Stroke. 1997; 28: 537–554.
16.
Villar-Cordova C, Morgenstern LB, Barnholtz JS, Frankowski RF, Grotta JC. Neurologists’ attitudes regarding rTPA for acute ischemic stroke. Neurology. 1998; 50: 1491–1494.
17. Cook TD, Campbell DT. Quasi-Experimentation: Design and Analysis Issues for Field Settings. Boston, Mass: Houghton Mifflin Co; 1979.
18. Population Projections, 1.0 Scenario, Texas State Data Center, Texas Population Estimates and Projections Programs, Texas A&M University, 1998. Available at: http://www.tdh.state.tx.us/dpa/popdata/ST1998.htm. Accessed August 2001.
19. Bartholomew LK, Parcel GS, Kok G. Intervention mapping: a process for designing theory- and evidence-based health education programs. Health Educ Behavior. 1998; 25: 545–563.[Abstract]
20.
Tilley BC, Lyden PD, Brott TG, Lu M, Levine SR, Welch KMA. Total quality improvement method for reduction of delays between emergency department admission and treatment of acute ischemic stroke. Arch Neurol. 1997; 54: 1466–1474.
21.
Tuomilehto J, Sarti C, Narva E, Salmi K, Kaarsalo E, Salomaa V, Torppa J. The FINMONICA Stroke Register: community based stroke registration and analysis of stroke incidence in Finland, 1983–1985. Am J Epidemiol. 1992; 135: 1259–1270.
22.
Morgenstern LB, Wein TH, Smith MA, Moyé LA, Pandey DK, Labarthe DR. Comparison of stroke hospitalization rates among Mexican Americans and non Hispanic whites. Neurology. 2000; 54: 2000–2002.
23. Asplund K, Tuomilheto J, Stegmayr B, Wester P, Tunstall-Pedoe H. Diagnostic criteria and quality control of the registration of stroke events in the MONICA project. Acta Med Scand. 1988; 728 (suppl): 26–39.
24.
Broderick J, Brott T, Kothari R, Miller R, Khoury J, Pancioloi A, Gebel J, Mills D, Minneci L, Shukla R. The Greater Cincinnati. Northern Kentucky Stroke Study: preliminary first ever and total incidence rates of stroke among blacks. Stroke. 1998; 29: 415–442.
25.
Furlan A, Higashida R, Wechsler L, Gent M, Rowley H, Kase C, Pessin M, Ahuja A, Callahan F, Clark WM, Silver F, Rivera F. Intra-arterial Prourokinase for acute ischemic stroke: the PROACT II study: a randomized controlled trial. JAMA. 1999; 282: 2003–2011.
26.
Onion CWR, Bartzokas CA. Changing attitudes to infection management in primary care: a controlled trial of active versus passive guideline implementation strategies. Fam Pract. 1998; 15: 99–104.
27. Cohen SJ, Halvorson HW, Gosselink CA. Changing physician behavior to improve disease prevention. Prev Med. 1994; 23: 284–291.[CrossRef][Medline] [Order article via Infotrieve]
28.
Soumerai SB, Avorn J. Principles of educational outreach ("academic detailing") to improve clinical decision making. JAMA. 1990; 263: 549–556.
29.
Alberts MJ, Perry A, Dawson DV, Bertels C. Effects of public and professional education on reducing the delay in presentation and referral of stroke patients. Stroke. 1992; 23: 352–356.
30. National Institute of Neurological Disorders and Stroke. Rapid identification and treatment of acute stroke. Proceedings of a national symposium. Bethesda, Md: National Institutes of Health Monograph; 1997.
31.
Marler JR, Tilley BC, Lu M, Brott TG, Lyden PD, Grotta JC, Broderick JP, Levine SR, Frankel MP, Horowitz SH, Haley EC, Lewandowski CA, Kwiatkowski TP. Early stroke treatment associated with better outcome: the NINDS rt-PA stroke study. Neurology. 2000; 55: 1649–1655.
32.
Albers GW, Bates VE, Clark WM, Bell R, Verro P, Hamilton SA. Intravenous tissue-type plasminogen activator for treatment of acute stroke: the Standard Treatment With Alteplase to Reverse Stroke (STARS) Study. JAMA. 2000; 283: 1145–1150.
33.
Wein TH, Staub L, Felberg R, Hickenbottom SL, Chan W, Grotta JC, Demchuk AM, Groff J, Bartholomew LK, Morgenstern LB. Activation of emergency medical services for acute stroke in a nonurban population: the TLL Temple Foundation Stroke Project. Stroke. 2000; 31: 1925–1928.
34.
Luepker RV, Raczynski JM, Osganian S, Goldberg RJ, Finnegan JR Jr, Hedges JR, Goff DC Jr, Eisenberg MS, Zapka JG, Feldman HA, Labarthe DR, McGovern PG, Cornell CE, Proschan MA, Simons-Morton DG. Effect of a community intervention on patient delay and emergency medical service use in acute coronary heart disease: the Rapid Early Action for Coronary Treatment (REACT) trial. JAMA. 2000; 284: 60–67.
35. Ho MT, Eisenberg MS, Litwin PE, Schaeffer SM, Damon SK. Delay between onset of chest pain and seeking medical care: the effect of public education. Ann Emerg Med. 1989; 18: 727–731.[CrossRef][Medline] [Order article via Infotrieve]
36.
Gaspoz JM, Unger PF, Urban P, Chevrolet JC, Rutishauser W, Lovis C, Goldman L, Heliot C, Sechaud L, Mischler S, Waldvogel FA. Impact of a public campaign on pre-hospital delay in patients reporting chest pain. Heart. 1996; 76: 150–155.
37. Mitic WR, Perkins J. The effect of a media campaign on heart attack delay and decision times. Can J Public Health. 1984; 75: 414–418.[Medline] [Order article via Infotrieve]
38.
Herlitz J, Blohm M, Hartford M, Karlson BW, Luepker R, Holmberg S, Risenfors M, Wennerblom B. Follow-up of a 1-year media campaign on delay times and ambulance use in suspected acute myocardial infarction. Eur Heart J. 1992; 13: 1–7.
39. Bett N, Aroney G, Thompson P. Impact of a national educational campaign to reduce patient delay in possible heart attack. Aust N Z J Med. 1993; 23: 157–161.[Medline] [Order article via Infotrieve]
40. Moses HW, Engelking N, Taylor GJ, Prabhakar C, Vallala M, Colliver JA, Silberman H, Schneider JA. Effect of a two-year public education campaign on reducing response time of patients with symptoms of acute myocardial infarction. Am J Cardiol. 1991; 68: 249–251.[CrossRef][Medline] [Order article via Infotrieve]
41.
Chiu D, Krieger D, Villar-Cordova C, Kasner SE, Morgenstern LB, Bratina PL, Yatsu FM, Grotta JC. Intravenous tissue plasminogen activator for acute ischemic stroke: feasibility, safety, and efficacy in the first year of clinical use. Stroke. 1997; 29: 18–22.
42.
Tanne D, Bates VE, Verro P, Kasner SE, Binder JR, Patel SC, Mansbach HH, Daley S, Schultz LR, Karanjia PN, Scott P, Dayno JM, Vereczkey-Porter K, Benesch C, Book D, Coplin WM, Dulli D, Levine SR. Initial clinical experience with IV tissue plasminogen activator for acute ischemic stroke: a multicenter study. Neurology. 1999; 53: 424–427.
43.
Zweifler RM, Brody ML, Graves GC, U TT, Drinkard R, Cunningham S, Rothrock JF. Intravenous tPA for acute ischemic stroke: therapeutic yield of a stroke code system. Neurology. 1998; 50: 501–503.
44.
Lopez-Yunez AM, Bruno A, Williams LS, Yilmaz E, Zurru C, Biller J. Protocol violations in community-based rTPA stroke treatment are associated with symptomatic intracerebral hemorrhage. Stroke. 2001; 32: 12–16.
45. Katzan IL, Furlan AJ, Lloyd LE, Frank JI, Harper DL, Hinchey JA, Hammel JP, Qu A, Sila CA. Use of tissue-type plasminogen activator for acute ischemic stroke: the Cleveland area experience JAMA. 2000; 282: 1151–1158.
46.
Glader EL, Stegmayr B, Johansson L, Hulter-Asberg K, Wester PO. Differences in long-term outcome between patients treated in stroke units and in general wards: a 2-year follow-up of stroke patients in Sweden. Stroke. 2001; 32: 2124–2130.
47. United States Bureau of the Census. Urban and Rural Population. Available at: http://www.census.gov/population/censusdata/urpop0090.txt. Accessed August 2001.
This article has been cited by other articles:
 |
|
 |
|
  D. Kleindorfer, Y. Xu, C. J. Moomaw, P. Khatri, O. Adeoye, and R. Hornung US Geographic Distribution of rt-PA Utilization by Hospital for Acute Ischemic Stroke Stroke, November 1, 2009; 40(11): 3580 - 3584. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Muller-Nordhorn, K. Wegscheider, C. H. Nolte, G. J. Jungehulsing, K. Rossnagel, A. Reich, S. Roll, A. Villringer, and S. N. Willich Population-Based Intervention to Reduce Prehospital Delays in Patients With Cerebrovascular Events Arch Intern Med, September 14, 2009; 169(16): 1484 - 1490. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. K. Bartholomew, W. C Cushman, J. A Cutler, B. R Davis, G. Dawson, P. T Einhorn, J. F Graumlich, L. B Piller, S. Pressel, E. J Roccella, et al. Getting clinical trial results into practice: design, implementation, and process evaluation of the ALLHAT Dissemination Project Clinical Trials, August 1, 2009; 6(4): 329 - 343. [Abstract] [PDF]
|
|
|
|
|
|
A. Tadros, T. Crocco, S. M. Davis, J. Newman, J. Mullen, R. Best, A. Teets, C. Maxwell, B. Slaughter, and S. Teter Emergency Medical Services-Based Community Stroke Education: Pilot Results From a Novel Approach Stroke, June 1, 2009; 40(6): 2134 - 2142. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 H.J. Cloft, A. Rabinstein, G. Lanzino, and D.F. Kallmes Intra-Arterial Stroke Therapy: An Assessment of Demand and Available Work Force AJNR Am. J. Neuroradiol., March 1, 2009; 30(3): 453 - 458. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
O. Williams and J. M. Noble 'Hip-Hop' Stroke: A Stroke Educational Program for Elementary School Children Living in a High-Risk Community Stroke, October 1, 2008; 39(10): 2809 - 2816. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M O McCarron, M Armstrong, and P McCarron Potential for quality improvement of acute stroke management in a district general hospital Emerg. Med. J., May 1, 2008; 25(5): 270 - 273. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
B. M. Demaerschalk, B. J. Bobrow, and M. Paulsen Development of a Metropolitan Matrix of Primary Stroke Centers: The Phoenix Experience Stroke, April 1, 2008; 39(4): 1246 - 1253. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R I Lindley Commentary on NICE guidelines for alteplase for the treatment of acute ischaemic stroke Heart, December 1, 2007; 93(12): 1617 - 1618. [Full Text] [PDF]
|
|
|
|
|
|
J. Juhl Majersik, M. A. Smith, D. B. Zahuranec, B. N. Sanchez, and L. B. Morgenstern Population-Based Analysis of the Impact of Expanding the Time Window for Acute Stroke Treatment Stroke, December 1, 2007; 38(12): 3213 - 3217. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. B. Morgenstern, N. R. Gonzales, K. E. Maddox, D. L. Brown, A. P. Karim, N. Espinosa, L. A. Moye, J. K. Pary, J. C. Grotta, L. D. Lisabeth, et al. A Randomized, Controlled Trial to Teach Middle School Children to Recognize Stroke and Call 911: The Kids Identifying and Defeating Stroke Project Stroke, November 1, 2007; 38(11): 2972 - 2978. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
J. E. Acker III, A. M. Pancioli, T. J. Crocco, M. K. Eckstein, E. C. Jauch, H. Larrabee, N. M. Meltzer, W. C. Mergendahl, J. W. Munn, S. M. Prentiss, et al. Implementation Strategies for Emergency Medical Services Within Stroke Systems of Care: A Policy Statement From the American Heart Association/ American Stroke Association Expert Panel on Emergency Medical Services Systems and the Stroke Council Stroke, November 1, 2007; 38(11): 3097 - 3115. [Full Text] [PDF]
|
|
|
|
|
|
I. Mosley, M. Nicol, G. Donnan, I. Patrick, F. Kerr, and H. Dewey The Impact of Ambulance Practice on Acute Stroke Care Stroke, October 1, 2007; 38(10): 2765 - 2770. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. B. Goldstein Acute Ischemic Stroke Treatment in 2007 Circulation, September 25, 2007; 116(13): 1504 - 1514. [Full Text] [PDF]
|
|
|
|
|
|
A. Chiti, S. Fanucchi, C. Sonnoli, S. Barni, and G. Orlandi Stroke Symptoms and the Decision to Call for an Ambulance: Turn on People's Minds! Stroke, July 1, 2007; 38(7): e58 - e59. [Full Text] [PDF]
|
|
|
|
|
|
S. M. Davis Community Stroke Education Using Mass Media: Past Results and Future Implications Stroke, July 1, 2007; 38(7): 2034 - 2035. [Full Text] [PDF]
|
|
|
|
|
|
H. P. Adams Jr, G. del Zoppo, M. J. Alberts, D. L. Bhatt, L. Brass, A. Furlan, R. L. Grubb, R. T. Higashida, E. C. Jauch, C. Kidwell, et al. Guidelines for the Early Management of Adults With Ischemic Stroke: A Guideline From the American Heart Association/American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Atherosclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: The American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists. Circulation, May 22, 2007; 115(20): e478 - e534. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. P. Adams Jr, G. del Zoppo, M. J. Alberts, D. L. Bhatt, L. Brass, A. Furlan, R. L. Grubb, R. T. Higashida, E. C. Jauch, C. Kidwell, et al. Guidelines for the Early Management of Adults With Ischemic Stroke: A Guideline From the American Heart Association/ American Stroke Association Stroke Council, Clinical Cardiology Council, Cardiovascular Radiology and Intervention Council, and the Atherosclerotic Peripheral Vascular Disease and Quality of Care Outcomes in Research Interdisciplinary Working Groups: The American Academy of Neurology affirms the value of this guideline as an educational tool for neurologists Stroke, May 1, 2007; 38(5): 1655 - 1711. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 E. C. Leira, J. K. Pary, P. H. Davis, K. J. Grimsman, and H. P. Adams Jr Slow Progressive Acceptance of Intravenous Thrombolysis for Patients With Stroke by Rural Primary Care Physicians Arch Neurol, April 1, 2007; 64(4): 518 - 521. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. J. Lindsberg, O. Happola, M. Kallela, L. Valanne, M. Kuisma, and M. Kaste Door to thrombolysis: ER reorganization and reduced delays to acute stroke treatment. Neurology, July 25, 2006; 67(2): 334 - 336. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. A. Yenari, L. Xu, X. N. Tang, Y. Qiao, and R. G. Giffard Microglia Potentiate Damage to Blood-Brain Barrier Constituents: Improvement by Minocycline In Vivo and In Vitro Stroke, April 1, 2006; 37(4): 1087 - 1093. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
S. C. Jones, A. Kharlamov, B. Yanovski, D. K. Kim, K. A. Easley, V. E. Yushmanov, S. K. Ziolko, and F. E. Boada Stroke Onset Time Using Sodium MRI in Rat Focal Cerebral Ischemia Stroke, March 1, 2006; 37(3): 883 - 888. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Part 9: Adult Stroke Circulation, December 13, 2005; 112(24_suppl): IV-111 - IV-120. [Full Text] [PDF]
|
|
|
|
|
|
M. A. Veazie, J. M. Galloway, D. Matson-Koffman, D. R. LaBarthe, J. N. Brownstein, M. Emr, E. Bolton, E. Freund Jr, R. Fulwood, J. Guyton-Krishnan, et al. Taking the Initiative: Implementing the American Heart Association Guide for Improving Cardiovascular Health at the Community Level: Healthy People 2010 Heart Disease and Stroke Partnership Community Guideline Implementation and Best Practices Workgroup Circulation, October 18, 2005; 112(16): 2538 - 2554. [Full Text] [PDF]
|
|
|
|
|
|
M. J. Alberts, R. E. Latchaw, W. R. Selman, T. Shephard, M. N. Hadley, L. M. Brass, W. Koroshetz, J. R. Marler, J. Booss, R. D. Zorowitz, et al. Recommendations for Comprehensive Stroke Centers: A Consensus Statement From the Brain Attack Coalition Stroke, July 1, 2005; 36(7): 1597 - 1616. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. W. Wojner-Alexandrov, A. V. Alexandrov, D. Rodriguez, D. Persse, and J. C. Grotta Houston Paramedic and Emergency Stroke Treatment and Outcomes Study (HoPSTO) Stroke, July 1, 2005; 36(7): 1512 - 1518. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N U Weir and A M Buchan A study of the workload and effectiveness of a comprehensive acute stroke service J. Neurol. Neurosurg. Psychiatry, June 1, 2005; 76(6): 863 - 865. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
Task Force Members, L. H. Schwamm, A. Pancioli, J. E. Acker III, L. B. Goldstein, R. D. Zorowitz, T. J. Shephard, P. Moyer, M. Gorman, S. C. Johnston, et al. Recommendations for the Establishment of Stroke Systems of Care: Recommendations From the American Stroke Association's Task Force on the Development of Stroke Systems Stroke, March 1, 2005; 36(3): 690 - 703. [Full Text] [PDF]
|
|
|
|
|
|
D. Kleindorfer, M. D. Hill, D. Woo, T. Tomsick, A. Pancioli, B. Kissela, A. M. Demchuk, D. Losiewicz, E. Jauch, A. Schneider, et al. A Description of Canadian and United States Physician Reimbursement for Thrombolytic Therapy Administration in Acute Ischemic Stroke Stroke, March 1, 2005; 36(3): 682 - 687. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. H. Schwamm, A. Pancioli, J. E. Acker III, L. B. Goldstein, R. D. Zorowitz, T. J. Shephard, P. Moyer, M. Gorman, S. C. Johnston, P. W. Duncan, et al. Recommendations for the Establishment of Stroke Systems of Care: Recommendations From the American Stroke Association's Task Force on the Development of Stroke Systems Circulation, March 1, 2005; 111(8): 1078 - 1091. [Full Text] [PDF]
|
|
|
|
|
|
California Acute Stroke Pilot Registry (CASPR) Inv Prioritizing interventions to improve rates of thrombolysis for ischemic stroke Neurology, February 22, 2005; 64(4): 654 - 659. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. C. Douglas, D. C. Tong, L. A. Gillum, S. Zhao, L. M. Brass, J. Dostal, and S. C. Johnston Do the Brain Attack Coalition's criteria for stroke centers improve care for ischemic stroke? Neurology, February 8, 2005; 64(3): 422 - 427. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Kwan, P. Hand, and P. Sandercock Improving the efficiency of delivery of thrombolysis for acute stroke: a systematic review QJM, May 1, 2004; 97(5): 273 - 279. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. B. Goldstein and P. Amarenco Prevention and Health Services Delivery Stroke, February 1, 2004; 35(2): 401 - 403. [Full Text] [PDF]
|
|
|
|
|
|
O. Camilo and L. B. Goldstein Statewide Assessment of Hospital-Based Stroke Prevention and Treatment Services in North Carolina: Changes Over the Last 5 Years Stroke, December 1, 2003; 34(12): 2945 - 2950. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. B. Morgenstern, L. K. Bartholomew, J. C. Grotta, L. Staub, M. King, and W. Chan Sustained Benefit of a Community and Professional Intervention to Increase Acute Stroke Therapy Arch Intern Med, October 13, 2003; 163(18): 2198 - 2202. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. E. Stahl, K. L. Furie, S. Gleason, and G. S. Gazelle Stroke: Effect of Implementing an Evaluation and Treatment Protocol Compliant with NINDS Recommendations Radiology, September 1, 2003; 228(3): 659 - 668. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
L. B. Goldstein Editorial Comment--Advertising Strategies to Increase the Public Knowledge of the Warning Signs of Stroke Stroke, August 1, 2003; 34(8): 1968 - 1969. [Full Text] [PDF]
|
|
|
|
|
|
M. Fisher Recommendations for Advancing Development of Acute Stroke Therapies: Stroke Therapy Academic Industry Roundtable 3 Stroke, June 1, 2003; 34(6): 1539 - 1546. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
K. Nedeltchev, M. Arnold, C. Brekenfeld, J. Isenegger, L. Remonda, G. Schroth, and H. P. Mattle Pre- and In-Hospital Delays From Stroke Onset to Intra-arterial Thrombolysis Stroke, May 1, 2003; 34(5): 1230 - 1234. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
C.S. Gray and J.E. O'Connell Stroke: beyond thrombolysis and back to basics QJM, March 1, 2003; 96(3): 179 - 181. [Full Text] [PDF]
|
|
|
|
|
|
S. L. Silliman, B. Quinn, V. Huggett, and J. G. Merino Use of a Field-to-Stroke Center Helicopter Transport Program to Extend Thrombolytic Therapy to Rural Residents Stroke, March 1, 2003; 34(3): 729 - 733. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Fisher and T. G. Brott Emerging Therapies for Acute Ischemic Stroke: New Therapies on Trial Stroke, February 1, 2003; 34(2): 359 - 361. [Full Text] [PDF]
|
|
|
|
|
|
R. C. Lisboa, B. D. Jovanovic, and M. J. Alberts Analysis of the Safety and Efficacy of Intra-Arterial Thrombolytic Therapy in Ischemic Stroke Stroke, December 1, 2002; 33(12): 2866 - 2871. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
H. J. Cloft, T. A. Tomsick, D. F. Kallmes, J. H. Goldstein, and J. J. Connors Assessment of the Interventional Neuroradiology Workforce in the United States: A Review of the Existing Data AJNR Am. J. Neuroradiol., November 1, 2002; 23(10): 1700 - 1705. [Full Text] [PDF]
|
|
|
|
 |
|
 J. Lenzer, C. Warlow, J. L Saver, C. S Kidwell, and S. Starkman Alteplase for stroke: money and optimistic claims buttress the "brain attack" campaign * Commentary: Who pays the guideline writers? * Commentary: Thrombolysis in stroke: it works! BMJ, March 23, 2002; 324(7339): 723 - 729. [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||