(Stroke. 1996;27:639-644.)
© 1996 American Heart Association, Inc.
Articles |
From the Departments of Neurology and Pharmacology and Clement J. Zablocki Veterans Affairs Medical Center (R.S.G.) and the Department of Family Medicine (A.J.H., C.E.G.), Medical College of Wisconsin, Milwaukee; and the Department of Neurology, the Department of Preventive Medicine and Environmental Health, and the Sanders Brown Center on Aging, University of Kentucky Medical Center, and the Neurology Service, VA Medical Center, Lexington, Ky (D.J.L.).
Correspondence to Arthur J. Hartz, MD, PhD, Family and Community Medicine, 8701 Watertown Plank Rd, Milwaukee, WI 53226.
| Abstract |
|---|
|
|
|---|
Methods An algorithm was developed by the authors to estimate the number of medically justified and unjustified hospital days for patients admitted with a primary diagnosis of ischemic stroke. Data for the algorithm were obtained from 177 stroke patients from an acute-care teaching hospital. The performance of the algorithm was evaluated on a subset of 46 patients by comparing the number of medically unjustified hospital days determined by the algorithm with the consensus determination of two neurologists.
Results The algorithm classified 68% of the 177 patients as having some unjustified hospital days and 41% of all hospital days as unjustified. With the neurologists as the gold standard, the sensitivity of the algorithm was .89 and the specificity was .91. The correlation between the number of unjustified days determined by the algorithm and the neurologists was .76.
Conclusions There is considerable unjustified length of stay for stroke patients. Physicians can develop simple clinical algorithms for detecting unjustified hospital stay in stroke patients that provide a reasonable approximation of complex clinical judgment.
Key Words: algorithms hospitalization quality of health care stroke management utilization review
| Introduction |
|---|
|
|
|---|
There are two approaches often used for careful monitoring of unjustified stay. One is to use risk-adjusted outcomes. This method classifies as unjustified a stay longer than expected for a patient with a given presentation.3 The other approach is to use clinical judgment to determine whether the stay is too long. The judgment may be implicit, ie, it depends entirely on a single clinician, or it may be explicit, ie, it depends entirely or in part on rules that have been derived by many clinicians. A comprehensive explicit review tool has been developed for LOS.4 Although procedures involving clinical judgment can discriminate between long and unjustified stays, they may be expensive if they require extensive data collection or the time of a clinical reviewer. The purpose of the present study was to develop and evaluate a computerized procedure that incorporates some of the philosophy of comprehensive clinical review systems but is less expensive to use. The procedure was developed for stroke patients, but similar procedures could be developed for other diseases and conditions.
| Subjects and Methods |
|---|
|
|
|---|
The nurse reviewers entered detailed information from the medical records into a computerized database. The types of information entered included diagnosis codes, procedures codes, demographic information, administrative information, vital signs, neurological findings on the physical examination (eg, level of consciousness, motor deficits, aphasia), activities of daily living (eg, mobility and feeding), medical history, laboratory studies, radiology results, interventions (eg, ICU, telemetry, all lines and tubes, medications, occupational therapy), complications (eg, pneumonia, deep vein thrombosis, pressure ulcers), and discharge disposition. All information that was entered on the patient was printed out on an abstract that facilitated the review of the medical record by the neurologist.
Description of the Algorithm
An algorithm was developed by the authors to estimate the number
of medically justified and unjustified hospital days for patients
admitted with a primary diagnosis of ischemic stroke. Although
it would have been better to use generally accepted guidelines for the
development of the algorithm, no such guidelines existed, and we used
our own clinical judgment to draft the algorithm. The algorithm was
revised after feedback from the neurological faculty at the Medical
College of Wisconsin and after the algorithm was tested on charts that
are not discussed in this report. The algorithm did not evaluate the
necessity of hospital admission, which has been addressed
elsewhere.5
The justifications for hospitalization for stroke patients included in
the algorithm are summarized in Table 1
. Since the first
3 hospital days usually involve close observation and
performance of various routine diagnostic tests,
they were always considered to be justified by the algorithm. Only the
remainder of the hospital stay was evaluated by the algorithm.
|
The justifications for hospital stay are shown in Table 1
. Ideally, the
justifications should only include the medical needs of the patient,
such as stroke progression (defined as a specific indication of stroke
progression in the chart or any deterioration in motor or mental
function) or impaired oral intake (including orders to give nothing
orally). However, an algorithm based only on patient characteristics
would be too complex to implement economically. For that reason most
justifications of hospital stay were physician decisions that depended
on the medical judgment of the physician treating the patient as well
as patient characteristics. These judgments included the need for the
patient to be in the ICU or to require procedures such as cardiac
telemetry or cerebral angiography. Since basic laboratory
investigations (eg, blood work,
electrocardiography, imaging studies, carotid
ultrasonography, or echocardiography) are routinely
completed in the first few days of hospitalization, specific allowance
was not made for these tests. A fewer number of additional days than
shown in Table 1
may be justified for some patients. However, we were
liberal in our allowances of the number of additional days to reduce
the false-positive rate for the algorithm.
The patient conditions and medical interventions listed in Table 1
not
only justify the hospital day on which they occur but also a number of
subsequent hospital days, as shown in the table. For example, stroke
progression justifies all hospital days while the stroke is progressing
and 3 additional days after the patient stabilizes. The patient's stay
is also justified for 3 days after discharge from the ICU, 2 days after
oral feeding was impaired, and 1 day after most of the other testing,
monitoring, and treatment procedures. For this retrospective review,
death automatically justified the entire LOS. Death was included as
part of the algorithm because it is a marker of severe disease that
might not be otherwise recognized by the algorithm.
Computerization of the stroke algorithm used CARS, a data application written by Kenneth Goldberg as part of his Howard Hughes fellowship in 1991 to 1992. The application was written in the Paradox Application Language6 and linked to the VP-Expert system7 for execution of the algorithm. With CARS, nurses who have little computer experience can create computerized algorithms and data entry screens.
Evaluation of the Algorithm
The reliability of the abstracted data was evaluated by
comparing algorithmic results on 38 randomly selected charts that were
abstracted by the two nurses who performed most of the data
abstraction. The reliability of the data for determining the presence
of any unjustified stay was measured by the
statistic, and the
reliability of the data for determining the number of unjustified
hospital days was measured by the Pearson correlation coefficient.
The performance of the algorithm was evaluated by comparing unjustified stay as determined by the algorithm with unjustified stay as determined by each of two board-certified neurologists (R.S.G., D.J.L.). Of the 177 charts abstracted for the computerized review, 50 were selected to be reviewed by the neurologists. Two of the selected charts could not be located, and two of the cases did not have a primary diagnosis of acute ischemic stroke. It is not known how many of the 129 unreviewed charts involved misdiagnoses. When reviewing the records, neither neurologist knew the results of the algorithm or the results of the review by the other neurologist. Each neurologist used his own judgment to determine the number of medically unjustified days that the patient stayed in the hospital. No explicit criteria were used for review. After independently reviewing each of the cases for unjustified stay, the neurologists discussed the cases in which they disagreed on the number of unjustified days to arrive at a consensus. This consensus was used as the gold standard for whether there was unjustified stay and for the number of unjustified days.
The two neurologists who performed the chart review were involved in the creation of the algorithm. Thus, the study tested the question of whether physicians could create an algorithm that reflected their own clinical judgment. The answer to this question was not evident a priori since clinical judgment requires a great deal of information that cannot be captured in a simple algorithm.
Based on clinical experience and a preliminary review of the charts, a stratified sample was used for chart review. The three strata were determined by length of hospital stay: 3 days or less, 3 to 15 days, and greater than 15 days. Charts with stays of 3 days or less were undersampled since almost all of these stays were entirely justified (five of five by physician review). Charts with stays of more than 15 days were also undersampled because almost all of these charts had unjustified LOS (12 of 13 by physician review). Since the physicians were only able to review a limited number of cases, we oversampled those cases that most required review because their status could not easily be determined by LOS alone. The reason for this type of sampling was to determine the effectiveness of the algorithm for the most difficult cases. For each year of hospitalization, the charts in a given strata were sampled consecutively by medical record number. Since requested charts were often not available on the first search, a high percentage of the charts were ordered.
The performance of the algorithm relative to the review of each neurologist was expressed in three ways: (1) The sensitivity of the algorithm was the percentage of cases determined by the neurologist to have any unjustified stay that were also found by the algorithm to have unjustified stay. (2) The specificity of the algorithm was the percentage of cases determined by the neurologist to have no unjustified stay that were also considered to have no unjustified stay by the algorithm. (3) The ability of the algorithm to determine the number of unjustified days was expressed as the correlation between the number of unjustified days determined by the algorithm and the number of unjustified days determined by the consensus of the neurologists. Because the sampling procedure selected cases that were likely to be most difficult for the algorithm, the performance of the algorithm was probably better in the complete data set than in the sample of reviewed cases.
| Results |
|---|
|
|
|---|
A characterization of patients abstracted for the study is shown in
Table 2
. The patients are divided into whether the
charts were reviewed by the neurologists. As expected for stroke, the
patients were generally elderly. A high percentage of the patients were
nonwhite, and less than 30% of the patients had private insurance. The
average LOS was 10.4 days for all patients, 11.1 days for the patients
whose charts were not reviewed, and 8.4 days for the charts of the
patients who were reviewed. The difference in LOS for the reviewed and
unreviewed charts occurred because of undersampling of charts with
extreme LOS, as described in "Subjects and Methods." The reviewed
charts with a positive algorithm (ie, indicating some unjustified stay)
had a shorter average LOS than the unreviewed charts because fewer
charts with very long stays were reviewed; the reviewed charts with a
negative algorithm (ie, indicating no unjustified stay) had a longer
average LOS than the unreviewed charts because fewer charts with stays
of 3 days or less were reviewed.
|
The distribution of the number of unjustified days as determined by the
algorithm is presented in Fig 1
. For 57 patients
(32%) there were no unjustified days (negative algorithm). For 67
patients (38%) there were between 1 and 4 unjustified days, and for
another 53 patients (30%) there were 5 or more unjustified days. For
the 57 patients with no unjustified stay, the average LOS was 5.6 days.
The average LOS for the remaining 120 patients was 12.6 days. Since the
average medically justified LOS was 6.4 days for these patients, the
average unjustified LOS was 6.2 days. For the entire group of 177
patients, 747 hospital days were judged by the algorithm to be
medically unjustified. This represented 41% of total
hospital days. Half of the unjustified days were accounted for by
patients with more than 8 unjustified days, although this group
included only 15% of the patients.
|
The distribution of the number of unjustified days differed for patients on the neurology service and those admitted to other services. Eighty-two percent of the patients were admitted to the neurology service. The average LOS for these patients was 9.6 compared with 14.1 for those with attending physicians on other services (P=.03). The average number of unjustified days for the stroke patients on the neurology service as determined by the algorithm was 3.5 compared with 7.7 for the patients with attending physicians on other services (P=.01).
The reasons for justified stay beyond 3 days are shown in Table 3
. Of the 146 patients who stayed in the hospital longer
than 3 days, 74% had at least 1 day that was justified by the
algorithm beyond the 3 initial days. Impaired oral intake was the most
commonly noted reason that justified some LOS greater than 3 days. The
use of the ICU and administration of heparin and other
intravenous medications were other frequently noted reasons
for extended stay, although they rarely justified the entire LOS.
|
For this retrospective review the algorithm considered death to justify the entire LOS. However, 85% of the days for the 10 subjects who died would have been justified by other components of the algorithm. This suggests that including death in the algorithm does not greatly affect the accuracy of the algorithm. An algorithm without death may provide an accurate method for concurrent review of hospitalized cases.
Evaluation of the Algorithm
For the 38 charts abstracted by two nurses, the determination by
the algorithm of whether or not the entire hospital stay was medically
justified was the same in 33 of the cases (
=.72,
P<.0001), indicating a moderate degree of interrater
agreement beyond chance. Disagreement in the 5 cases was attributed to
one abstraction error (failure to record the change from
intravenous to oral anticonvulsants), disagreement as to
the day that oral intake became adequate, and disagreement on whether
there was progression of the stroke (3 cases). The correlation between
the number of unjustified days determined by the algorithm from the
data of the two abstractors was .92 (P<.001).
For the 46 charts reviewed by both neurologists, there was agreement
concerning whether the entire hospital stay was medically justified in
38 cases (
=.59, P<.0001). Of the remaining 8 cases, each
neurologist rated 4 as justified, while the other neurologist rated
them with some unjustified stay. In 5 of these 8 cases the neurologist
who noted unjustified LOS noted only 1 day of unjustified stay. The
correlation between neurologists with respect to the number of
unjustified days was .78. After independent review, the cases were
discussed in an attempt to attain a consensus review. The neurologists
arrived at a consensus except for 2 cases. For each of these 2 cases,
one neurologist thought there was 1 unjustified day and the other
thought there were no unjustified days.
The agreement of the algorithm with each of the neurologists was
comparable to the agreement of the neurologists with each other (Table 4
). The results of the algorithm were also compared with
the consensus of the two neurologists. Of the 33 cases that had
unjustified stay by consensus, 29 had unjustified stay by the
algorithm, ie, the sensitivity was .89. Only 1 of the 11 cases that the
neurologists classified as having no unjustified stay had unjustified
stay by the algorithm, ie, the specificity was .91. The algorithm was
negative for both cases on which the neurologists disagreed.
|
The agreement between the algorithm and the consensus of the
neurologists for 44 cases is also shown in Fig 2
. In
approximately half of the reviews the algorithm and neurologists agreed
exactly on the number of medically justified hospital days (points
falling on the 45-degree line). The slight preponderance of points
falling below the line indicates that the neurologists judged fewer
hospital days medically justified than did the algorithm. The
correlation between the results of the algorithm and the consensus of
the two neurologists was .76 for the number of unjustified days. The
correlation was even greater for the number of justified days (.91)
because the justified days account for most of the stay.
|
The reasons for the unjustified hospital stay according to a consensus
of the neurologists are shown in Table 5
for the 33
patients whose charts were reviewed by the neurologists and who had
unjustified stay. Nearly half of the patients were awaiting placement
for rehabilitation (n=15), but other reasons, such as awaiting tests
(n=8) or consultations (n=5), were also important. Since some patients
had more than one reason for unjustified stay, more than 33 reasons are
listed.
|
| Discussion |
|---|
|
|
|---|
The results of this study strongly support the ability of physicians to generate a simple algorithm that is easily computerized and often agrees with complex medical judgments about the appropriate time for discharging a stroke patient. The level of agreement between the algorithm and the neurologists was about the same as that between the two neurologists. There may be less agreement between the algorithm and the judgments of other neurologists who do not believe that the components used in the present algorithm justify hospital stay. However, at a minimum the results show that physicians can generate an algorithm that reflects their own medical judgment. This was a surprising finding since clinical judgment is complex and must consider many aspects of patient care. Yet the clinicians agreed with the findings of this simple algorithm as often as they agreed with the judgment of each other.
The purpose of the algorithm was to address unjustified LOS due to inefficiencies in providing services or discharging the patient. The algorithm was not designed to address other issues: (1) unnecessary hospitalization, (2) unnecessary LOS caused by errors in the quality of care (eg, errors causing stroke progression or additional ICU stay), and (3) unnecessary use of resources (eg, admission to the ICU) that obscures unnecessary LOS. Although the ability to address these issues would be useful, it would add to the complexity of an algorithm designed to monitor LOS.
The algorithm can be used in paper form as a screening tool for unjustified LOS. Cases found by the algorithm to have unjustified stay can then be reviewed by physicians to confirm that there was a problem. The algorithm will be much more useful if it is computerized, however, for the following reasons: (1) electronically stored data can be entered into the computer and may substantially reduce the time of data abstraction and calculations of unjustified stay, and (2) with computerized data it is much easier to obtain clinical practice profiles of the computerized algorithm results. Profiling of the algorithm results may be useful to suggest causes of delays (eg, practices of specific physicians or differences in care on weekends) and to monitor the effect of changes in care (eg, adding more social workers). A secondary benefit of obtaining profiles is that it may be useful to profile components of the algorithm. For example, the components include the number of days in the ICU, on telemetry, or on intravenous medication. Profiling these components will be useful to monitor resource utilization. It will also be possible with these profiles to determine whether the rates of unjustified stay may be underestimated because the use of certain resources is high.
Based on the results of the computerized algorithm, we have instituted clinical guidelines at Froedtert Hospital to help reduce LOS. The algorithm will be used to monitor the effect of these guidelines. Previous studies have found that the institution of clinical protocols can reduce hospital LOS and associated charges.9 10 11 Although the average LOS for stroke patients has significantly decreased over the past few decades, large regional differences persist. These differences are apparently due to widespread variation in patient management.12
Other methods have been used to obtain practice profiles for LOS. The least expensive and most commonly used is to compare average LOS at the index hospital with a benchmark LOS that is usually determined from other hospitals. An LOS the same or less than the benchmark would be reassuring; a longer LOS would suggest that the hospital is wasting resources. For the following reasons, however, the comparison with a benchmark may not be useful: (1) Patients at the benchmark hospitals may have different LOS than the index hospital because they differ with respect to severity of illness. This problem can be reduced by the use of methods such as the Computerized Severity Index3 to adjust for severity of illness. (2) The benchmark may not be ideal; ie, it may also have many cases with unjustified LOS. (3) Unjustified LOS at the index hospital may be partially masked by cases that are discharged prematurely. Reduction in the LOS with premature discharge may become more of a problem as pressures to reduce LOS increase.
In contrast to benchmarking, practice profiles derived from a computerized algorithm avoid these disadvantages of benchmarking since the algorithm evaluates the course of disease and treatment instead of only LOS.
The Appropriateness Evaluation Protocol4 is similar to the algorithm described in this study in that it makes use of clinical criteria to identify cases with unjustified hospital stay. With this protocol each hospital day is reviewed by explicit criteria to determine whether hospitalization for that day is medically justified. The protocol differs from the current approach in two important ways: (1) Since it is generic (ie, it uses the same criteria for all cases regardless of the patient's diagnosis), it requires the abstraction of more data elements and does not deal with the specific issues required in decision making for stroke patients. (2) Since it is not computerized and allows the nurse reviewers to use their own judgment for overriding the criteria, it is more subjective and may be more variable than the computerized algorithm described in this report. The computerized algorithm will have more advantages as more data elements become computerized and the time for data abstraction is reduced or eliminated.
The use of computerized algorithms for reviewing care has recently been introduced as a method for monitoring health care. The first use of the method was the UCDSS, which was developed by the HCFA.13 14 15 The initial purpose of the UCDSS was to computerize generic algorithms that had been used by nurses for reviewing quality of care and the necessity of hospital admission for individual cases. Because of the low specificity of the algorithms in the first version of UCDSS and a shift toward profiling care rather than identifying errors in individual cases, HCFA has largely abandoned the UCDSS as originally conceived.14 There are several fundamental differences, however, between the algorithm tested in this study and those in the original UCDSS: (1) The approach in this study is disease specific rather than the generic approach first used by HCFA. (2) The algorithm focuses on resource utilization after hospital admission rather than the need for admission and quality of care. (3) The algorithm was developed locally for a specific hospital rather than nationally for all hospitals. For these reasons the approach to algorithm development in this study may be more successful than that first tried by HCFA.
The use of computerized clinical algorithms to monitor health care is in the early stages of development. There has been little work defining the role of algorithms in monitoring care and even less in developing specific algorithms. As extensive electronic clinical data bases become more available, the cost of using algorithms will be greatly reduced and experimentation with implementing algorithms is likely to increase. The results presented in this report suggest that algorithms can be valuable in monitoring LOS for stroke patients. Since assessment of the LOS for stroke patients is as complex as the assessment of many other aspects of medical care, it is likely that computerized algorithms to evaluate aspects of care for other diseases or conditions may also perform well.
| Selected Abbreviations and Acronyms |
|---|
|
| Acknowledgments |
|---|
Received November 13, 1995; revision received January 8, 1996; accepted January 8, 1996.
| References |
|---|
|
|
|---|
2. Myerson AR. Helping health insurers say no. New York Times. March 20, 1995:C1.
3. Horn SD, Sharkey PD, Buckle JM, Backofen JE, Averill RF, Horn RA. The relationship between severity of illness and hospital length of stay and mortality. Med Care. 1991;27:305-317.
4. Gertman PM, Restuccia JD. The Appropriateness Evaluation Protocol: a technique for assessing unnecessary days of hospital care. Med Care. 1981;19:855-871. [Medline] [Order article via Infotrieve]
5.
Lanska DJ. Review criteria for hospital
utilization for patients with cerebrovascular disease.
Neurology. 1994;44:1531-1532.
6. PAL Programmer's Guide. Scotts Valley, Calif: Borland International, Inc; 1992.
7. Rule Bases Expert System Development Tool VP Expert. Paperback Software International; 1989.
8. Schluep M, Bogousslavsky J, Regli F, Tendon M, Prod'hom LS, Kleiber C. Justification of hospital days and epidemiology of discharge delays in a Department of Neurology. Neuroepidemiology. 1994;13:40-49. [Medline] [Order article via Infotrieve]
9. Wachtel T, Moulton AW, Pezzulo J, Hamolsky M. Inpatient management protocols to reduce health care costs. Med Decis Making. 1986;6:101-109.
10.
Odderson IR, McKenna BS. A model for management
of patients with stroke during the acute phase.
Stroke. 1993;24:1823-1827.
11.
Bowen J, Yaste C. Effect of a stroke protocol on
hospital costs of stroke patients. Neurology. 1994;44:1961-1964.
12. Lanska DJ. Length of hospital stay for cerebrovascular disease in the United States. J Neurol Sci. 1994;127:214-220. [Medline] [Order article via Infotrieve]
13.
Audet AM, Scott HD. The UCDS: an evaluation of
the proposed national database for Medicare's quality review
program. Ann Intern Med. 1993;119:1209-1213.
14.
Lanska DJ. Medicare hospital utilization review
for ischemic cerebrovascular disease.
Neurology. 1993;43:650-654.
15. Lanska D. A public/private partnership in the quest for quality: development of cerebrovascular disease practice guidelines and review criteria. Am J Med Qual. 1995;10:100-106.
This article has been cited by other articles:
![]() |
R. G. Holloway, B. G. Vickrey, C. Benesch, J. A. Hinchey, J. Bieber, M. D. Hill, and C. Zahn Development of Performance Measures for Acute Ischemic Stroke Editorial Comment Stroke, September 1, 2001; 32(9): 2058 - 2074. [Abstract] [Full Text] [PDF] |
||||
![]() |
N. E. Mayo, S. Wood-Dauphinee, D. Gayton, and S. C. Scott Nonmedical Bed-Days for Stroke Patients Admitted to Acute-Care Hospitals in Montreal, Canada Stroke, March 1, 1997; 28(3): 543 - 549. [Abstract] [Full Text] |
||||
![]() |
A. van Straten, J.H.P. van der Meulen, G.A.M. van den Bos, and M. Limburg Length of Hospital Stay and Discharge Delays in Stroke Patients Stroke, January 1, 1997; 28(1): 137 - 140. [Abstract] [Full Text] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 1996 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |