Relationship Between Provider Volume and Mortality for Carotid Endarterectomies in New York State
Background and Purpose—The objective of this study was to assess the relationship between each of 2 provider volume measures for carotid endarterectomies (CEs) (annual hospital volume and annual surgeon volume) and in-hospital mortality. New York’s Statewide Planning and Research (SPARCS) administrative database was used to identify all 28 207 patients for whom carotid endarterectomy was the principal procedure performed in New York State hospitals between January 1, 1990, and December 31, 1995.
Methods—A statistical model was developed to predict in-hospital mortality using age, admission status, and several conditions found to be associated with higher-than-average mortality. This model was then used to calculate risk-adjusted mortality rates for various intersections of hospital and surgeon volume ranges.
Results—Risk-adjusted in-hospital mortality ranged from 1.96% (95%confidence interval, 1.47 to 2.57) for patients having surgeons with annual CE volumes of <5 in hospitals with annual CE volumes of ≤100 to 0.94% (95% confidence interval, 0.73 to 1.19) for patients having surgeons with annual volumes of ≥5 in hospitals with annual CE volumes of >100. These 2 rates were statistically different.
Conclusions—We conclude that the in-hospital mortality rates for carotid endarterectomies performed by surgeons with extremely low annual volumes (<5) and for hospitals with low volumes (≤100) are significantly higher than the in-hospital rates of higher-volume surgeons and hospitals, even after taking preprocedural patient severity of illness into account.
Since the late 1970s, numerous studies have appeared in medical and health services research journals documenting the relationship between the outcomes of care for patients either with specific diagnoses or undergoing specific procedures and the number of patients of that type who were treated in a given hospital or by a given physician/surgeon.1 2 3 4 5 6 7 8 9 10 11 12 13 A wide variety of procedures (operative and diagnostic) and medical conditions have been studied in this regard, and many of them have been shown to yield better outcomes when the responsible physician or surgeon has had more experience treating that type of patient. These include such medical conditions as myocardial infarction, perinatal illness, and respiratory infection. However, most volume-outcome relationships discovered have been for surgical procedures, with the operations ranging from relatively risk-free operations such as appendectomies and hysterectomies to higher-risk procedures such as abdominal aortic aneurysm surgery and lung resection.
The purpose of this study was to examine the relationship between in-hospital mortality for CEs and 2 provider volume measures for CE—the annual hospital volume and the annual surgeon volume—with use of a large, population-based database. This was accomplished while controlling for differences in patient severity of illness through use of demographic variables, patient diagnoses, and other measures of severity, such as whether or not the admission was elective.
Subjects and Methods
The database used for the study was New York State’s acute-care discharge database, the Statewide Planning and Resource Cooperative System (SPARCS). SPARCS, which was created by the New York State Department of Health (NYSDOH) for reimbursement and planning purposes, contains automated discharge data for each discharge from nonfederal acute care facilities in the state. This includes information relating to the patient’s disposition, age, gender, race, admission status, physician and hospital identifiers, principal diagnosis, secondary diagnoses, principal procedure, and secondary procedures. Diagnoses and procedures are coded using the International Classification of Disease, Ninth Revision, Clinical Modification (ICD-9-CM) classification system. Data in the system are abstracted from medical records by trained medical records personnel in each hospital, and the NYSDOH is responsible for verifying the accuracy of reported information.
The time frame of the study was 1990 through 1995. All patient discharges with CE coded as the principal procedure were included in the study (a total of 28 207 cases). Thus, patients undergoing procedures in the same admission with higher mortality rates than CE (eg, coronary artery bypass graft surgery) were excluded from the study. Annual hospital and surgeon volumes for CE were calculated as the number of times the hospital and surgeon, respectively, performed the procedure in the same calendar year.
For each year included in the study, the number of CEs performed in New York, the number of surgeons and hospitals performing the procedure, and the crude (observed) mortality rate were calculated so as to better understand trends in the performance of CEs.
Each discrete patient demographic characteristic (gender, race) was examined to determine its prevalence rate and mortality rate. For continuous variables (age, number of procedures per hospital per year, and number of procedures per surgeon per year), means and SDs were calculated. Prevalence rates and mortality rates were also calculated by diagnosis. Because principal and secondary diagnoses could be interchanged depending on the coder and hospital, it was decided to treat all diagnoses the same by computing prevalence rates for diagnoses without regard to whether the diagnosis was coded as a principal or secondary diagnosis.
The percentages of CEs associated with different surgeon volume and hospital volume ranges and the observed mortality rates for each of these volume ranges were computed. Volume ranges were chosen so that the mortality rates were relatively homogeneous within each range and so that no range contained an exceptionally high or low percentage of all cases. The purpose of this process was to get a sense of whether there were significant provider volume-mortality relationships without adjusting for differences in patient severity of illness.
A stepwise logistic regression model (with P<0.05) was developed to predict mortality with use of all available patient risk factors. The dependent variable in the logistic regression analysis was a binary variable that denoted whether the patient had died in the hospital or had been discharged alive. Independent variables that were candidates for the logistic regression model were age, gender, race, admission status (elective, nonelective), and relevant diagnoses.
Linear and quadratic functions of age were simultaneously tested in the model. A diagnosis was chosen for consideration in the final model if its prevalence rate was at least 1% and the mortality rate for all patients with that diagnosis was at least 1.5 times the mortality rate for all patients (1.19%) with CE as the principal procedure. Another criterion for inclusion was that the diagnosis had to be regarded as a comorbidity rather than a complication of care. Any diagnosis with a reasonably high probability of being a complication of care (eg, fluid/electrolyte imbalance, cerebral artery occlusion, and cerebral infarction) was not considered for inclusion in the regression analyses. Diagnoses such as atrial fibrillation, which could potentially occur as a complication of surgery but is more likely to be a comorbidity, were considered in the regression analyses. For categorical variables with more than 2 categories (eg, elective, urgent, and emergency admissions), the reference category used was that with the lowest mortality rate (eg, elective admissions). If more than 1 category proved to be significantly different from the reference category (ie, entered the stepwise logistic regression model), the categories were combined if their model coefficients were not significantly different from one another.
The next step in the analysis was to use the results of the logistic regression model to compute risk-adjusted mortality rates for different ranges of hospital and surgeon volume. First, 2 ranges were chosen for each of the volume measures so as to maximize the difference in the mortality rates of the 2 groups while maintaining a reasonably large number of cases in each group.
The expected mortality rate was then computed for each of the 4 intersections of groups (eg, all patients undergoing surgery performed by surgeons with annual volumes of <5 CEs in hospitals with annual volumes of ≤100 CEs). The expected mortality rate for a group was obtained by summing the logistic regression model’s predicted probabilities of death for each patient in the group and dividing by the number of patients in the group. The observed mortality rate for the group was then obtained by dividing the number of patients in the group who died before discharge by the total number in the group. The risk-adjusted mortality rate for the group was calculated by dividing its observed mortality rate by its expected mortality rate and then multiplying this quotient by the 6-year statewide mortality rate of 1.19%. Confidence intervals (CIs) were then calculated for the risk-adjusted mortality rates of the groups.
Table 1⇓ presents the annual volume and mortality rate for CEs performed in New York in 1990–1995 as the principal procedure, the number of surgeons performing the procedure in each year, and the number of hospitals in which the procedures were performed in each year. During this time, the volume has risen from 2835 in 1990 to 7823 in 1995, an increase of 176% in 5 years. However, in some 2-year periods (eg, 1992–1993) the increase was negligible, whereas in other 2-year periods there was a very large increase in volume (eg, 1994–1995, with an increase of 48%).
The number of surgeons performing the procedure annually increased from 406 in 1990 to 518 in 1995, an increase of 28%. The annual number of hospitals in which the procedure was performed was more stable, ranging from 145 in 1990 to 161 in 1995 (out of a total of approximately 250 acute care hospitals in the state).
The mortality rate has been surprisingly uneven, with a low of 0.85% in 1992, a high of 1.46% in 1993, and similar rates at the beginning and end of the 6-year period (1.23% and 1.21%, respectively).
Table 2⇓ presents the prevalence and mortality rates for CE patients in New York in 1990–1995 for 2 demographic categories (gender and race), for nonelective admissions, and for diagnoses judged to be comorbidities rather than complications that have prevalence rates exceeding 1% and relatively high mortality rates (>1.5 [1.19%]=1.79%).
The mortality rates for female gender and for white race are not significantly different from those for males and nonwhites, respectively. However, nonelective patients have a mortality rate of 2.28%, which is significantly higher than the rate for elective patients (P<0.05). These patients include “emergency” patients, who are defined as needing immediate medical intervention as a result of a severe, life-threatening condition, and “urgent” patients, who require immediate attention for the treatment of a physical disorder and were generally admitted to the first available accommodation.
Four diagnosis codes are associated with significantly higher mortality rates than the rates for patients without those diagnoses. Patients with aortic valve disorders, mitral valve disorders, atrial fibrillation, and congestive heart failure all had mortality rates significantly higher than patients without those conditions.
Table 2⇑ also presents the means and SDs for patient age, annual hospital volume, and annual surgeon volume. The average age was 69.7 years, with an SD of 8.6 years. The average annual surgeon volume was 8.8, with a median volume of 3. The average hospital volume was 33.0, with a median of 16.
Table 3⇓ presents odds ratios (ORs) and P values for risk factors that proved to be significantly related to in-hospital mortality using a stepwise logistic regression model with P=0.10 as the cutoff for including independent variables in the model. Significant patient risk factors in the model were age (OR=1.03 for each 1-year increase), nonelective admissions (OR=2.43), congestive heart failure (OR=4.16), atrial fibrillation (OR=2.02), mitral valve disorder (OR=1.81), and aortic valve disorder (OR=2.61). The variables used as candidates for inclusion in the model are presented in Table 2⇑. Note that gender, race, and diabetes did not prove to be significantly related to mortality in a multivariate model (diabetes was not present in Table 2⇑ because patients with diabetes did not have a significantly higher bivariate mortality rate than those without diabetes).
Table 4⇓ presents, for 6 different surgeon volume ranges, the percentage of all operations performed by surgeons with those annual volumes and the crude mortality rate of patients undergoing CEs performed by those surgeons. The table indicates that nearly 10% of all procedures were performed by surgeons with annual volumes of <5. Another 12% of all procedures were performed by surgeons with annual volumes of between 5 and 9 procedures. Twenty-five percent of all procedures were performed by surgeons with annual volumes of ≥50 procedures.
Table 4⇑ also demonstrates that the crude (observed) in-hospital mortality rate is sensitive to annual surgeon volume. The risk-adjusted mortality rate of 1.89% for patients undergoing surgery by surgeons with annual volumes of fewer than 5 procedures was significantly higher than the statewide mortality rate (P<0.05), and no other volume group had a risk-adjusted mortality rate that was statistically different from the statewide rate. The risk-adjusted mortality rate was 1.39% for patients with surgeons having annual volumes of 5 to 9 procedures and 1.16% for patients of surgeons with annual volumes between 10 and 14 procedures. The surgeon volume group with the lowest risk-adjusted mortality rate was the group with 25 to 49 procedures per year, with a rate of 1.02%. The risk-adjusted rate rose slightly to 1.08% for surgeons with annual volumes of ≥50 per year.
Table 5⇓ presents the same type of information for hospital volume ranges as presented in Table 4⇑ for surgeons. The same criteria as in Table 4⇑ were used for choosing volume ranges. As indicated in the table, approximately 5% of all procedures were performed in hospitals with annual volumes of <10, and another 8% were performed in hospitals with annual volumes of between 10 and 19. About 30% of all procedures were performed in hospitals with annual volumes of ≥100.
With regard to risk-adjusted mortality rates for the different hospital volume groups, a somewhat uneven pattern of rates occurred for the first 4 volume groups, with the highest rate (1.42%) associated with annual hospital volumes between 10 and 19 and between 50 and 99 procedures per year. The risk-adjusted mortality rate for the latter group was significantly higher than the statewide rate. However, the lowest risk-adjusted rates were experienced by patients in the fifth and highest volume group (≥100), with a risk-adjusted mortality rate of 0.94%. This rate was significantly lower than the statewide mortality rate.
Table 6⇓ presents observed and risk-adjusted mortality rates, along with volumes and CIs for each of 4 intersections of hospital volume/surgeon volume ranges. As indicated, the maximum risk-adjusted mortality rate of 1.96% is for patients undergoing surgery performed by surgeons with annual CE volumes of <5 in hospitals with annual CE volumes of ≤100. The minimum risk-adjusted mortality rate is for patients in the higher-volume surgeon group and the higher-volume hospital group (0.94% for patients undergoing surgery performed by surgeons with annual CE volumes of ≥5 in hospitals with annual CE volumes >100). Both of these risk-adjusted rates are significantly different from the statewide rate of 1.19% for the 6-year period, since their CIs do not include the statewide rate (the upper limit on the latter rate is slightly lower than the statewide rate, although they are identical to 2 decimal places).
It can also be seen from the last column that the risk-adjusted mortality rate for patients undergoing surgery performed by surgeons with annual volumes of <5 have significantly higher risk-adjusted mortality rates (1.89%) than patients undergoing surgery performed by surgeons with annual volumes of ≥5 (1.11%), because their CIs do not overlap. Also, the former group has a significantly higher rate than the statewide rate because its CI does not include the statewide rate. With respect to hospital volume groups, the totals row indicates that patients undergoing surgery in hospitals with CE volumes of ≤100 have higher (but not significantly higher at the 95% level) mortality rates than patients undergoing surgery in hospitals with CE volumes of ≥101. However, these 2 groups do have significantly different risk-adjusted mortality rates when 90% CIs are used.
It should also be noted that the number of hospitals with an annual CE volume of ≤100 procedures ranged from a minimum of 141 in 1995 to a maximum of 153 in 1991, whereas the number of hospitals with an annual CE volume of ≥101 ranged from a minimum of 3 in 1990 to a maximum of 20 in 1995. The number of surgeons with an annual CE volume of <5 CE procedures ranged from a minimum of 223 in 1992 to a maximum of 257 in 1990, whereas the number of surgeons with an annual volume of ≥5 procedures ranged from a minimum of 149 in 1990 to a maximum of 288 in 1995.
Carotid endarterectomy was first performed in the early 1950s based on conceptualization by C. Miller Fisher that carotid bifurcation atherosclerosis causing either “downstream” embolization or hypoperfusion was a significant cause of stroke. The first successful carotid endarterectomy was performed by DeBakey in 1953; over the ensuing 44 years, the operation has experienced periods of great popularity and other periods during which the appropriateness of the procedure was questioned. In 1971, an estimated 15 000 carotid endarterectomies were performed in the United States. The procedure reached its highest volume in 1984, with 100 000 operations being performed nationwide.10
Then, reports that were relatively isolated as well as throughout the country14 15 16 documented unacceptably high perioperative morbidity and mortality. In addition, a study17 demonstrated that another procedure to reduce the risk of stroke, the extracranial-intracranial bypass, did not protect against a stroke in comparison with medical management. Together, this news resulted in a period of careful analysis of the efficacy of CE that was accompanied by a decline to only 83 000 procedures being performed in 1986.
In the early 1990s, the results of the North American Symptomatic Carotid Endarterectomy Trial (NASCET)18 and the Asymptomatic Carotid Atherosclerosis Study (ACAS)19 indicated the value of CEs in certain categories of symptomatic and asymptomatic patients, and this led to a resurgence in the number of procedures being performed. In 1994, the rate of carotid endarterectomy among persons ≥65 years of age rose to 24.4 per 10 000, with 86% of this increase occurring between 1991 and 1994.20
This study has demonstrated that, on average, patients undergoing CEs performed by surgeons with annual CE volumes of ≥5 in hospitals with annual volumes of >100 have significantly lower risk-adjusted mortality rates (0.94%; 95% CI, 0.73 to 1.19) than patients undergoing CEs performed by surgeons with extremely low (<five procedures per year) in lower-volume hospitals (1.96%; 95% CI, 1.47 to 2.57). Also, although there was not sufficient statistical power to establish statistical significance, findings from the study suggest that both hospital volume and surgeon volume individually contribute to lower risk-adjusted mortality rates. That is, patients with higher-volume surgeons who undergo surgery in low-volume hospitals or patients with low-volume surgeons who undergo surgery in higher-volume hospitals have, on average, lower risk-adjusted mortality rates than patients with lower-volume surgeons who undergo surgery in lower-volume hospitals. It should also be noted that surgeons with annual volumes of ≥50 had much lower patient mortality rates than surgeons with annual volumes of <50; however, a cut at 50 was not used in the analyses that examined intersections of hospital and surgeon volumes because the sample size was not sufficiently large. In summary, the results suggest that low-volume surgeons and hospitals, particularly, should track outcomes and demonstrate good results if they are to continue to perform CEs.
Findings from previous studies that have examined volume-outcome effects for CEs are mixed. Using 1979–1988 hospital discharge data for 11 199 patients undergoing a CE in a single (unnamed) state, Edwards et al21 compared actual (unadjusted) mortality and stroke rates for 3 hospital volume groups (1 to 12/y, 13 to 49/y, ≥50/y) and 3 surgeon volume groups (with the same ranges). Significant surgeon volume effects were found for both mortality and perioperative stroke; no significant hospital volume effects were found. It should be noted that no risk adjustments were made to account for possible differences in case mix among the different volume groups.
Segal et al11 used 26 months of Pennsylvania data (5657 patients undergoing CEs) between December 1989 and January 1992 to compute observed in-hospital mortality rates for surgeons performing <30 procedures in the time period and surgeons performing ≥30 procedures. The lower-volume surgeons were found to have significantly higher mortality rates (2.6%) than the higher-volume surgeons (1.2%); again, the data were not risk-adjusted for preoperative severity of illness of the patients.
Brook et al22 surveyed medical records from 1302 CE patients ≥65 years old from three geographic areas in the United States in 1981. The impact of surgeon volume (treated as a continuous variable) on mortality and postoperative stroke or heart attack was tested while controlling for the effect of patient severity, race, income, gender, and various hospital characteristics. No significant volume effects were found for any of the adverse outcomes studied.
Ruby et al10 analyzed data from 3997 CE patients from Connecticut to determine the relationship between surgeons’ operative volume and specialty, and morbidity and mortality. The authors concluded that surgeons with an average annual volume of ≤1 CE were 2.5 times as likely to have a poor postoperative outcome (stroke and/or death) as surgeons who performed ≥10 CEs per year. No risk-adjusted rates were reported.
Another study that involved the specialty of neurosurgery, by Solomon et al,12 demonstrated that the in-hospital mortality rate for New York State patients undergoing craniotomies for cerebral aneurysms between 1987 and 1993 was significantly inversely related to the annual volume of these procedures in the hospitals in which they were performed.
This study differs from earlier studies in that it had a much larger sample size. Also, 3 of the 4 previous studies did not examine the hospital volume-mortality relationship, and 3 of the studies (not the same 3) did not risk-adjust the mortality rates by controlling for other patient risk factors. Thus, none of the other studies simultaneously examined the relationship between mortality and surgeon/hospital volume measures while controlling for the impact of patient risk factors.
A drawback of the current study is that postoperative stroke is another important adverse outcome in addition to in-hospital mortality. Unfortunately, complications of care such as stroke are not always reported accurately or completely in administrative databases such as SPARCS, and they are sometimes difficult to distinguish from preoperative occurrences. The low prevalence rates in SPARCS for strokes among CE patients led us to conclude that it would be unwise to trust the accuracy of reporting for postoperative strokes. This is a serious limitation because in addition to being an important adverse event in its own right, postoperative stroke is a cause for mortality. Thus, postoperative stroke must also be studied to provide a complete accounting of the success of CE.
Another limitation of SPARCS is that it does not contain important clinical data such as the degree of carotid artery stenosis. Because information on carotid artery stenosis was not available, it was impossible to determine whether each CE was appropriate or whether, based on earlier studies, the patient was either too high-risk for surgery or not in sufficient danger of a future stroke to warrant surgery.15 18 19 Ideally, these 2 categories of inappropriate patients would be removed from the analyses before testing for a volume-mortality relationship. If low-volume hospitals had a much higher percentage of high-risk, inappropriate patients or high-volume hospitals had a much higher percentage of low-risk, inappropriate patients, this could have unfairly biased the analysis. Obviously, inappropriate surgery is a quality problem also, but it should be treated separately rather than combining inappropriate with appropriate patients in the volume-mortality analyses. However, it should be pointed out that none of the references summarized above had access to carotid artery stenosis. In summarizing the 2 limitations just discussed, SPARCS is limited as a tool for exploring volume-outcome relationships in depth because it necessitates the use of retrospective analysis of data that were not generated expressly for investigating the hypotheses of interest.
It should also be mentioned that there are 2 possible causal explanations for volume-outcome relationships. One is “practice makes perfect,” whereby providers with more experience achieve better results. Another is the “selective referral hypothesis,” whereby patients gravitate or are sent to providers who have better outcomes.8 9 If the former hypothesis is correct, low-volume providers who increase their volumes could expect improved outcomes; if the latter explanation is correct, low-volume surgeons with poor results who attempt to perform more procedures to achieve a volume limit would not necessarily improve their outcomes. The ideal way of testing these alternative hypotheses is to use longitudinal data in an environment in which individual provider outcomes have exhibited considerable volume variations over time. Unfortunately, this type of information is rarely available.
A final caveat is that although the findings of the study are that higher-volume hospitals and higher-volume surgeons have lower risk-adjusted mortality rates, these findings are based on combining the results of numerous low-volume providers (hospitals and surgeons) and comparing them with the results of numerous high-volume providers. There are individual providers who are exceptions to this rule, but an accurate assessment of individual provider performance should not be undertaken without high-quality clinical data whose accuracy and completeness have been verified. For assessing the performance of low-volume providers, it may also be necessary to aggregate several years of data to obtain precise estimates.
The authors wish to acknowledge the support of the H. Schaffer Foundation and the New York State Department of Health, New York State Task Force on Clinical Guidelines and Medical Technology Assessment.
Reprint requests to A. John Popp, MD, Professor and Chairman, Department of Surgery, Albany Medical College, Albany, NY 12208.
- Received April 28, 1998.
- Revision received July 14, 1998.
- Accepted July 28, 1998.
- Copyright © 1998 by American Heart Association
Hannan EL, Siu AL, Kumar D, Kilburn H Jr, Chassin MR. Explaining declining CABG surgery mortality in New York: the role of surgeon volume. JAMA. 1995;273:209–213.
Luft HS. The relationship between surgical volume and mortality: an exploration of causal factors and alternative models. Med Care. 1980;9:940–959.
Luft HS, Garnick DW, Mark DH, McPhee SJ. Hospital Volume, Physician Volume, and Patient Outcomes. Ann Arbor, Mich: Health Administration Press; 1990.
Segal HE, Rummel L, Wu B. The utility of PRO data on surgical volume: the example of carotid endarterectomy. Quality Research Bulletin. May 1993:152–157.
Solomon RA, Mayer SA, Tarmey JJ. Relationship between the volume of craniotomies for cerebral aneurysm performed at New York State hospitals and in-hospital mortality. Stroke. 1996;27:13–17.
Williams RL. Measuring the effectiveness of perinatal care. Med Care. 1979;2:95–109.
Dyken ML, Pokras R. The performance of endarterectomy for disease of the extracranial arteries of the head. Stroke. 1984;15:948–950.
Easton JD, Sherman DG. Stroke and mortality data in carotid endarterectomy: 228 consecutive patients. Stroke. 1977;8:565–568.
EJ Graves, BS Gillum. Detailed diagnoses and procedures, National Hospital Discharge Survey, 1994. Vital Health Stat 13. No. 127; 1997. DHHS Publication (PHS) 97–1778.
Brook RH, Park RE, Chassin MR, Kosecoff J, Keesey J, Solomon DH. Carotid endarterectomy for elderly patients: predicting complications. Ann Intern Med. 1990;113:747–753.