Improving the Appropriateness of Carotid Endarterectomy
Results of a Prospective City-Wide Study
Background and Purpose—In light of previously reported concerns regarding carotid endarterectomy (CEA) use in our city, our goal was to determine the influence of a prospective audit and educational campaign on the performance of CEA with respect to surgical appropriateness and complication frequency.
Methods—Results of our previous audit of 291 CEAs, along with CEA practice guidelines and notification of prospective surveillance, were supplied to surgeons performing CEA in our city. After this, 184 consecutive patients undergoing CEA from September 1996 to August 1997 were followed prospectively. On the basis of blinded standardized remeasurements of angiographic carotid stenoses, CEA was classified as appropriate for patients with symptomatic carotid stenoses ≥70%, uncertain for those with symptomatic stenoses <70% or asymptomatic stenoses ≥60%, and inappropriate for patients with asymptomatic carotid stenoses <60% or preoperative neurological or medical instability.
Results—Forty percent of patients were asymptomatic. Compared with our prior audit, the rate of appropriate CEAs improved from 33% previously to 49% of cases in the present study (P=0.0005), uncertain indications did not change significantly (49% versus 47%; P=0.61), and inappropriate indications dropped from 18% to 4% (P=0.00002). Perioperative stroke or death occurred in 6.4% of symptomatic patients but developed in only 2.7% of asymptomatic patients, which was improved from the 5.1% rate previously found.
Conclusions—In our city, the use of a surgical audit identified areas of concern regarding CEA, and subsequent education and ongoing surveillance significantly improved the use and performance of this procedure.
Carotid endarterectomy (CEA) is a surgical procedure well suited for examination of its appropriateness and associated complications. Indications for CEA are becoming increasingly defined in randomized controlled trials,1 2 3 4 5 6 and the major events that can complicate CEA—stroke and death—are readily detected when hospital records are reviewed and patients are contacted for follow-up.
Using a classification of CEA appropriateness based on the results of randomized controlled trials evaluating the procedure, we previously audited a consecutive series of CEAs performed in our city and found a high rate of uncertain and inappropriate operations, as well as an unacceptably high stroke rate among patients with asymptomatic carotid stenosis.7 In the study reported here we wished to determine whether distribution and presentation of the results of this previous audit along with published practice guidelines and notification of ongoing monitoring could influence the performance of CEA in our city.
Subjects and Methods
The results of our previous retrospective study (hereafter referred to as “part 1” of the audit) were mailed to all surgeons performing CEA in our city, along with clinical practice guidelines for the use of CEA8 and notification that prospective surveillance of the use of this procedure was to commence (hereafter referred to as “part 2” of the audit). In addition, educational rounds presenting the results of the prior audit were held at both of the tertiary hospitals where CEA is performed in our region. The same surgeons who participated in part 1 of the audit were enrolled into part 2, except one who relocated out of the country. Prior approval for this study was obtained from our health region’s ethical review board.
In part 2 we performed a prospective study of all patients undergoing CEA in the city of Edmonton, Alberta, Canada, from September 1, 1996, to August 31, 1997. Edmonton has 2 tertiary care centers (which are the only hospitals in which CEAs are performed in Northern Alberta) and has a referral population of roughly 1.2 million people. Study methods were similar to that of part 1, with the exception that whereas part 1 was a retrospective 18-month study,7 part 2 had a prospective design. Hospital charts were reviewed by a nurse familiar with cerebrovascular disease who was independently associated with the regional medical quality control department. Patient demographic information, operative indications, and surgical results were recorded in a computer database (Access 97, Microsoft Corporation). Patients were considered symptomatic from their carotid stenosis if there was a documented history of prior ipsilateral retinal or hemispheric ischemia. Patients without such ipsilateral lateralizing neurological symptoms, including those with nonspecific complaints such as dizziness or cognitive impairment, and those in whom carotid stenosis was found incidentally were considered asymptomatic.
Preoperative carotid or cerebral angiograms were reviewed by an investigator (J.M.F. or J.H.W.) who was blinded to patient identity and degree of stenosis as read and reported by the radiologist who performed the procedure. Carotid stenosis was quantified according to the method used in the North American Symptomatic Carotid Stenosis Trial (NASCET) (ie, by comparing the greatest degree of linear stenosis at the carotid bifurcation with the normal distal internal carotid artery diameter).9
For each patient, the appropriateness of surgery was classified on the basis of the results of 5 randomized controlled trials studying CEA1 2 4 5 6 and by clinical practice guidelines established by the Canadian Neurosurgical Society,8 which had been circulated to the enrolled surgeons in the city. These criteria were used in conjunction with angiographic stenoses as recorded in the radiologists’ reports as well as with the remeasurement values of carotid narrowing. CEA was considered appropriate for patients with symptomatic carotid stenoses ≥70%. Uncertain indications for surgery were for those patients with symptomatic stenoses <70% or asymptomatic stenoses ≥60%. Patients were judged to have an inappropriate indication for surgery if they had an asymptomatic carotid stenosis <60% or if they were neurologically or medically unstable before CEA. Neurologically unstable patients were those who underwent surgery in the setting of a progressive neurological deficit or those who underwent surgery within 1 day of a fixed neurological deficit.10 11 High-risk preoperative medical conditions were unstable angina (defined as angina at rest or of new onset), myocardial infarction within 3 months before CEA, or uncontrolled congestive heart failure.
Patients were followed for the development of complications during their hospital course. The primary outcome was postoperative stroke, defined as the onset of a new neurological deficit (unrelated to cranial nerve injury) lasting >24 hours, or death within 30 days of surgery and was determined from both review of the hospitalization course and follow-up telephone interview of all discharged patients. A secondary outcome was the development of ≥1 cardiac complication in the hospital, specifically myocardial infarction, congestive heart failure, unstable angina, or atrial fibrillation.
Variables were coded dichotomously and analyzed with univariate techniques (χ2 or Fisher’s exact test, as appropriate) with the use of statistical computer software (SPSS 6.1, SPSS Incorporated). All tests were 2-tailed. Level of significance was set at a P value <0.05. The measure of agreement in angiographic accuracy between the radiologists’ reports and blinded remeasurements was quantified with the κ test statistic with 95% CIs. In determining a κ value, we dichotomized percent carotid stenoses into <70% or ≥70% categories and <60% or ≥60% categories for symptomatic and asymptomatic patients, respectively.
Over a 12-month period, 184 CEAs were performed in 172 patients by 8 surgeons from the neurosurgery, general surgery, and vascular surgery services. Eighty-one operations were performed by 2 neurosurgeons, and the remaining 103 operations were performed by 6 general or vascular surgeons. The number of procedures performed by individual surgeons ranged from 2 to 75; 3 surgeons each performed >20 operations during the study period, and the other 5 each performed <20. Patients ranged from 46 to 90 years of age, and 116 (63%) were male. When the preoperative symptom status of the patients was considered, 110 patients (60%) were symptomatic from their carotid disease and 74 (40%) were asymptomatic (Table⇓). Further examination of these asymptomatic patients revealed that 57% (42/74) had either a contralateral carotid occlusion or a severe (≥80%) ipsilateral carotid stenosis.
The accuracy of angiographic measurement was determined by comparing the original angiographic reading by the radiologist with the blinded remeasurement value and was quantified as a κ value. Readings of stenoses in part 2 showed moderate agreement with a κ value of 0.49 (95% CI, 0.28 to 0.69), which was not significantly different (P>0.05) from the angiographic accuracy found in part 1 of the audit (κ=0.71; 95% CI, 0.65 to 0.77). Of the 180 angiograms obtained for remeasurement in part 2 (since 4 angiograms were unavailable for analysis), discrepancies in angiographic interpretation were found in 18 cases, of which 15 were instances of overestimation of the degree of stenosis made by the original reporting radiologist.
Using our classification of operative appropriateness and based on the degrees of carotid stenosis determined by blinded remeasurement of the angiograms, we found that 49% of patients had appropriate indications for operation, 47% had uncertain indications for CEA, and 4% underwent surgery inappropriately. When these rates of appropriateness were compared with those determined in part 1, we found that the rate of appropriate CEAs had significantly improved from 33% (P=0.0005), the rate of CEAs for uncertain indications did not change significantly from 49% (P=0.61), and the inappropriate use of CEA dropped significantly from 18% (P=0.00002).
We considered that it might be fairer to the surgeons enrolled in this study to consider appropriateness not just according to remeasured degrees of stenosis but instead in relation to the stenosis values as reported by the radiologist and presumably available at the time of surgery. However, similar results were obtained; of the 184 cases of CEA, 95 patients (52%) underwent CEA for appropriate indications, 85 patients (46%) underwent surgery for uncertain reasons, and 4 patients (2%) had CEA inappropriately. These results were not significantly different from those determined with the use of blinded remeasurement of the carotid angiograms.
Among those CEAs classified as uncertain according to angiographic remeasurement, 20 cases (11% of the total) had <70% stenosis, and the remaining 64 (36% of the total) had ≥60% asymptomatic stenosis. All of the inappropriate operations were for <60% asymptomatic stenosis, and none were in medically or neurologically unstable patients.
In the series of patients examined in this study, 8 patients (4.3%) suffered a postoperative stroke and 1 patient (0.5%) died within 30 days of surgery, giving a total postoperative stroke or death rate of 4.9%. Six strokes and 1 death due to myocardial infarction occurred in the 110 patients with symptomatic carotid disease, and 2 strokes (and no deaths) occurred in the 74 asymptomatic cases, giving stroke or death rates of 6.4% and 2.7%, respectively.
Stroke or death rates among the different surgeons varied from 0% to 17%, although both extremes were from surgeons performing <10 CEAs each during the study period. Among those surgeons each performing >20 CEAs, the stroke or death rate ranged from 3.6% to 11%. When stratified according to surgical specialty, the stroke or death rate was the same (4.9%) for the neurosurgeons and the general or vascular surgeons.
Perioperative cardiac complications were also documented in the entire patient group. Two patients (1.1%) suffered myocardial infarcts, 1 patient (0.5%) developed congestive heart failure, 1 patient (0.5%) developed unstable angina, and 1 patient (0.5%) developed atrial fibrillation.
Since recent randomized controlled trials have firmly supported the use of CEA under certain circumstances, there has been renewed interest in CEA and a significant increase in its use.12 13 Such studies have allowed stratification of the appropriateness of surgical indications for CEA on the basis of scientific evidence rather than expert opinion alone.14 Several published randomized controlled trials have indicated that CEA, in conjunction with optimal medical therapy, is superior to medical therapy alone in reducing the risk of stroke in patients with symptomatic severe (≥70%) carotid stenosis, thus making CEA for these patients clearly appropriate.1 2 5 Although a randomized controlled study has shown an efficacy of CEA in lowering stroke risk among asymptomatic patients with ≥60% carotid disease, the demonstrated benefit of surgery was marginal and evident only with a very low operative complication rate.6 Controversy over the use of CEA for asymptomatic patients continues, thus suggesting that the appropriateness of CEA for this group is uncertain on the basis of currently available evidence.8
Concerned that the standardized entrance criteria of these randomized trials were not being followed during patient selection in our community, we examined the issues of appropriateness and complications of CEA in a retrospective audit.7 In that study of 291 consecutive CEAs performed on 265 patients from April 1994 through September 1995, we found appropriate indications for surgery in only 33% of patients, uncertain indications in 49%, and inappropriate indications in 18%. As well, we found that while the overall stroke or death rate was 5.2%, it was an unacceptably high 5.1% among patients with asymptomatic stenosis. In response to that analysis, we launched an educational campaign and prospective audit in an effort to address these problems.
The results of the prospective part of our audit contained in the present report indicate a significant increase in the proportion of appropriate operations as well as an important decrease in the use of CEA for inappropriate indications. In our study population, there was a greater use of CEA for patients with severe, symptomatic carotid disease, which is the patient group that benefits most from surgery.1 2 As well, fewer patients underwent surgery for asymptomatic carotid stenosis <60%, which remains a clearly inappropriate reason for CEA. The exact motivations behind this change in physician behavior are unclear. Given that the interval between parts 1 and 2 was <1 year and that substantial changes in local patient selection were found, we suggest that our regional audit and educational initiative are likely to be responsible for at least some of the observed changes in physician practice patterns. However, certainly other factors besides education, both quantifiable and not quantifiable, are likely to have influenced physician behavior and were not identified in this study.
With respect to the proportion of surgery for uncertain indications, there was no significant change between the 2 audits. In the present series, three quarters of the 84 patients in the uncertain group had ≥60% asymptomatic stenoses, and the remainder had moderate (<70%) symptomatic stenoses. Examining the uncertain group further, we found that just over one half of the asymptomatic patients had either a high-grade (≥80%) ipsilateral stenosis or a contralateral carotid occlusion, which are factors that may strengthen the argument for surgery in selected asymptomatic patients.8 Furthermore, the stroke or death rate in our asymptomatic patients was reduced to an acceptable level of risk (2.7%) that probably increased the benefit of CEA in this patient group. The reason for this decline in complications in our asymptomatic patients between parts 1 and 2 is unclear, although in both parts the asymptomatic subgroups were small enough that 1 or 2 outcome events could have made a significant impact on overall complication rates. It is possible that asymptomatic patients were more carefully selected in part 2, thus leading to a lower stroke or death rate.
Although the appropriateness of surgery for symptomatic patients with <70% stenosis will likely be influenced by NASCET study results, it should be noted that 18 of the 20 patients in this subgroup in the present series had stenoses that were remeasured as being between 60% and 70%. The appropriateness of CEA for these patients, likely to be justified, will depend on forthcoming results and analysis of the completed NASCET.
The accuracy of quantifying angiographic stenosis remains an important issue, as we and others have previously reported.7 15 16 17 18 It should be noted that before commencing our prospective study, we informed our radiologists of the results of the prior audit and requested specific use of the NASCET method in quantifying carotid stenoses. The roughly 10 radiologists that perform carotid angiography regionally were receptive to this suggestion, as evidenced by the almost uniform mention of NASCET criteria during angiogram reporting. While agreement between the original readings and the blinded remeasurements was acceptable, in cases in which a discrepancy between the 2 measurements was found, most (15/18, or 83%) were errors due to the overestimation of the degree of stenosis made by the reporting radiologist.
The results of an audit process such as ours depends on what is considered to constitute appropriate surgery. For example, a recent retrospective analysis of 1945 CEAs performed in Georgia in 1993 found that 96% were, according to their criteria, performed for appropriate indications.19 However, in that study carotid angiograms were not independently remeasured, and invasive imaging results were not uniformly obtained. As well, over half of patients were asymptomatic, which we have considered at best an uncertain indication since the benefit of CEA in this patient group is especially dependent on a low perioperative complication rate.20 Our classification of appropriateness will possibly change with release and analysis of the NASCET results for patients with symptomatic moderate carotid stenosis. Of note, however, is that only 11% of the patients in the present series were in this symptomatic moderate stenosis category.
The results of this study indicate that, at least locally, a strong interest in operating for asymptomatic carotid disease continues, despite designation of this practice as uncertain in the guidelines circulated and expression of significant concerns from authorities regarding surgery in this group of patients.21 However one chooses to classify the appropriateness of carotid surgery, this report does provide evidence that audits are useful in improving the performance of CEA. It has been suggested, and we would agree, that self-assessment of surgical performance by individual surgeons may be unreliable and that audits by independent third parties provide more accurate data regarding postoperative complications.22 Our results suggest that CEA audits that identify problems with procedure use, followed by dissemination of that information along with clinical practice guidelines and continued surveillance, can improve the appropriateness of CEA performed in a community. This study substantiates the practical use of audits in favorably influencing clinical practice patterns.
In conclusion, surgical audits can identify areas of concern in the performance of CEA. Combined with an educational campaign, further prospective surveillance of CEA use clearly reduced the number of inappropriate operations and increased the use of CEA for appropriate indications. As well, our audit may have helped to play a role in reducing the rate of complications in asymptomatic CEA patients. This is a group of patients in whom the benefit of surgery is especially dependent on careful patient selection combined with low operative risk. Surgical audits are recommended for the identification and resolution of problems in CEA use.
- Received July 20, 1998.
- Revision received September 9, 1998.
- Accepted September 22, 1998.
- Copyright © 1999 by American Heart Association
Mayberg MR, Wilson SE, Yatsu F, Weiss DG, Messina L, Hershey LA, Colling C, Eskridge J, Deykin D, Winn HR, Veterans Affairs Cooperative Studies Program 309 Trialist Group. Carotid endarterectomy and prevention of cerebral ischemia in symptomatic carotid stenosis. JAMA. 1991;266:3289–3294.
Wong JH, Findlay JM, Suarez-Almazor ME. Regional performance of carotid endarterectomy: appropriateness, outcomes, and risk factors for complications. Stroke. 1997;28:891–898.
Findlay JM, Tucker WS, Ferguson GG, Holness RO, Wallace MC, Wong JH. Guidelines for the use of carotid endarterectomy: current recommendations from the Canadian Neurosurgical Society. Can Med Assoc J. 1997;157:653–659.
McCrory DC, Goldstein LB, Samsa GP, Oddone EZ, Landsman PB, Moore WS, Matchar DB. Predicting complications of carotid endarterectomy. Stroke. 1993;24:1285–1291.
Gillum RF. Epidemiology of carotid endarterectomy and cerebral angiography in the United States. Stroke. 1995;26:1724–1728.
Barnett HJ, Warlow CP. Carotid endarterectomy and the measurement of stenosis. Stroke. 1993;24:1281–1284.
Alexandrov AV, Bladin CF, Maggisano R, Norris JW. Measuring carotid stenosis: time for a reappraisal. Stroke. 1993;24:1292–1296.
Rothwell PM, Gibson RJ, Slattery J, Warlow CP, for the Carotid Surgery Trialists’ Collaborative Group. Prognostic value and reproducibility of measurements of carotid stenosis: a comparison of the three methods on 1001 angiograms. Stroke. 1994;25:2440–2444.
Young GR, Sandercock PAG, Slattery J, Humphrey PRD, Smith ETS, Brock L. Observer variation in the interpretation of intra-arterial angiograms and the risk of inappropriate decisions about carotid endarterectomy. J Neurol Neurosurg Psychiatry. 1996;60:152–157.
Karp HR, Flanders D, Shipp CC, Taylor B, Martin D. Carotid endarterectomy among Medicare beneficiaries: a statewide evaluation of appropriateness and outcome. Stroke. 1998;29:46–52.
Biller J, Feinberg WM, Castaldo JE, Whittemore AD, Harbaugh RE, Dempsey RJ, Caplan LR, Kresowik TF, Matchar DB, Toole JF, Easton JD, Adams HP Jr, Brass LM, Hobson RW II, Brott TG, Sternau L. Guidelines for carotid endarterectomy: a statement for healthcare professionals from a special writing group of the Stroke Council, American Heart Association. Stroke. 1998;29:554–562.
Barnett HJ, Eliasziw M, Meldrum HE, Taylor DW. Do the facts and figures warrant a 10-fold increase in the performance of carotid endarterectomy on asymptomatic patients? Neurology. 1996;46:603–608.
Rothwell PM, Slattery J, Warlow CP. A systematic review of the risks of stroke and death due to endarterectomy for symptomatic carotid stenosis. Stroke. 1996;27:260–265.