Abstract
Background— Doppler ultrasound (duplex) tests are commonly applied after carotid endarterectomy to detect possible recurrent stenosis. The appropriate frequency and the benefits are unknown. We investigated the costs and effects of various follow-up strategies to determine the optimal strategy after carotid endarterectomy.
Methods— Using decision-analytic methods, a Monte Carlo Markov model was constructed. Probabilities and costs were obtained by systematic literature review. From empirical data regarding restenosis, a disease model was constructed to test the effect of various follow-up strategies using duplex testing and angiography. Main outcome measures were quality-adjusted life-years (QALYs), probability of stroke, and costs (for both the Dutch and the American situation).
Results— The average quality-adjusted life expectancy for a 66-year-old patient was 6.31 years for the symptom-guided strategy (with duplex scanning only being performed in case of symptoms of cerebral ischemia). The mean lifetime costs for this strategy were $5 600 for the US and 4 600 Euro for the Netherlands. The cumulative probability of stroke was 13%. Yearly routine duplex tests up to 5 years after operation resulted in similar QALYs and a similar probability of stroke, but higher costs ($7 300 for the US and 5 600 Euro for The Netherlands situation). No other strategy, including routine duplex surveillance, increased QALYs. When MR instead of conventional angiography was used as confirmatory test, no improvement was observed either.
Conclusions— Routine duplex surveillance does not result in an increase in quality-adjusted life expectancy, but it does increase costs. After successful carotid endarterectomy, a symptom-guided follow-up is an appropriate approach.
- carotid endarterectomy
- carotid stenosis
- cost-benefit analysis
- decision analysis
- ultrasonography, Doppler, duplex
- vascular surgery
Recently, results of several randomized controlled trials have demonstrated a benefit of carotid endarterectomy for severe carotid stenosis.1–4⇓⇓⇓ As a consequence, the number of carotid endarterectomies has more than doubled since 1989 in the US (amounting to 108 275 in 1996)5 as well as in The Netherlands.6 Although carotid endarterectomy decreases the risk of stroke or death, recurrent stenosis may develop in operated patients, which is associated with a modestly increased risk of stroke.7 Therefore, active follow-up of the operated artery seems warranted with the aim of performing redo surgery before complications occur. Although most surgeons use duplex scanning for follow-up of these patients, there is little consensus on the preferred frequency and duration of follow-up. In a recent survey among Dutch surgeons, substantial variation was observed in the frequency, timing, and duration of duplex surveillance after successful carotid endarterectomy.7a As yet, the benefit of routine duplex surveillance8,9⇓ and its cost-effectiveness10 are unknown. We performed a decision and cost utility analysis to identify the optimal strategy of (duplex) follow-up after successful carotid endarterectomy.
Subjects and Methods
We performed a decision analysis, using Monte Carlo Markov modeling methods, to simulate the course of patients who underwent carotid endarterectomy. This technique enabled us to compare outcomes over time in individual patients who underwent different follow-up regimens.11 The probabilities needed for the model were obtained by systematic review of the literature. Using these data, an underlying disease model was constructed that simulated the development of recurrent stenosis of the reconstructed carotid artery and its possible complications. Main outcomes were quality-adjusted life-years (QALYs) and costs. Intermediary outcomes were the number of strokes, number of reoperations, and number of true- and false-positive duplex tests. To compare cost and effect, 50 000 patients were simulated for each strategy, and 95% confidence intervals (CI) were calculated.
A multistate time-dependent transition decision model was constructed, using the software package DATA (Decision Analysis by TreeAge) version 3.5. The model assumed that patients have survived the initial 30-day perioperative period without complications.
Disease
The disease course of patients was structured using a limited set of predefined states of health that could change over time as a result of disease or medical interventions. Time was subdivided into fixed length cycles of 3 months, and each patient was assumed to occupy 1 of the available health states per cycle. The following 5 main health states were discerned: (1) NED (no evidence of disease, including minor disability), (2) reoperative treatment, (3) NED after reoperation, (4) major disability, and (5) death (Fig. 1). Figure 1A shows that only patients in the first health state are subject to further follow-up. This part of the model is displayed in Figure 1B, which indicates (dependent on the chosen strategy) when duplex surveillance takes place. The subsequent (reference) diagnostic work-up is displayed in Figure 1C.
Simplified representation of the decision model. NED indicates no evidence of disease. *Duplex analysis or no follow-up, dependent on chosen strategy.
The actual disease course of patients was modeled using a 2-level approach. At the first level, the patency and restenosis of the operated carotid artery and its associated risks of transient ischemic attacks (TIAs) and strokes, ie, underlying disease, is modeled as it actually evolves over time. Health states, changes, and events that are modeled at this level may well go unnoticed by patients and doctors and are thus independent of the actual follow-up regimen deployed and of the diagnostic accuracy that this regimen entails. The second level represents clinical observations, actions, and outcomes that result from the interaction between the first-level underlying disease process on the one hand and medical care supplied by follow-up on the other.
Underlying Disease
Carotid patency and restenosis was modeled as a time-dependent process for each individual simulated patient. However, because articles only report cumulative restenosis in groups of patients, published group data had to be reworked to disease histories of individual patients. For this purpose we used a binomial approach, in which total carotid diameter is subdivided into 20 “patency units” of 5% diameter each (this level of detail was chosen because published data were reported in intervals of no less than 5%). Per patency unit, there is a time-dependent risk R of restenosis, thereby decreasing the total diameter patency of the carotid artery. Finally, because not all carotid arteries experience restenosis, it is assumed that a certain cumulative proportion P of all carotid arteries is at risk of restenosis. This binomial model was implemented in a spreadsheet (Microsoft Excel 2000), using as input parameters R (occlusion risk per patency unit per time cycle) and P (cumulative proportion of population restenosis). These input parameters were varied in an iterative process until a best possible fit was reached between model predictions and published data. This best fit was obtained assuming a restenosis risk per 5% patency unit per time cycle R of 20%, and a cumulative proportion of population restenosis P of 15% (correlation coefficient = 0.94 between model stenosis predictions and published data for the first 5 years).
Clinical Events and Outcomes
The second level represents clinically observable events and actions. For example, a patient may experience a stroke as a consequence of angiography and may subsequently die, end up in the “major disability” state or the “minor disability” state, or recover completely (Fig. 1). Because each state has its specific quality-adjustment factor (utility), by multiplying the time spent in each state with the utility of that state, the quality-adjusted life expectancy (QALY) is obtained.
Follow-Up Strategies
The reference strategy was symptom-guided care, in which diagnostic and subsequent therapeutic medical interventions only take place in case of symptoms of cerebral ischemia. Besides symptom-guided care, the 6 most common strategies in The Netherlands were analyzed, in which patients undergo duplex scanning at predefined intervals (Table 1). In the base-case analyses, duplex scanning is the first test performed and is considered positive if it demonstrates a restenosis ≥70%. In case of a positive duplex test, angiography takes place, which, if positive, is followed by reoperation (Fig. 1C).
Strategies Using Routine Duplex Surveillance, of Which Costs and Effect Were Analyzed
Because strategies based on noninvasive testing alone, such as duplex scanning and MRA, have become increasingly common in the US,12 we also analyzed whether a different diagnostic work-up would change the outcome of routine surveillance in subsequent sensitivity analyses.
Patients who suffer from major disability as a result of stroke are not followed up any longer, because the benefit of any intervention is assumed to be very low. Reoperated patients are assumed to have received full patent arteries but are again at risk of restenosis in subsequent periods.13,14⇓ Reoperation was assumed to occur no more than once. We did not differentiate between the course of patients operated on for symptomatic and those operated on for asymptomatic carotid stenosis, because there is no convincing evidence that their course would differ notably.
In our model, we only took account of surveillance of the operated carotid artery, because the benefit of regular screening of an asymptomatic contralateral carotid artery is a different research question, which is nevertheless unresolved as well.15
Clinical Parameters
Quantitative data were obtained by systematic review of the literature. Averages, weighted for number, were calculated for each model parameter. We searched the literature using the MEDLINE database (1985 to July 2001) of the National Library of Medicine, updating and extending a recently published literature review.7 Details of the search strategies and the literature review can be found at http://www.medfac.leidenuniv.nl/mdmu/litreview.htm, and its results are presented in Table 2. Because individuals with severe carotid stenosis exhibit excess mortality as compared with the general population,16 we applied a relative mortality risk of 3.2 in the mortality tables used for the decision analyses.
Estimates of the Model Parameters, Obtained by Literature Review (Weighted Averages)
To evaluate realistic scenarios, we complied with commonly used duplex criteria, as proposed by Moneta et al,17 using the criteria to determine the degree of stenosis as proposed by the North American Symptomatic Carotid Endarterectomy Trial (NASCET) (Table 2).1 To obtain the sensitivity and specificity for MRA, we only included articles that reported the results of the better-performing gadolinium contrast-enhanced MRA.
Outcome and Utility of Stroke
Because reliable data on the outcome of stroke caused by endarterectomy or angiography are sparse, we used studies reporting on all ischemic strokes occurring in a defined population, whatever the cause, to obtain (population-based) estimates of case fatality and proportions of patients suffering disability after stroke. The modified Rankin scale was used to differentiate between major disability (Rankin 4 to 5) and minor disability (Rankin 2 to 3).18
Utilities for these 2 states were obtained by a separately published literature review.19 Articles were included if they reported elicitation of utilities using the time-trade-off method among subjects at risk for stroke, because patients at risk for stroke are most similar to patients at the moment of decision.20 Furthermore, time-trade-off method utilities are likely to be least prone to bias.21
Costs
The model includes the direct medical costs of diagnostic procedures, carotid surgery, and treatment of stroke (Table 3). Both were retrieved for the Netherlands and the US. All costs were adjusted to the 2001 price level. Future costs and QALYs were both discounted at 3% per year.
Costs Associated with Procedures and Consequences of Stroke
For the costs per diagnostic test and carotid surgery in the US, we used Medicare reimbursement rates.22 For the Netherlands situation, a detailed analysis was performed in our institution for the real costs of diagnostic procedures, and charges were used to value carotid surgery. Because our objective was to determine the appropriate frequency of duplex testing, we did not include the costs of follow-up visits in our (base case) analyses. The costs of stroke were divided into acute costs of stroke and annual costs of major and minor disability after stroke (also applicable to the initial year). The lifetime direct medical costs of stroke in the Netherlands were estimated at 81 000 guilders, of which 36% were incurred during the first year.23 Taking mortality, inflation, discounting, and the fixed Euro exchange rate into account, these estimates correspond to EUR 12 000 acute costs, plus EUR 8 200 per life year (1 Euro=0.89 US$ on October 24th, 2001). The lifetime direct medical costs of stroke in the US were estimated at $44 000, $17 000 of which were incurred during the first year.24 These estimates correspond to $22 000 acute costs, plus $8 000 per life year. Finally, according to a recent Canadian publication,25 the annual costs of major and minor stroke are 157% and 45%, respectively, of the annual costs of an average stroke (Table 3).
Results
Base-Case Analyses
Our reference strategy of symptom-guided care yielded 6.31 (95% CI 6.27 to 6.35) quality-adjusted life years, at a cost of 4 600 Euro (CI 4 400 to 4 800) for the Netherlands and $5 600 (CI 5 400 to 5 800) for the US (Table 4). The average lifetime risk of stroke was 12.6% (CI 12.3 to 12.9). Almost half of the patients underwent a duplex test, resulting in a positive angiogram, and thus reoperation, for 12% of the patients. Active follow-up using regular duplex scanning generally resulted in more angiographies, but these were apparently performed mainly after false-positive duplex test results, because the proportion of positive angiographies did not increase. For example, duplex testing at 3 and 12 months resulted in 13% positive angiographies per patients, whereas the proportion with a negative angiography increased to 27%. As a consequence, it resulted in similar QALYs. As shown in Table 5, none of the other strategies either significantly decreased the risk of stroke or resulted in significantly higher QALYs.
Main Results of the Cost-Effectiveness Analyses for the Most Common Strategies of Follow-Up After Carotid Endarterectomy in The Netherlands
Results of Sensitivity Analyses Comparing the Symptom-Guided Strategy with Yearly* Duplex up to 5 Years After Operation
We also tested a hypothetical strategy in which duplex tests or angiographies are never done (not even after a TIA or minor stroke occurred). This resulted in a small but significant increase in the proportion of strokes (13.9%; CI 13.6 to 14.2), although costs were slightly lower and QALYs were similar to the symptom-guided strategy. Because no strategy resulted in significantly higher QALYs, incremental cost-effectiveness ratios were not relevant.
Sensitivity Analyses
For the relevant sensitivity analyses, we compared the symptom-guided strategy (without routine duplex scans) with yearly duplex scans up to 5 years after surgery, including a test after 3 months (strategy VI).
Increasing the specificity to 97%, with a decrease in sensitivity to 53% (as published by Faught et al),26 reduced the average number of false-positive duplex tests to 0.13 (as compared with 0.56 in the base case scenario) and reduced the costs. Nonetheless, it did not result in better QALYs (Table 5). When we decreased the relative risk of TIA due to restenosis to a rather low value of 2 at a 70% to 99% restenosis (lowest reported relative risk is 4.0), no improvement was seen for the yearly routine duplex strategy.
To examine the effect of omitting routine angiography after a positive duplex on the outcome of routine surveillance, we analyzed 6 different diagnostic work-up schedules (Table 5). Using either duplex testing or MRA without confirmation by angiography did not result in higher QALYs for the yearly routine testing strategy. On the contrary, duplex alone resulted in reoperation in 1 of every 2 simulated patients. Apparently, the number of strokes and deaths saved by omitting angiography did not outweigh those occurring during (unnecessary) reoperations. The strategy in which MRA is performed after duplex, and angiography only in case of discordant duplex and MRA test results (“concordant MRA”), resulted in 15% reoperations but did not improve the outcome. No improvement was observed either when MRA was used as confirmation of a positive duplex test result (“conclusive MRA”) and a positive MRA resulted in reoperation. The costs associated with this strategy were similar to those obtained when angiography was used as confirmation test (base-case strategy). A second conclusive duplex test (without MRA or angiography) performed worse than the conclusive MRA strategy.
Finally, we tested a favorable, but hypothetical, scenario to make sure that our model was able to detect a beneficial effect of routine duplex scanning. When we assumed a relative risk of stroke due to restenosis of 5 at 70% to 99% restenosis (which is unlikely high), and decreased the relative risk of TIA to 2 at 70% to 99% stenosis (which is unlikely low), a 17% decrease of stroke risk was achieved by yearly duplex testing, resulting in significantly higher QALYs (5.94; CI 5.90 to 5.98) than the symptom-guided strategy (5.86; CI 5.82 to 5.90).
Discussion
In contrast to differences in costs, we found no significant differences in QALYs between the various strategies of follow-up. Moreover, no strategy that included routine duplex scanning was superior to the strategy that only prompted a duplex test when a TIA or minor stroke occurred. In a recent cost-effectiveness analysis, Patel et al.10 observed a marginal benefit of 0.02 QALYs for annual duplex scanning (including a scan at 6 weeks and at 6 months), albeit with an unfavorable incremental cost-effectiveness ratio.
The hypothetical scenario in which duplex tests and reoperations were never performed did not result in less QALYs but was associated with a slightly increased risk of stroke. Indeed, an increase in the number of strokes causes a decrease in QALYs. However, duplex surveillance is inevitably associated with angiography- and surgery-related mortality, which also causes a decrease in QALYs and apparently cancels out the improvement caused by the prevention of strokes.
The main problem with the duplex test in detecting recurrent carotid stenosis seems to be its low predictive value. According to Bayes theorem, the probability of a true positive test is a function of the sensitivity and specificity but also of the prior probability. Using this theorem, it can be shown that if the prior probability is as low as 4.5% and the sensitivity 78% and specificity 87%,17 the positive predictive value of a positive duplex test is 0.22. This means that only 1 in about 5 positive duplex tests actually indicates a severe carotid stenosis.
If a symptom-guided approach would be adopted in The Netherlands, the currently performed routine tests could be omitted. Based on approximately 1 020 carotid endarterectomies per year, and assuming that the number of tests applied after symptoms is the same for all strategies, the number of duplex tests omitted amounts to 2 122. Because approximately 1 in every 5 duplex tests would result in an angiography, 424 angiographies could be omitted in this approach. In The Netherlands, this would save 367 000 Euro per year (costs of physicians’ consultations not included).
It is important to note that 1 postoperative duplex test is justified for quality control of the surgeon’s performance. Moreover, most patients will need at least 1 physician’s consultation for standard care in an outpatient clinic after an operation such as carotid endarterectomy.
Methodological Considerations
Results of a Monte Carlo Markov simulation are always dependent on the assumptions of the model. Our main assumption was that estimates of the variables of the model could be obtained by summarizing the literature. This seems a fair assumption, although publication bias may have biased some of the estimates. However, varying these estimates in the sensitivity analyses did not change the results. The number of articles reporting the relative risk of TIA and stroke was limited, especially for the risk of TIA. Nonetheless, assuming the relative risk for TIA associated with 70% to 99% stenosis to be 2 (which is much lower than the lowest reported relative risk) did not change the results. The relative risk of stroke as a result of restenosis is unlikely to be higher than 2.7 Moreover, the NASCET collaborators recently reported that the relative risk of stroke does not exceed 2 among patients with asymptomatic severe primary carotid stenosis.27
It is relevant to note that we used the same sensitivities and specificities for routine tests and tests performed after neurological symptoms (for which the sensitivity is likely to be higher). As a consequence, we probably underestimated the yield of the symptom-guided approach.
Although we found a marginal difference in QALYs between some strategies, these differences were not significant. Unlike a classical Markov cohort analysis, a Monte Carlo Marlov analysis simulates the course of a large number of individual patients (50 000 in our simulations), resulting in a distribution of survival values. The mean of this distribution has the same interpretation as the expected utility obtained by a classical Markov (cohort) simulation, but it has a confidence interval.11 We used the Monte Carlo Markov modeling because it permitted us to specify transition probabilities that vary with time and with past history, viz. the incidence of restenosis and its associated relative risks. Because the 95% CI around the QALY estimate amounted to 0.04 QALYs to both sides, we could have missed a difference of at most 0.04 QALYs or 15 quality-adjusted life days.
Some limitations of our cost analysis should be mentioned. Firstly, the productivity costs (not likely to be an important issue in a patient group with a mean age of 66±8 years old) and other nonmedical costs associated with stroke were not taken into account. Secondly, the costs of stroke were obtained from articles that were not tailor-made for our model. Therefore, various assumptions had to be made with respect to mortality, inflation, and the relative costs of major and minor stroke. Nonetheless, because no differences in the probability of stroke or QALYs were observed between the various strategies, these limitations do not influence our conclusions.
Conclusions
For improving quality-adjusted life expectancy after an uncomplicated carotid endarterectomy, there is no scientific basis to perform routine duplex surveillance. Therefore, a symptom-guided follow-up is an appropriate approach.
However, for quality control of the operation, and for reassurance of the patient, 1 routine postoperative duplex test is justifiable. We therefore recommend 1 routine duplex test, followed by a symptom-guided approach.
Acknowledgments
This study was financially supported by a grant from the Leiden University Medical Center.
- Received September 28, 2001.
- Revision received November 13, 2001.
- Accepted November 27, 2001.
References
Jump to
This Issue
Article Tools
Share this Article
- Routine Duplex Surveillance Does Not Improve the Outcome After Carotid Endarterectomy
