Optimizing Protocols for Risk Prediction in Asymptomatic Carotid Stenosis Using Embolic Signal Detection
The Asymptomatic Carotid Emboli Study
Background and Purpose—Improved methods are required to identify patients with asymptomatic carotid stenosis at high risk for stroke. The Asymptomatic Carotid Emboli Study recently showed embolic signals (ES) detected by transcranial Doppler on 2 recordings that lasted 1-hour independently predict 2-year stroke risk. ES detection is time-consuming, and whether similar predictive information could be obtained from simpler recording protocols is unknown.
Methods—In a predefined secondary analysis of Asymptomatic Carotid Emboli Study, we looked at the temporal variation of ES. We determined the predictive yield associated with different recording protocols and with the use of a higher threshold to indicate increased risk (≥2 ES). To compare the different recording protocols, sensitivity and specificity analyses were performed using analysis of receiver-operator characteristic curves.
Results—Of 477 patients, 467 had baseline recordings adequate for analysis; 77 of these had ES on 1 or both of the 2 recordings. ES status on the 2 recordings was significantly associated (P<0.0001), but there was poor agreement between ES positivity on the 2 recordings (κ=0.266). For the primary outcome of ipsilateral stroke or transient ischemic attack, the use of 2 baseline recordings lasting 1 hour had greater predictive accuracy than either the first baseline recording alone (P=0.0005), a single 30-minute (P<0.0001) recording, or 2 recordings lasting 30 minutes (P<0.0001). For the outcome of ipsilateral stroke alone, two recordings lasting 1 hour had greater predictive accuracy when compared to all other recording protocols (all P<0.0001).
Conclusions—Our analysis demonstrates the relative predictive yield of different recording protocols that can be used in application of the technique in clinical practice. Two baseline recordings lasting 1 hour as used in Asymptomatic Carotid Emboli Study gave the best risk prediction.
The majority of strokes caused by carotid stenosis occur without previous TIA or stroke. Carotid endarectomy for asymptomatic carotid stenosis (ACS) offers an opportunity to prevent such strokes. Large clinical trials have shown the risk of stroke in patients with ACS can be significantly reduced by carotid endarectomy.1,2 However, the benefit was small, with ≈32 operations having to be performed to prevent 1 disabling stroke or death over a 5-year follow-up period. This reflects the low risk of stroke in medically treated patients with ACS. In the clinical trials this was ≈2% per year, but recent data have suggested that with current best medical treatment it may be ≤1% per year.3–5 This has led to the suggestion that carotid endarectomy for ACS offers little benefit, unless a subgroup of patients at particularly high risk for stroke could be identified.
Various risk stratification techniques have been suggested for identification of such a group. One of the most promising is embolic signal (ES) detection by transcranial Doppler (TCD).5–7 This is a simple noninvasive technique. The recent international multicenter Asymptomatic Carotid Emboli Study (ACES)7 found that ES detected on either of 2 recordings at baseline each lasting 1 hour predicted stroke risk over a 2-year follow-up period. Meta-analysis of ACES with previous smaller studies confirmed this association.7 However, ES detection is time-consuming, both for the patient and clinician, and it would be optimal to use as short a recording duration as possible. Whether similar predictive information could be obtained from a single baseline recording or by reducing the duration of recording to <1 hour is unknown. Currently, different clinical laboratories use different protocols and different recording durations. In a predefined secondary analysis of ACES, we looked at the temporal variation of ES and determined the predictive yield associated with different recording protocols. It has been suggested that use of a higher threshold8 (≥2 ES) may identify a group at higher risk group; therefore, we also analyzed the ACES data using this cutoff.
Subjects and Methods
ACES7 was designed to determine whether ES detected with TCD in the ipsilateral middle cerebral artery of patients with ≥70% ACS predicted stroke and TIA risk. The full protocol has been published previously.9 ACES recruited 482 subjects from 26 centers worldwide. The main results have been published.7
Inclusion criteria were carotid stenosis of 70% to 99% (established by carotid duplex ultrasound) that had been asymptomatic for at least 2 years. If patients had previous symptoms in the contralateral carotid artery or vertebrobasilar territory, then they were only eligible if symptoms occurred >2 years ago. Patients with previous contralateral carotid endarectomy were eligible 1 year after carotid endarectomy, assuming that they had been asymptomatic in the territory of the endarterectomized artery since the operation. Exclusion criteria included the following: other current diseases that were likely to limit life expectancy to <3 years; if the patient, physician, or surgeon was unwilling to manage ACS medically; absence of an acoustic window necessary for TCD; and nonbiological prosthetic heart valves.
Brain imaging with CT or MRI was performed at baseline. Subjects were followed-up at 6, 12, and 18 months, with final follow-up at 24 months. There was blinded assessment of all of the end points (stroke and TIA). If stroke or TIA occurred during follow-up, then a repeat brain CT/MRI was performed blinded to TCD information. This study was approved by the local ethics committees and all subjects gave written informed consent.
TCD ES Detection
A standard TCD recording protocol was followed. Briefly, a 2-MHz transducer was used to insonate the middle cerebral artery ipsilateral to the ACS. Two recordings were performed at baseline separated by 1 week. All ES data were recorded onto digital audiotape and analyzed centrally by investigators who were masked to clinical information. For central data analysis, the audio signal was played back into the 1 Doppler machine (Pioneer). Standard consensus criteria on ES identification were used in addition to an intensity threshold of ≥7 dB. ES were identified visually and audibly. All potential ES were then reviewed by a second experienced observer (H.M.).
Risk Factor Definitions
Hypertension was defined as using antihypertensive medication and/or systolic or diastolic blood pressures >140 or >90 mm Hg, respectively. Diabetes mellitus included a clinical diagnosis of type 1 and type 2 diabetes. Ischemic heart disease was recorded when patients had a history of angina or myocardial infarction. Peripheral vascular disease was recorded if there was a history of symptomatic disease. A history of smoking (current, former, and never were recorded). Atrial fibrillation was recorded if there was a history of it or it was present on ECG at entry. All medications were recorded.
The primary end point was ipsilateral TIA and stroke over the 2-year follow-up period. All end points were centrally adjudicated with review of clinical details and brain imaging and were blinded to the results of the ES recordings.
We compared the association between ES and future stroke and TIA risk using different recording protocols. This included 30-minute versus 1-hour recordings and performing single and repeated recordings. To identify independent predictors of ES at baseline, a hierarchical logistic regression analysis was performed to determine the interaction and relationship between baseline risk factors, demographics, and treatment. Fisher exact test was used for comparisons of categorical variables. Agreement between the 2 ES recordings was measured using Cohen κ. Cox proportional hazards regression models were used to calculate hazard ratios (HR) and their 95% confidence intervals (CI). SPSS software (version 18.0; SPSS) was used for analysis. P<0.05 was considered statistically significant. Receiver-operator characteristic curves were used to illustrate sensitivity and specificity of the recordings and give area under the curve statistics.10 These were then compared and adjusted for ties. Results are given as univariate associations and after controlling for antiplatelet therapy, because this was shown to be a significant predictor of risk.7
Four hundred seventy-seven patients were recruited and met the inclusion criteria.7 The first baseline recording was performed in all patients; 459 of these recordings could be analyzed. The second baseline recording was performed in 425 patients; 407 recordings could be analyzed. Four hundred sixty-seven patients had at least 1 recording performed (Table 1); 9.4% of patients were ES-positive on recording 1 and 10.6% were ES-positive on recording 2. Of the 399 patients who had 2 baseline recordings, 43 had ES on the first recording and 43 had ES on the second recording; 15 (3.8%) patients had ES on both recordings, and 328 (82.2%) were ES-negative on both recordings (Table 1). There was a highly significant association between ES status on the 2 recordings (P<0.0001). However, there was poor agreement between the 2 recordings, with a κ statistic of 0.266.
Baseline characteristics in those 60 patients who had only the first recording and the 399 with both recordings are shown in Table 2. There were more current smokers or ever-smokers in the group of those patients who had 2 baseline recordings (64.9% versus 45%; P=0.008). Those patients with 2 recordings showed a nonsignificant trend toward being younger (P=0.054), but they were significantly more likely to have a history of previous stroke/TIA (32.6% versus 11.7%; P<0.001). All other demographics, risk factors, and treatments were comparable between those with 1 or 2 baseline recordings.
Predictive Value of Different Recording Algorithms
HR and CI for the different recording protocols in predicting the outcomes of ipsilateral stroke alone and ipsilateral stroke/TIA are shown in Table 3. As previously reported, the presence of ES on either of 2 baseline recordings lasting 1 hour each predicted both ipsilateral stroke/TIA (HR, 2.54; CI, 1.20–5.36; P=0.015) and ipsilateral stroke alone (HR, 5.57; CI, 1.61–19.23; P=0.007). Reducing the recording to a single 1-hour baseline recording still resulted in significant prediction of ipsilateral stroke/TIA (HR, 2.78; CI, 1.20–6.45; P=0.015) but not ipsilateral stroke alone, for which the HR halved (HR, 2.46; CI, 0.52–11.63; P=0.26). Restricting the recording to the first 30 minutes of the first recording alone did not significantly predict either ipsilateral stroke/TIA or ipsilateral stroke alone (Table 3). Restricting the recording to 30 minutes on both recordings did not predict ipsilateral stroke/TIA or ipsilateral stroke alone (Table 3).
Forty patients had ≥2 ES on either baseline recording compared with 71 patients with ≥1 ES on 1 or both recordings. Analyzing both 1-hour recordings and using this cutoff of ≥2 ES to identify a group at high risk resulted in a worse prediction than using a cutoff of ≥1 ES. The HR using the ≥2 ES cutoff were much lower and were not significant (HR, 1.22l; CI, 0.37–4.00; P=0.747) for ipsilateral stroke/TIA and (HR, 2.99; CI, 0.63–14.08; P=0.17) for ipsilateral stroke alone (Table 3).
These results were similar after adjustment for antiplatelet therapy and also after adjustment for smoking and previous stroke or TIA, the 2 baseline variables that differed between those with 1 and 2 recordings (Table 3).
Sensitivity and Specificity of Different Recording Protocols
To compare the different recording protocols further, sensitivity and specificity analysis was performed by calculating the receiver-operator characteristic curves for these protocols (Table 4). Use of both 1-hour recordings had fair predictive accuracy for ipsilateral stroke/TIA (area under curve [AUC], 0.785; CI, 0.716–0.853) and strong predictive accuracy for ipsilateral stroke alone (AUC, 0.924; CI, 0.881–0.967; both P<0.0001). Use of the first recording only had poor predictive accuracy for both ipsilateral stroke/TIA (AUC, 0.696; CI, 0.616–0.777) and for ipsilateral stroke alone (AUC, 0.635; CI, 0.558–0.712; both P<0.0001). Analyzing the first 30 minutes of the first baseline recording only failed to show any predictive value for either outcome (ipsilateral stroke/TIA: AUC, 0.555; CI, 0.476–0.634; P=0.130; and ipsilateral stroke alone: AUC, 0.543; CI, 0.467–0.62; P=0.229). Reducing both the recordings to the first 30 minutes alone resulted in poor predictive accuracy for ipsilateral stroke/TIA (AUC, 0.603; CI, 0.520–0.685; P=0.004) but fair, although lower, predictive accuracy for ipsilateral stroke alone (AUC, 0.737; CI, 0.663–0.811; P<0.0001).
Using a cutoff of ≥2 ES on 1 or both of the 2 baseline recordings lasting 1 hour, predictive accuracy was slightly lower but fair for the outcome ipsilateral stroke/TIA (AUC, 0.738; CI, 0.658–0.819) but poor for the outcome ipsilateral stroke alone (AUC, 0.639; CI, 0.56–0.718; both P<0.0001).
Comparing Sensitivity and Specificity
Comparing the area under the curve statistics for the outcome of ipsilateral stroke/TIA showed the use of 2 baseline recordings lasting 1 hour each had significantly greater predictive accuracy than either the first baseline recording alone (z=3.48; P=0.0005), the first 30-minute baseline recording (z=9.11; P<0.0001), or 2 baseline recordings lasting 30 minutes (z=9.11; P<0.0001). Comparing the area under the curve for the different cutoffs of ≥1 ES versus ≥2 ES (number of ES required for a patient to be classified as ES-positive), there was no significant difference between the them (z=1.01; P=0.313).
For the outcome of ipsilateral stroke alone, using 2 recordings lasting 1 hour had significantly greater predictive accuracy when compared to the 1 recording lasting 1 hour only (z=11.82; P<0.0001). The use of two recordings lasting 1 hour results in an improved curve with better sensitivity and specificity values compared with a single recording lasting 1 hour (Figure 1), a single 30-minute baseline recording (z=16.56; P<0.0001), or 2 recordings lasting 30 minutes (z=7.59; P<0.0001). The use of 2 recordings lasting 1 hour results in an improved curve with better sensitivity and specificity values compared with 2 recordings lasting 30 minutes (Figure 2). Use of a threshold of ≥1 ES was also significantly better than a cutoff of ≥2 ES (z=10.96; P<0.0001).
ACES has shown that the presence of ES detected on TCD is an independent predictor of stroke risk over a 2-year follow-up period in patients with asymptomatic carotid stenosis. Meta-analysis of the results of this study with previous smaller studies confirmed this association.7 In ACES, the recording protocol was 2 recordings lasting 1 hour at study entry separated by 1 week. Other studies and clinical protocols have suggested that shorter recording periods of 30 minutes or the use of a single recording might have similar predictive value.5,6,8 We used the ACES dataset to compare the predictive value of these different protocols.
Previous studies have identified temporal variation of ES in asymptomatic carotid stenosis.11 Consistent with this, we found that although there was a highly significant correlation between ES on the 2 baseline recordings, the κ statistic for reproducibility between the 2 was poor. Using both Cox regression and analysis of receiver-operator characteristic curves, we found that the use of 2 baseline recordings lasting 1 hour was the best predictor of future risk. A single 1-hour recording also showed good predictive value and was a significant predictor of the primary end point of ipsilateral stroke or TIA but did not predict ipsilateral stroke alone; however, there were few stroke end points reducing the power to detect associations with this secondary end point. Reducing the duration of the recording to 30 minutes reduced the predictive value.
It has been suggested that using a cutoff of ≥2 ES improves prediction and perhaps identifies the higher-risk plaques that show more frequent embolization.5,8 However, we found that this reduced the predictive power of the technique; this largely reflected the fact that most subjects with ES had only 1 ES on the recordings and there were few with frequent ES.7
TCD ES detection may provide a useful tool for risk stratification in ACS. However, the technique is relatively labor-intensive. Currently, detection of ES is best performed by an experienced operator and commercially available ES detection systems have not been adequately validated in patients with ACS. This makes shorter recording durations attractive for the clinician. We show, for the first time to our knowledge, the relative yield of different recording protocols that can be used to plan application of the technique in clinical practice. Our results indicate that 2 baseline recordings lasting 1 hour with a threshold of ≥1 ES on 1 or both of the recordings gave the best prediction and receiver-operating characteristics.
Sources of Funding
The authors are grateful to the British Heart Foundation for funding this study (programme grant RG99/073 and renewal RG04/002).
ACES Study Personnel
Coordinating center office: St. George's University of London, Hugh Markus (principal investigator), Alice King (study coordinator October 17, 2008 to current), Jennifer Siegel (study coordinator April 30, 2007 to October 17, 2008), Sheila Reihill (study coordinator March 7, 2003 to April 29, 2007), Marisa Cullinane (study coordinator August 16, 1999 to March 6, 2003), Helen McCorie, Emma Morgan, Sun Kwon, Raffi Topakian, Kelly Jones, and Ruth Keating.
Study statistician: Martin Shipley, University College, London, UK.
Participating Centers, Study Personnel, and Number of Patients Recruited
Bretonneau Hospital, France (Francois Tranquart and Aurore Bleuzen, 2)
Charing Cross Hospital, UK (Alun Davies, 6)
Harbin Medical University, China (Song-Bin Qu, 20)
Institute of Psychiatry and Neurology, Poland (Anna Czlonkowskia, Anna Rozenfeld, Anna Piorkowska, and Marta Skowronska, 5)
James Connolly Memorial Hospital, Ireland (Dermot Fitzgerald and Nuala McMahon, 8)
J. W. Goethe University, Germany (Matthias Sitzer and Oliver Singer, 14)
Kings College Hospital, UK (Paul Baskerville, Colin Deane, and David Goss, 31)
Leicester Royal Infirmary, UK (Ross Naylor and Jo Walker, 23)
Martini Ziekenhuis Groningen, the Netherlands (Arjen Schaafsma and An Fokkens, 84)
Prince of Wales Hospital, Hong Kong (Lawrence Wong, Sunny Qing Hao, and Roxanna Liu, 3)
Rabin Medical Centre, Israel (Jonathan Streifler and Tilda Sabah, 7)
San Martino Hospital Genova, Italy (Giulia Brusa, Vittorio Montano, and Gian Andrea Ottonello, 21)
Singapore General Hospital Campus, National Neuroscience Institute, Singapore (Hui-Meng Chang, Moi Pin Lee, Meng Cheong Wong, and Christopher PLH Chen, 15)
South Manchester University Hospital, UK (Charles McCollum, Sarah Welsh, and Zoe Bonner, 26)
State Medical Academy, Tbilisi, Georgia (Marina Alpaidze, and Nana Meterveli, 12)
St. George's University of London, UK (Hugh Markus and Jennifer Siegel, 71)
Tel Aviv Sourasky Medical Centre, Israel (Natan Bornstein, Alex Gur, and Sigal Lorenz, 46)
UCL Institute of Neurology, UK (Martin M. Brown, 1)
UCLA School of Medicine, Los Angeles, California (Jeffrey Saver and Gina Paek, 5)
University Hospital Josep Trueta, Spain (Joaquin Serena and Xavier Ustrell, 19)
University Hospital Zagreb, Croatia (Vida Demarin and Vlasta Vukovic, 12)
University Medical Centre Ljubljana, Slovenia (Bojana Zvan and Janja Pretnar, 4)
University of Dusseldorf, Germany (Mario Siebler, Holger Schade, Torge Brosig, Christina Boettcher, and Verica Jovanovic, 8)
University of Münster, Germany (E. Bernd Ringelstein, Martin Ritter, and Ralf Dittrich, 19)
Vilnius University, Lithuania (Dalius Jatuzis, 19)
Wagner-Jauregg Hospital, Linz (Franz Aichner and Stefan Guggenberger, 1)
- Received April 5, 2011.
- Accepted April 26, 2011.
- © 2011 American Heart Association, Inc.
- Abbott AL
- Marquardt L,
- Geraghty OC,
- Mehta Z,
- Rothwell PM
- Abbott AL,
- Chambers BR,
- Stork JL,
- Levi CR,
- Bladin CF,
- Donnan GA
- Spence JD,
- Tamayo A,
- Lownie SP,
- Ng WP,
- Ferguson GG
- Mackinnon AD,
- Aaslid R,
- Markus HS