(Stroke. 2000;31:1335.)
© 2000 American Heart Association, Inc.
Original Contributions |
Evaluation of New Online Automated Embolic Signal Detection Algorithm, Including Comparison With Panel of International Experts
From the Department of Clinical Neurosciences (M.C., R.D., Z.K., H.S.M.), St George’s Hospital Medical School, London, UK; Nicolet-EME GmbH (G.R.), Kleinostheim, Germany; the Department of Neurology (R.D., D.W.D.), Munster, Germany; the Department of Clinical Neurophysiology (R.A.), St Antonius Hospital, Utrecht, Netherlands; the Department of Neurology (V.B.), Boston University of Medicine, Boston, Mass; the Department of Neurology (D.G.), Southern General Hospital, Glasgow, UK; the Department of Neurology (M.S.), University of Dusseldorf, Dusseldorf, Germany; and the Department of Neurology (L.V.), University of Toulouse, Toulouse, France.
Correspondence to Prof Hugh Markus, Clinical Neuroscience, St George’s Hospital Medical School, Cranmer Terrace, London, SW17 ORE, England. E-mail h.markus{at}sghms.ac.uk
 |
Abstract |
|---|
 Top Abstract IntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Background and Purpose—The clinical application of Doppler detection of circulating cerebral emboli will depend on a reliable automated system of embolic signal detection; such a system is not currently available. Previous studies have shown that frequency filtering increases the ratio of embolic signal to background signal intensity and that the incorporation of such an approach into an offline automated detection system markedly improved performance. In this study, we evaluated an online version of the system. In a single-center study, we compared its performance with that of a human expert on data from 2 clinical situations, carotid stenosis and the period immediately after carotid endarterectomy. Because the human expert is currently the "gold standard" for embolic signal detection, we also compared the performance of the system with an international panel of human experts in a multicenter study.
Methods—In the single-center evaluation, the performance of the software was tested against that of a human expert on 20 hours of data from 21 patients with carotid stenosis and 18 hours of data from 9 patients that was recorded after carotid endarterectomy. For the multicenter evaluation, a separate 2-hour data set, recorded from 5 patients after carotid endarterectomy, was analyzed by 6 different human experts using the same equipment and by the software. Agreement was assessed by determining the probability of agreement.
Results—In the 20 hours of carotid stenosis data, there
were 140 embolic signals with an intensity of 
7 dB. With the software
set at a confidence threshold of 60%, a sensitivity of 85.7% and a
specificity of 88.9% for detection of embolic signals were obtained.
At higher confidence thresholds, a specificity >95% could be
obtained, but this was at the expense of a lower sensitivity. In the 18
hours of post–carotid endarterectomy data, there
were 411 embolic signals of 7-dB intensity. When the same confidence
threshold was used, a sensitivity of 95.4% and a specificity of 97.5%
were obtained. In the multicenter evaluation, a total of 127 events
were recorded as embolic signals by at least 1 center. The total
number of embolic signals detected by the 6 different centers was 84,
93, 108, 92, 63, and 78. The software set at a confidence threshold of
60% detected 90 events as embolic signals. The mean probability of
agreement, including all human experts and the software, was 0.83, and
this was higher than that for 2 human experts and lower than that for 4
human experts. The mean values for the 6 human observers were averaged
to give P=0.84, which was similar to that of the
software.
Conclusions—By using the frequency specificity of the intensity increase occurring with embolic signals, we have developed an automated detection system with a much improved sensitivity. Its performance was equal to that of some human experts and only slightly below the mean performance of a panel of human experts
Key Words: carotid artery diseases • cerebral embolism • observer variation • signal processing, computer-assisted • ultrasonography
|
Introduction |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Transcranial Doppler ultrasound (TCD) can be used to detect asymptomatic emboli in the cerebral circulation.1 Embolic signals have been detected in patients with a wide variety of potential embolic sources, including symptomatic and asymptomatic carotid stenosis,2 3 atrial fibrillation,4 5 and prosthetic cardiac valves.6 They have also been detected during and after surgical procedures, including carotid endarterectomy1 7 and cardiopulmonary bypass.8
Increasing evidence suggests that at least in certain situations, embolic signals do have clinical significance, and embolic signals have been found to be predictive of increased stroke and transient ischemic attack risk in patients with asymptomatic or symptomatic carotid stenosis9 10 11 and in patients during the immediate post–carotid endarterectomy period.12 The technique has a number of promising applications, including the selection of high-risk patients for appropriate surgical and pharmacological intervention, determining the pathophysiology of stroke in individual cases, assessing the effectiveness of novel antiplatelet therapies, and perioperative monitoring. The major technical impediment to its widespread clinical use is the lack of a reliable automated method of embolic signal detection. The prevalence of embolic signals may be low, requiring many hours of patient recordings to detect only 1 or 2 embolic signals. Currently, the gold standard is to record the Doppler signal onto tape and review it later in real time. This is extremely time-consuming and only practical for research studies.
Previous attempts at producing an automated system have failed to reach
the same level of sensitivity and specificity as the current gold
standard of the human analyst.13 14 15 The most promising
system to date has been a neural network,13 but although
good levels of specificity were achieved, sensitivity of the system
remained relatively low. The major difficulty in previous systems has
been the detection of low-intensity embolic signals and their
differentiation from normal Doppler speckle. Therefore, systems
have tended to perform better for the more intense embolic signals seen
in patients with prosthetic cardiac valves but have been
unreliable for the less intense signals detected in patients with
carotid stenosis and atrial fibrillation. Any signal-processing
system that will improve the embolic signal–to–Doppler blood
signal intensity ratio is likely to aid detection of these
low-intensity signals. A characteristic feature of embolic signals is
that they are frequency-focused, having a maximum intensity over a
narrow frequency range. Recent work has demonstrated that the embolic
signal–to–Doppler blood signal intensity ratio can be increased
by 
3 dB by the use of frequency-filtering techniques.16
For automated signal detection, it is also essential to determine which
characteristics allow optimal discrimination of embolic signals from
both Doppler speckle and artifact. Using a novel signal
analysis approach, which provided both high temporal and
frequency resolution, we determined these characteristics in a previous
study.17 The frequency-filtering approach that we
developed was computationally intensive; therefore, we adapted the
approach by using the fast Fourier transform (FFT) as a frequency
filter, analyzing the output from each frequency bin independently. In
an offline system, we obtained a high sensitivity for embolic signals
using this approach.17 In the present study, we
describe the implementation of this system online and its testing in a
number of situations. We have tested it with the use of large amounts
of data from 2 clinical situations, carotid artery stenosis and
the period after carotid endarterectomy. The
current gold standard for embolic signal detection is the experienced
human observer; therefore, in addition, we have compared the
performance of the software with that the performance
of a panel of experienced human observers from a number of centers with
extensive research experience in the technique.
|
Subjects and Methods |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Transcranial Doppler Recordings
All recordings were made from the middle cerebral artery (MCA). A Nicolet/EME Pioneer TC4040 TCD machine was used for all recordings with a standard 2-MHz transducer. Depth settings were between 45 and 54 mm, and a sample volume of 5 mm was used. Doppler signals were recorded onto digital audio tape for subsequent analysis. All analysis was performed by an experienced observer blinded to patient group; all candidate signals were then reviewed by a second observer, and embolic signals were included only if both observers agreed. Standard criteria on the FFT spectral display were used to identify embolic signals in addition to the characteristic chirping or clicking sound.18
Data for Single-Center Evaluation
First Patient Group: Carotid Stenosis
TCD recordings were obtained from the ipsilateral MCA in
21 patients with 70% carotid artery stenosis. The group
comprised 18 symptomatic and 3 asymptomatic
patients. Recordings were performed for 1 hour in 19 patients
and 30 minutes in 2 patients, resulting in a 20-hour data set.
Second Patient Group: Carotid
Endarterectomy
Ipsilateral MCA recordings were made in 9 patients after
carotid endarterectomy, starting 30 minutes after
skin closure and continuing for 2 hours, resulting in an 18-hour data
set. No patients were patched.
Data for Multicenter Evaluation
Recordings, starting at least half an hour after skin
closure, were made from the ipsilateral MCA of patients after carotid
endarterectomy. The recording duration was
20 minutes in 3 patients and 30 minutes in 2 patients, resulting in a
2-hour data set. Six identical copies of these data sets were made and
sent to participating centers.
Embolic Signal Detection Algorithm Design
The algorithm uses a conventional FFT to analyze the
quadrature audio signal. To make the algorithm independent of user
interference, the algorithm computes an additional FFT, not displayed
to the user, based on the audio signal. To achieve high time and
frequency resolution, this FFT is always 64 point and is computed once
every millisecond. A Hanning function is used to window the FFT
with the overlap fixed at 89%, regardless of the FFT displayed to the
user.
The algorithm continually calculates an average background signal level over a range of FFT columns from 750 ms before each event to 750 ms after each event. The background level is determined by using a 2D median filter over the whole signal, except for those frequencies immediately adjacent to zero frequency. At the same time, any events that are of significant area (on the time-frequency surface) and >3 dB in intensity above the background level are further analyzed as embolic signal candidates. Candidate events are analyzed as 3D volumes in intensity-time-frequency space just above the background-level time-frequency surface. For each possible event found, the following parameters were measured:
1. Peak intensity volume: The volume under the intensity frequency curve is considered in 1-ms periods, each of which constitutes an intensity volume. The maximum of these is referred to as the peak intensity volume.
2. Peak intensity: This is the peak intensity at any frequency or time measured above the background level surface.
3. Event frequency disorder: This parameter measures how internally consistent or repeatable the shape of the event volume is, measured from the highest to the lowest frequency coordinates of the event volume.
4. Event time disorder: This parameter measures how internally consistent or repeatable the shape of the volume is, measured from the highest to the lowest time coordinates of the event volume.
5. Intensity volume reflection ratio: This is the ratio of total intensity volume of the event to the intensity volume of the equivalent area reflected across the zero frequency line.
6. Event distance to zero frequency: This is the frequency difference between the event center and zero frequency; events with reverse flow directions always have negative distance.
7. Teardrop shape of event area near zero frequency: It is often the case that an artifact can be "cut off" by TCD high-pass filters, resulting in a teardrop shape in the spectral display.
8. Localization in frequency: This is an inverse measure of the extent in frequency of an event.
9. Nearby high-intensity speckle interference: The presence of speckle near an event in time or frequency indicates that the event is unlikely to be an embolus if the nearby speckle and the event are of similar intensity.
The optimal parameters were derived from previous detailed analysis of embolic signals.17 This demonstrated that an artifact could be differentiated by the bidirectionality of the intensity increase and by the observation that the maximal intensity was at low frequency. The event disorder parameters mentioned were developed from the use of nonlinear forecasting, which has shown that embolic signals are highly ordered in time and frequency. Previous work has used the time domain data with this technique to distinguish embolic signals from speckle and artifact.19 This algorithm differs by working on the frequency domain (with use of FFT data).
Each of these parameters has some value in predicting whether an event is an embolic signal or an artifact signal or neither. For example, time or frequency disorder tends to be low for embolic signals but high for speckle and artifact, whereas the intensity volume reflection ratio is near 1 only for an artifact. Each parameter is converted into an index representing an estimate of the probability of an event being an embolic signal or an artifact. All the parameters are aggregated by using standard techniques from fuzzy logic. This results in both an embolus probability (or confidence level) and an artifact probability, expressed as a percentage, that a given signal is an embolic signal or that it is an artifact. Events with a high artifact probability are then identified as such and rejected as embolic signals. Remaining events with a high embolus probability are identified as embolic signals. Each event (embolic signal or artifact) is saved to disk along with its frequency coordinates, time extent coordinates, decibel intensity, and embolus probability level, allowing the event to be viewed and further analyzed offline.
Data Analysis
Single-Center Analysis
All 38 hours of data were played through the automated software.
All embolic signals saved by the software were reviewed by an
experienced human observer. The exact time and appearance of each
signal detected by the software was matched against the times
recorded by the human expert. Each signal detected by the software
was classified as a true positive signal if it appeared as a
characteristic embolic signal and matched the time noted by the human
analysis. The software detected a few additional signals. These
were reviewed and classified as false positives if they did not match
the standard criteria for embolic signals.18 There were a
few signals that were detected by the software and appeared to be clear
embolic signals but that had been missed by the human observer. These
were classified as true embolic signals. The intensity of each embolic
signal was determined from the color-coded intensity scale as
previously described.9 Interobserver agreement is
relatively poor for embolic signals of very low intensity; therefore,
many centers use an intensity threshold as one criterion for embolic
signal detection.18 Therefore, we used our standard
threshold of 7 dB as one criterion. Signals detected by the software
that were characteristic of embolic signals18 but were
below the 7-dB intensity threshold were classified as extra signals.
All other saved signals were classified as false-positive signals.
Sensitivity and specificity were then calculated with the threshold for
signal detection set at different embolus probability levels in
increments of 5%.
Multicenter Analysis
Each center analyzed the tape by playing the
recording back through an Nicolet-EME TCD machine and
monitoring both the audio signal and visual Doppler spectra with
use of the fastest sweep speed available to achieve best FFT temporal
overlap. They were asked to apply standard criteria for embolic signal
detection18 as they would for research or clinical studies
in their department. No intensity threshold was specified. This is
because absolute measurements of intensity vary with the method of
measurement18 and would not be the same in each center.
The exact time of each embolic signal was then noted by the observer.
All centers were blinded to patient details. The results were returned
to the coordinating center, where an independent observer had also
analyzed the same recording. The same 2-hour tape was
analyzed by the online software. The times of all embolic
signals detected by all observers, including the software, were noted.
An intensity value for each true embolic signal was determined from the
intensity color-coded spectral display as previously
described9 and calculated by the coordinating center.
Interobserver agreement was determined by a method based on the
proportion of specific agreement, as has previously been used to
examine inter-rater agreement in embolic signal
detection.20 This allows calculation of the probability
that a specified observer will detect an embolic signal compared with
the performance of 1 or more other observers. A value of 1
indicates perfect agreement; 0, no agreement. The inclusion of an
intensity threshold as one criterion for embolic signal identification
was not possible because of the measurement of intensity by use of
different methodologies in different centers. However, the probability
tests were performed both with and without the application of an 7-dB
intensity threshold by using the intensity measurements from the
original analysis by the coordinating center.
|
Results |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
Single-Center Evaluation
First Patient Group: Carotid Stenosis
Human analysis of the data detected 134 embolic signals with intensity
7 dB. Software analysis of the data detected
an additional 6 embolic signals that were 7 dB but had been missed by
the human observer, making the total number of true positive signals
140. When the threshold for detection was set at an embolus probability
value of 65%, a sensitivity of 85.7% and a specificity of 88.9% were
obtained. Table 1
shows how the
performance varied according to the embolus probability or
confidence threshold at which the software was used. At higher
thresholds, a specificity of >95% could be obtained, but this was at
the expense of a lower sensitivity.
|
Of those 20 true embolic signals missed at an embolus probability threshold setting of 65%, 6 were missed because of the presence of coincident artifact, leading to categorization as an artifact, whereas 7 were missed probably because they were low-intensity embolic signals that the software failed to detect. In the 7 remaining cases, the reason for the failure to detect the embolic signal was unclear.
Second Patient Group: Carotid Endarterectomy
Human analysis of the data detected 402 embolic signals of
7-dB intensity. Software analysis of the data detected an
additional 9 embolic signals that were 7 dB but had been missed by
the human observer, making the total number of true positive signals
411. After analysis of the first patient group, we found that
using an embolus probability threshold of 65% produced the same
sensitivity as using an embolus probability threshold of 60% but that
it increased specificity. For the purpose of this analysis, the
minimum threshold used was 65%. With use of this threshold, a
sensitivity of 95.4% and a specificity of 97.5% were obtained. Table 2 shows the effect of altering the
threshold on the sensitivity and specificity.
|
Of those 19 true embolic signals missed at a embolus probability threshold setting of 65%, 8 were missed because of the presence of a coincident artifact, leading to categorization as an artifact, whereas 11 were missed probably because they were low-intensity embolic signals that the software failed to detect.
The mean±SD intensity of the embolic signals, as defined above,
including a 7-dB intensity threshold, was significantly higher in the
carotid endarterectomy group than in the carotid
stenosis group: 15.3±4.9 versus 12.8±4.4 dB
(P=<0.0001).
Multicenter Evaluation
A total of 127 events were recorded as embolic signals by at
least 1 center. The total number of embolic signals detected by the 6
different centers were 84, 93, 108, 92, 63, and 78. With the software
set at an embolus probability threshold of 60%, 90 events were
detected as embolic signals. The probability that an observer from a
second center would detect an embolic signal if an observer from 1
center had also detected an embolic signal is shown in Table 3. The values for each center have been
averaged to give a mean value, as shown in the rightmost column of
Table 3. The mean probability of agreement value for the
software was 0.83, and this was higher than that for 2 centers and
lower than that for 4 centers. The mean values for the 6 human
observers were averaged to give a probability of agreement value of
0.84, which was similar to that of the software.
|
Two centers detected rather different numbers of embolic signals, with
center 3 detecting 108 events and center 5 detecting 63 events. On
review, center 5 had missed some clear embolic events, whereas center 3
had included some events that appeared not to fulfill the standard
criteria for embolic signals.18 In view of this and to
evaluate the software against the most rigorous criteria, namely, the
"best" human experts, the analysis was repeated with the
data from centers 3 and 5 omitted. The results are shown in Table 4. The values for each center have been
averaged to give a mean probability value as shown in the rightmost
column of Table 4. The mean value for the software was 0.85, and
this was higher than that for 1 center and lower than that for 2
centers. The mean values for the 4 human observers were averaged to
give a value of 0.87, which was only slightly higher than that for the
software.
|
There was a highly significant relationship between the proportion of
centers agreeing that a certain signal was an embolic signal and of a
certain intensity (Spearman 
=0.667, P=0.001). All embolic
signals detected by 1 or more centers were reevaluated. and their
intensities were measured by the standard method used in the
coordinating center.9 The analysis was then
repeated for all centers using a 7-dB intensity threshold as an
additional criteria in the definition of an embolic signal. The results
comparing the performance of the software with that of all
centers are shown in Table 5. The mean
probability value for the software was 0.87, and this was higher than
that for 2 centers and lower than that for 4 centers. The mean values
for the 6 human observers were averaged to give a value of 0.88 which
was similar to that of the software. As discussed above, the
analysis was repeated after omitting data from centers 3 and 5,
which had reported notably more and notably less embolic signals,
respectively, than the other centers. The results are shown in Table 6. The mean value for the software was
0.87, and this was higher than that for 1 center and lower than that
for 2 centers. The mean values for the 4 human observers were averaged
to give a value of 0.90, which was only slightly higher than that of
the software.
|
|
|
Discussion |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
The present study tested a new online algorithm for the automated detection of embolic signals that is based on a novel approach that uses the fact that embolic signals have a maximum intensity over a narrow frequency range. We demonstrated consistently good performance across 2 clinical situations, with the software compared with expert observers in 1 center and with the software compared with an international panel of experts from a number of centers. The results are considerably better than those of any previously published automated detection system,21 and the levels of sensitivity and specificity obtained are of the order of those required if automated analysis is to replace the trained human observer in the detection of embolic signals.
In the first half of the study, the software was analyzed in a single center and compared with the performance of trained human observers. We studied data sets from 2 clinical situations, carotid artery stenosis and the period after carotid endarterectomy. In both situations, good levels of detection were obtained, with a sensitivity and specificity of 85.7% and 88.9%, respectively, for carotid stenosis and 95.4% and 97.5%, respectively, for the period after carotid endarterectomy. This was with the threshold for detection set at 65%. Higher specificity could be obtained when the threshold was raised, but this was at a loss of sensitivity. In many clinical situations, the frequency of embolic signals is low. In patients with carotid stenosis and atrial fibrillation, the median number per hour in embolic signal–positive patients is only 1 to 3.3 4 In such situations, the sensitivity must be high; therefore, a threshold set at 65% would be optimal. This then requires an experienced observer to check the saved segments after analysis. This is facilitated because the software saves the detected segments with the corresponding audio soundtrack so that they can be rapidly reviewed. This setup is similar to that currently used by 24-hour ECG monitoring analysis systems.
The performance of the software was better for the embolic signals detected after carotid endarterectomy than for those in patients with carotid stenosis. Our clinical impression was that the embolic signals recorded in the post–carotid endarterectomy patients were of greater intensity, and this was confirmed by quantitative signal analysis. This is likely to be the reason for the improved performance of the software in the post–carotid endarterectomy group. However, the mean intensity values in the post–carotid endarterectomy patients were below those that are commonly found in patients with prosthetic heart valves or during interventional radiological procedures, when the majority of embolic signals are believed to be due to gaseous emboli. We believe that this is due to the fact that embolic signals in the post–carotid endarterectomy setting represent larger solid emboli, and this is supported by their marked reduction after administration of the antiplatelet agent S-nitrosoglutathione.22 This emphasizes that it is important that an automatic detection system be tested on the data set that it will be subsequently be used on. One might expect the performance of this system to be even better in patients with prosthetic heart valves, in whom embolic signals are of even greater intensity,23 but this needs testing. Similarly, the software needs evaluating in patients with atrial fibrillation, in whom embolic signals are infrequent and of low intensity4 ; in this group, its performance may not be as good. Our initial pilot data suggest a sensitivity of only about 50% in this group.
In the single-center study, we evaluated the detection of embolic
signals defined as having an intensity of 7 dB. This is the standard
intensity threshold that we use in all our studies. The presence of
embolic signals, defined by using this threshold, has been shown to be
predictive of stroke and transient ischemic attack risk in
patients with carotid artery stenosis.9
Interobserver agreement is not as good for embolic signals of very low
intensity, and this was confirmed in our multicenter study. The use of
an intensity threshold has been shown to increase reproducibility
without too great a loss of sensitivity, and its use is
recommended in recent consensus criteria.18 A difficulty
with this approach in the evaluation of an automated system is that the
software may detect embolic signals that have an intensity below the
threshold. This occurred in the present study, but for the purpose
of the study, these signals were excluded and counted as neither true
positives or true negatives. However, in online use, the software
calculates the intensity of any events its saves; therefore,
instructing it to detect embolic signals only above a certain threshold
is possible. It should be remembered that the absolute intensity value
depends on the method in which it is calculated and, in particular, how
the relative intensity of both the embolic signal and background are
calculated.24 The 7-dB threshold was determined by using
previously described methods from the color-coded intensity
scale24 and not calculated by the computer
algorithm. Therefore, the absolute value of an appropriate threshold
for use by the software is not necessarily 7 dB but could be easily
determined.
In the second part of the present study, we evaluated the software
against a panel of experienced human observers. We felt that this was
appropriate because the present gold standard for embolic signal
identification is the human observer. Although generally high levels of
agreement between human experts have been found in previous
work,20 25 there is some interobserver variation.
Therefore, we determined whether the software was as good as an
experienced human observer. When compared for all 6 centers, the
performance of the software was very similar, with a mean
proportion of specific agreement of 0.83 compared with 0.84 for the
averaged value for the human experts. Two centers performed
significantly differently from the other centers; one reported fewer
embolic signals, whereas the other reported additional embolic signals,
which the other centers did categorize as embolic signals. This
reflects the fact that very-low-intensity embolic signals may occur,
and only certain centers included these as definite signals, which they
would report as embolic signals when performing the technique for
research and clinical studies. Essentially, some centers seem to be
using an implicit intensity threshold. To provide the most rigorous
test of the software, we reevaluated it against the "best" humans
experts, with the data from these 2 outlying experts removed. Even this
selected group of human experts performed only slightly better than the
software, with an averaged value of 0.87 compared with 0.84 for the
software. Embolic signals that were most commonly disagreed on between
centers, including the software, were those of low intensity, as
reported in previous studies,20 25 and this emphasizes the
benefit of applying an intensity threshold. Our results also
demonstrate that despite the publication of detailed consensus criteria
for the detection of embolic signals,18 a minority of
experienced research centers are not applying these rigorously. This
emphasizes the need for continuing intercenter reproducibility studies
as part of an ongoing quality control program. There is also the
potential problem of human error, particularly when large amounts of
data have to be analyzed. Even in our single-center evaluation,
we found a number of typical embolic signals of intensity 7 dB that
had been missed by the human observer but were detected by the
software; this constituted 4% of the carotid stenosis embolic
signals and 2% of the carotid endarterectomy
embolic signals. This emphasizes a major potential advantage of an
automated system over the human observer; it does not suffer from
fatigue.
The software missed only 5% to 10% of embolic signals in the
different data sets. In some cases, these were low-intensity signals.
The detection of these may be improved by adjustments to the algorithm,
but the detection of some may require a different method of signal
analysis that maximizes the embolic signal–to–Doppler
blood signal intensity ratio further. For example, the wavelet
transform is particularly suited to the analysis of
short-duration transient signals, and pilot data suggest that it
describes embolic signals better than the FFT.26 In other
cases, the software missed embolic signals that occurred at the same
time as artifact. In these cases, the use of a multigate system may
improve detection. This could work in an offline mode, with detection
occurring in 1 channel but with 2 channels being saved for subsequent
review by the human expert. In uncertain cases, embolic signals could
then be identified by the time delay occurring between the proximal and
distal channels.27 This would also help distinguish
between embolic signals and artifact on the rare occasions on which a
predominantly, but not exclusively, unidirectional signal occurs with
maximum intensity at low velocity; such signals can result from emboli
or, rarely, artifact.
For the first time, this automated system provides a method with sufficient performance for routine clinical use. However, potential limitations need to be borne in mind. First, we have only validated its use in 2 situations, carotid stenosis and the period after carotid endarterectomy. Its performance needs to be similarly evaluated in other situations. It may work less well for the very few infrequent signals seen in patients with atrial fibrillation. It may also work less well in patients with the more intense embolic signals occurring during, rather than after, interventional procedures, such as cardiopulmonary bypass, carotid endarterectomy, and cerebral angiography. A proportion of the emboli in such situations are believed to arise from gaseous bubbles and to result in embolic signals of much higher intensity. This can lead to receiver overload and a degree of aliasing. This appears as a bidirectional intensity increase and may lead to mistaken identification of embolic signals as artifact. However, modification of the algorithm and the use of a TCD system with sufficient dynamic range should overcome this problem. Second, although we tested the algorithm on a large amount of unselected routine clinical data, it may work less well on certain data sets. Third, the saved signals need to be reviewed by a human expert for optimal performance; therefore, the use of the software needs to be combined with appropriate training.
Despite these potential limitations, this automated system is significantly better than previous published approaches and, for the first time, provides a system whose performance is similar to that of the human expert, the current gold standard.
|
Acknowledgments |
|---|
This study was supported in part from a project grant from the British Heart Foundation (PG96176). We thank Dr Jane Molloy for help with patient monitoring.
Received January 31, 2000; revision received March 21, 2000; accepted March 21, 2000.
|
References |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionReferences |
|---|
-
Spencer MP, Thomas GI, Nicholls SC, Sauvage
LR. Detection of middle cerebral artery emboli during carotid
endarterectomy using transcranial
Doppler ultrasonography. Stroke.. 1990;21:415–423.
[Abstract/Free Full Text] -
Siebler M, Kleinschmidt A, Sitzer M, Steinmetz H,
Freund HJ. Cerebral microembolism in symptomatic and
asymptomatic high-grade internal carotid artery
stenosis. Neurology.. 1994;44:615–618.
[Abstract/Free Full Text] -
Markus HS, Thomson ND, Brown MM.
Asymptomatic cerebral embolic signals in
symptomatic and asymptomatic carotid artery
disease. Brain.. 1995;118:1005–1011.
[Abstract/Free Full Text] -
Cullinane M, Wainwright R, Brown A, Monaghan M, Markus
HS. Asymptomatic embolization in subjects with atrial
fibrillation not taking anticoagulants: a prospective study.
Stroke.. 1998;29:1810–1815.
[Abstract/Free Full Text] -
Sliwka U, Job FP, Wissuwa D, Diehl RR, Flachskampf FA,
Hanrath P, Noth J. Occurrence of transcranial Doppler
high-intensity transient signals in patients with potential cardiac
sources of embolism: a prospective study. Stroke.. 1995;26:2067–2070.
[Abstract/Free Full Text] - Rams JJ, Davis DA, Lolley DM, Berger MP, Spencer M. Detection of microemboli in patients with artificial heart valves using transcranial Doppler: preliminary observations. J Heart Valve Dis.. 1993;2:37–41.[Medline] [Order article via Infotrieve]
- Gaunt ME. Transcranial Doppler: preventing stroke during carotid endarterectomy. Ann R Coll Surg Engl.. 1998;80:377–387.[Medline] [Order article via Infotrieve]
- Gallagher EG, Pearson DT: Ultrasonic identification of sources of gaseous microemboli during open heart surgery. Thorax.. 1973;28:295–305.[Medline] [Order article via Infotrieve]
-
Molloy J, Markus HS. Asymptomatic
embolization predicts stroke and TIA risk in patients with carotid
artery stenosis. Stroke.. 1999;30:1440–1443.
[Abstract/Free Full Text] - Siebler M, Nachtmann A, Sitzer M, Rose G, Kleinschmidt A, Rademacher J, Steinmetz H. Cerebral microembolism and the risk of ischemia in asymptomatic high-grade internal carotid artery stenosis. Stroke. 1995; 26:2184–2186.
- Valton L, Larrue V, Pavy le Traon A, Massabuau P, Geraud G. Microembolic signals and risk of early recurrence in patients with stroke or transient ischemic attack. Stroke. 1998; 29:2125–2128.
-
Levi CR, O’Malley HM, Fell G, Roberts AK, Hoare MC,
Royle JP, Chan A, Beiles BC, Chambers BR, Bladin CF, Donnan GA.
Transcranial Doppler detected cerebral microembolism
following carotid endarterectomy: high
microembolic signal loads predict postoperative
cerebral ischemia. Brain.. 1997;120:621–629.
[Abstract/Free Full Text] -
Siebler M, Rose G, Sitzer M, Bender A, Steinmetz H.
Real-time identification of cerebral microemboli with US feature
detection by a neural network. Radiology.. 1994;192:739–742.
[Abstract/Free Full Text] -
Droste DW, Hagedorn G, Notzold A, Siemens HJ, Sievers,
HH, Kaps M. Bigated transcranial Doppler for the
detection of clinically silent circulating emboli in normal persons and
patients with prosthetic cardiac valves. Stroke.. 1997;28:588–592.
[Abstract/Free Full Text] -
Kemeny V, Droste DW, Hermes S, Nabavi DG,
Schulte-Altedorneburg G, Siebler M, Ringelstein EB. Automatic embolus
detection by a neural network. Stroke.. 1999;30:807–810.
[Abstract/Free Full Text] - Markus HS, Reid G. Frequency filtering improves ultrasonic embolic signal detection. Ultrasound Med Biol.. 1999;25:857–860.[Medline] [Order article via Infotrieve]
-
Markus H, Cullinane M, Reid G. Improved automated
detection of embolic signals using a novel frequency filtering
approach. Stroke.. 1999;30:1610–1615.
[Abstract/Free Full Text] -
Ringlestein EB, Droste DW, Babikian VL, Evans DH,
Grosset DG, Kaps M, Markus HS, Russell D, Siebler M. International
Consensus Group on Microembolus Detection: consensus on microembolus
detection by TCD. Stroke.. 1998;29:725–729.
[Abstract/Free Full Text] -
Keunen RWM, Stam CJ, Tavy DLJ, Mess WH, Titulaer BM,
Ackerstaff RGA. Preliminary report of detecting microemboli signals in
the transcranial Doppler time series with nonlinear
forecasting. Stroke.. 1998;29:1638–1643.
[Abstract/Free Full Text] -
Markus H, Bland JM, Rose G, Sitzer M, Siebler M. How
good is intercenter agreement in the identification of embolic signals
in carotid artery disease? Stroke.. 1996;27:1249–1252.
[Abstract/Free Full Text] -
Van Zuilen EV, Mess WH, Jansen C, Van der Tweel I, van
Gijn J, Ackerstaff RGA. Automatic embolus detection compared with human
experts: a Doppler ultrasound study. Stroke.. 1996;27:1840–1843.
[Abstract/Free Full Text] -
Molloy J, Martin JF, Baskerville PA, Fraser SCA, Markus
HS. S-Nitrosoglutathione reduces the rate of embolization in humans.
Circulation.. 1998;98:1372–1375.
[Abstract/Free Full Text] -
Grosset DG, Georgiadis D, Kelman AW, Lees KR.
Quantification of ultrasound emboli signals in patients with cardiac
and carotid disease. Stroke.. 1993;24:1922–1924.
[Abstract/Free Full Text] -
Markus HS, Molloy J. The use of a decibel threshold in
the detection of embolic signals. Stroke.. 1997;28:692–695.
[Abstract/Free Full Text] -
Markus HS, Ackerstaff RG, Babikian VL, Bladin C, Droste
D, Grosset D, Levi C, Russell D, Siebler M, Tegeler C. Inter-center
agreement in reading Doppler embolic signals: a multicenter
international study. Stroke.. 1997;28:1307–1310.
[Abstract/Free Full Text] - Aydin N, Padayachee S, Markus HS. The use of the wavelet transform to describe embolic signals. Ultrasound Med Biol.. 1999;25:953–958.[Medline] [Order article via Infotrieve]
-
Molloy J, Markus HS. Multigated Doppler ultrasound
in the detection of emboli in a flow model and embolic signals in
patients. Stroke.. 1996;27:1548–1552.
[Abstract/Free Full Text]
This article has been cited by other articles:
 |
|
 |
|
  R. M. Kwee, R. J. van Oostenbrugge, L. Hofstra, G. J. Teule, J.M.A. van Engelshoven, W. H. Mess, and M. E. Kooi Identifying vulnerable carotid plaques by noninvasive imaging Neurology, June 10, 2008; 70(24_Part_2): 2401 - 2409. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Dittrich, M. A. Ritter, M. Kaps, M. Siebler, K. Lees, V. Larrue, D. G. Nabavi, E. B. Ringelstein, H. S. Markus, and D. W. Droste The Use of Embolic Signal Detection in Multicenter Trials to Evaluate Antiplatelet Efficacy: Signal Analysis and Quality Control Mechanisms in the CARESS (Clopidogrel and Aspirin for Reduction of Emboli in Symptomatic carotid Stenosis) Trial Stroke, April 1, 2006; 37(4): 1065 - 1069. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y. Kalairajah, A. J. Cossey, G. M. Verrall, G. Ludbrook, and A. J. Spriggins Are systemic emboli reduced in computer-assisted knee surgery?: A PROSPECTIVE, RANDOMISED, CLINICAL TRIAL J Bone Joint Surg Br, February 1, 2006; 88-B(2): 198 - 202. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
A. L. Abbott, B. R. Chambers, J. L. Stork, C. R. Levi, C. F. Bladin, and G. A. Donnan Embolic Signals And Prediction of Ipsilateral Stroke or Transient Ischemic Attack in Asymptomatic Carotid Stenosis: A Multicenter Prospective Cohort Study Stroke, June 1, 2005; 36(6): 1128 - 1133. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. A Rodriguez, K. A Williams, A. Babaev, F. Rubens, and H. J Nathan Effect of perfusionist technique on cerebral embolization during cardiopulmonary bypass Perfusion, January 1, 2005; 20(1): 3 - 10. [Abstract] [PDF]
|
|
|
|
 |
|
 I. K. Moppett and R. P. Mahajan Transcranial Doppler ultrasonography in anaesthesia and intensive care Br. J. Anaesth., November 1, 2004; 93(5): 710 - 724. [Full Text] [PDF]
|
|
|
|
|
|
A. D. Mackinnon, R. Aaslid, and H. S. Markus Long-Term Ambulatory Monitoring for Cerebral Emboli Using Transcranial Doppler Ultrasound Stroke, January 1, 2004; 35(1): 73 - 78. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. Laas, S. Kseibi, M. Perthel, A. Klingbeil, L'E. El-Ayoubi, and A. Alken Impact of high intensity transient signals on the choice of mechanical aortic valve substitutes Eur. J. Cardiothorac. Surg., January 1, 2003; 23(1): 93 - 96. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. Brucher and D. Russell Automatic Online Embolus Detection and Artifact Rejection With the First Multifrequency Transcranial Doppler Stroke, August 1, 2002; 33(8): 1969 - 1974. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
G. Devuyst, G.A. Darbellay, J.-M. Vesin, V. Kemeny, M. Ritter, D.W. Droste, C. Molina, J. Serena, R. Sztajzel, P. Ruchat, et al. Automatic Classification of HITS Into Artifacts or Solid or Gaseous Emboli by a Wavelet Representation Combined With Dual-Gate TCD Stroke, December 1, 2001; 32(12): 2803 - 2809. [Abstract] [Full Text] [PDF]
|
|
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||