Published online before print October 16, 2008, doi: 10.1161/STROKEAHA.107.513341
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2009;40:230.)
© 2009 American Heart Association, Inc.
Original Contributions |
Cerebral Microemboli and Brain Injury During Carotid Artery Endarterectomy and Stenting
From the Departments of Neurology (M.S., D.R.), Thoracic and Cardiovascular Surgery (K.K.-S.), and Radiology (B.T., S.J.B.), Rikshospitalet University Hospital, Oslo, Norway; and the Department of Medical Engineering (R.B.), University of Applied Sciences, Ulm, Germany.
Correspondence to Mona Skjelland, MD, Department of Neurology, Rikshospitalet Universty Hospital, 0027 Oslo, Norway. E-mail mona.skjelland{at}rikshospitalet.no
 |
Abstract |
|---|
 Top Abstract IntroductionSubjects and MethodsResultsDiscussionConclusionsReferences |
|---|
Background and Purpose— Cerebral microembolic signals detected by transcranial Doppler are frequent during carotid angioplasty with stenting and carotid endarterectomy (CEA). Their potential harmful effects on the brain are, however, unclear. The aim of this study was to relate the frequency and type of per-procedural microembolic signals to procedure-related ipsilateral ischemic strokes and new ipsilateral ischemic lesions on diffusion-weighted cerebral MRI.
Methods— Eighty-five patients who were prospectively treated with CEA (61) or carotid angioplasty with stenting (30) for high-grade (
70%) internal carotid artery stenoses were monitored during the procedures using multifrequency transcranial Doppler with embolus detection and differentiation. Pre- and postprocedural cerebral diffusion-weighted cerebral MRIs were performed on a subset of patients.
Results— Solid and gaseous microemboli were independently associated with procedure-related ipsilateral ischemic strokes (solid: P=0.027, gaseous: P=0.037) or new ipsilateral diffusion-weighted cerebral MRI lesions (solid: P=0.043, gaseous: P=0.026). Microembolic signals were detected during all procedures except one (CEA); 17% and 21% of all emboli were solid during carotid angioplasty with stenting and CEA, respectively. Patients undergoing carotid angioplasty with stenting had more solid (P<0.001) and gaseous (P<0.001) emboli and more new ipsilateral ischemic strokes (P=0.033) compared with patients undergoing CEA. Echolucent plaques (P=0.020) and preprocedural diffusion-weighted cerebral MRI ischemic lesions (P=0.002) were associated with increased numbers of solid emboli.
Conclusions— Solid and gaseous microemboli were increased in patients with procedure-related ipsilateral ischemic strokes or new diffusion-weighted cerebral MRI lesions, which suggests that both solid and gaseous emboli may be harmful to the brain during CEA and carotid angioplasty with stenting.
Key Words: embolic stroke • embolism • endarterectomy • endovascular treatment • TCD • transcranial Doppler
|
Introduction |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionConclusionsReferences |
|---|
Cerebral microembolic signals (MES) are frequent during both carotid endarterectomy (CEA) and carotid angioplasty with stenting (CAS).1–3 Some authors have found that these emboli may predict cerebrovascular symptoms,4–6 whereas this was not confirmed by others.2,3,7,8 Evidence suggesting that per-procedural microemboli may in themselves damage the brain has also been inconclusive. One study found an association between MES and new cerebral ischemic lesion demonstrated on diffusion-weighted MRI (DWI),6 whereas others did not.2,8,9 The aim of this study was therefore to assess the frequency and type of per-procedural cerebral microemboli during CEA and CAS and to relate these emboli to ipsilateral ischemic stroke or new ipsilateral cerebral ischemic lesions on DWI.
|
Subjects and Methods |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionConclusionsReferences |
|---|
Patients
Eighty-five consecutive patients with 91 high-grade carotid artery stenoses with diameter reductions
70% (median 80%) were prospectively included in the study. Seventy-five patients (82%) were men and the mean age of all patients was 66 years (SD, 8.9; range, 42 to 83). Sixty-one (67%) stenoses were treated with CEA and 30 (33%) with CAS. All patients had a color duplex ultrasound examination of the precerebral vessels, CT angiography of the carotid and intracranial arteries, and a clinical neurological examination (by a neurologist) within 2 days before and after the procedure. One of the goals of the study was to examine patients with DWI within 2 days before and after the procedure. However, this proved to be impossible in all patients due to capacity limitations at the department of neuroradiology at our hospital. All patients who experienced symptoms during or after the procedure were examined with DWI after the procedure. In 73 (80%) of the 91 procedures, cerebral DWI was performed within 2 days before the procedure; DWI was repeated within 2 days after 58 (79%) of these 73 procedures. A stenosis was classified as symptomatic if the patient had experienced ipsilateral cerebrovascular or ocular ischemic symptoms within 180 days before treatment confirmed by cerebral CT or MRI. According to this definition, 51 (56%) of the stenoses were symptomatic. The 40 asymptomatic stenoses were treated for the following reasons: prophylactic operations before coronary artery bypass grafting (9 stenoses [22.5%]), occlusion/high-grade stenoses of the contralateral carotid artery with reduced collateral cerebral circulation (26 stenosis [65%]), and ipsilateral symptoms more than 6 months before the procedure (5 stenosis [12.5%]). Transcranial Doppler, intracerebral MR angiography, MR perfusion images, or single photon emission computerized tomography was used to assess the patient’s intracerebral collateral circulation. The criteria for procedure selection were those which we routinely use at our hospital. These criteria were established from available recommendations based on the CT angiographic appearance, color duplex ultrasound results, and clinical evaluation.10 The CT angiographic features that we considered prohibitive for CAS were marked calcification and vascular kinking or tortuosity. High bifurcations, tandem lesions, restenosis after CEA, prior neck radiation therapy, and high surgical risk (planned coronary artery bypass grafting within 2 weeks or severe pulmonary disease) favored CAS treatment. The 2 treatment groups were similar with regard to baseline characteristics (Table 1). All patients gave informed consent before the study and the study was approved by the regional ethics committee.
|
Carotid Endarterectomy
Sixty (98.4%) patients undergoing CEA were shunted during surgery and one (1.6%) operation was performed without a shunt. Two patients (3.3%) were given a patch (Dacron). All patients were treated with at least one antiplatelet agent (75 to 160 mg acetylsalicylic acid and/or 75 mg clopidogrel daily) before surgery and this treatment was continued after the procedure. Heparin (5000 IU intravenously) was given routinely before clamping the carotid artery. Surgery was performed under general anesthesia by experienced vascular surgeons.
Carotid Artery Stenting
All 30 patients received dual antiplatelet agents (75 mg clopidogrel and 75 mg aspirin daily) at least 3 days before and 3 months after CAS. Heparin was given (5000 IU intravenously) before stent delivery. Five patients (16.7%) underwent predilatation of the stenoses before stent delivery. One (3.3%) procedure was performed without a protection device; the remainder was performed with distal filter devices with mesh size approximately 110 µm (EZ filter wire EX; Boston Scientific). The procedures were performed under local anesthesia through percutaneous transfemoral access by 2 experienced interventional radiologists. Ipsilateral carotid and intracranial angiography (contrast agent Visipaque 270 mg/mL; Iodixanol; GE Healthcare, AS, Oslo, Norway) was performed to assess technical success.
Ultrasound Examinations
Transcranial Doppler monitoring of the ipsilateral middle cerebral artery was carried out continuously during CEA and CAS using a multifrequency transcranial Doppler (EmboDop; DWL, Singen, Germany). Cerebral MES were automatically identified and differentiated and their time of occurrence was simultaneously registered. The criteria for the automatic detection and differentiation of cerebral MES using multifrequency transcranial Doppler were based on those described previously11,12 but refined as follows: the detection level for MES was a 7-dB power increase above background level (dEBR; embolus blood ratio), which lasted 4 ms simultaneously in both 2.0- and 2.5-MHz frequency channels and the lower dEBR detection limit for solid emboli was y=–0.1x–0.12 dB where y=dEBR and x=2.0 MHz EBR.13 The insonation and reference gate depths were 55 and 45 mm, and the sample volume 12 mm, filter setting 200 Hz, and power 188 mW. The carotid stenoses were classified according to the degree of stenosis by color Duplex ultrasound examinations (HDI 5000; Philips)14 and morphological analysis (echolucency) of the plaque was performed as described previously.15
Radiological Assessments
The carotid stenoses were classified by CT angiography according to consensus criteria.16 Cerebral MRI with DWI (diffusion-weighted images, T2, and fluid-attenuated inversion recovery) was assessed by experienced neuroradiologists, who were aware of the purpose of the study but blinded for all clinical details and the number of types of per-procedural microemboli.
Clinical Assessment
The clinical outcome measures were ipsilateral (to the procedure) cerebral or retinal ischemic stroke during or within 30 days after the procedure. Any new focal neurological deficit (caused by cerebral or retinal ischemia) that persisted for more than 24 hours was classified as an ischemic stroke or retinal infarct, whereas symptoms lasting <24 hours were classified as transitory ischemic attacks.
Statistical Analysis
The numbers of MES during the monitoring periods are shown as median and range (nonnormal distribution). Statistical analyses included descriptive statistics with mean (±SD) and median (range) for continuous variables and number (percentage) for categorical variables. For comparison of normally distributed data, we used Student t test and nonparametric tests for data with a nonnormal distribution. Pearson 
2 or Fisher exact test was used to assess associations between categorical variables. A probability value of <0.05 was considered statistically significant. All probability values were 2-sided. All calculations were carried out using the statistical software SPSS for Windows (Version 14; SPSS Inc, Chicago, Ill).
|
Results |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionConclusionsReferences |
|---|
Cerebral Microembolic Signals and Cerebrovascular Symptoms
Cerebral microemboli were detected in all 30 patients during the CAS procedures and in 60 (99%) of the 61 patients during the CEAs. A total of 8965 microemboli was detected during the 30 CAS procedures; 1561 (17%) of these were solid and 7404 (83%) gaseous. The corresponding numbers for the 61 CEA procedures were a total of 1342 emboli of which 286 (21%) were solid and 1056 were (79%) gaseous. The median number of solid emboli was 44 (range, 8 to 116) during CAS and 4 (range, 0 to 18) during CEA (P<0.001) and of gaseous, respectively, 236 (range, 27 to 524) and 13 (range, 2 to 106; P<0.001; Table 2). Most solid emboli during CAS occurred during stent delivery and during dilatation procedures, whereas gaseous emboli were seen during the introduction of catheters, which were flushed with saline and during angiograms performed with intra-arterial contrast agent. In CEA, most solid emboli occurred during the establishment of shunt flow and carotid clamp release, whereas gaseous emboli were detected during the establishment and termination phases of shunt flow.
|
There were more procedure-related ipsilateral ischemic strokes/retinal infarction in the CAS group (2 patients with minor stroke, one patient with retinal infarction) compared with the CEA group (no patients; P=0.033; Table 2). In addition, 2 patients undergoing CAS had ipsilateral hemispheric transitory ischemic attacks during the procedure. Both of these patients had a contralateral internal artery occlusion. Patients with procedure-related ipsilateral ischemic stroke/retinal infarct had more solid (70; range, 13 to 116) and more gaseous (235; range, 85 to 524) emboli compared with patients without these symptoms (6; range, 0 to 128 solid and 19; range, 2 to 510 gaseous). There was a significant association between solid (P=0.027) and gaseous (P=0.037) microemboli and procedure-related ipsilateral ischemic strokes (Table 3). Patients with echolucent plaques had higher numbers of solid (P=0.020) but not gaseous (P=0.112) emboli compared with patients with echogenic/heterogeneous plaque (Table 4).
|
|
There were no statistical associations between new ipsilateral ischemic stroke and the degree of stenosis, plaque echolucency (echolucent plaque in 67% of patients with stroke compared with 30% of patients without stroke), previous cerebrovascular symptoms, and previous ipsilateral cerebral DWI ischemia (Table 3).
Cerebral Microembolic Signals and Cerebral MRI Lesions
Fifty-five (75%) of the 73 patients who had a preprocedural cerebral DWI had ipsilateral ischemic lesions on DWI. Thirty-one (56%) of these 55 patients had experienced symptoms from the ipsilateral cerebral hemisphere during the 6-month period before the procedure. Patients with preprocedural DWI lesions had higher numbers of solid (P=0.002) but not gaseous (P=0.075) emboli compared with patients without such lesions (Table 4).
Six (21%) of the 28 patients undergoing CAS who had postprocedural DWI and 2 (7%) of the 30 patients undergoing CEA with a postprocedural DWI had new ipsilateral ischemic DWI lesions (P=0.103; Table 2). Three of these 8 patients had relevant clinical symptoms.
Patients with new ipsilateral DWI lesions had higher numbers of solid (44; range, 4 to 116) and gaseous (250; range, 16 to 524) emboli compared with patients without new ischemic lesions (9; range, 0 to 108 and 33; range, 2 to 510). There was an independent association between solid (P=0.043) and gaseous (P=0.026) microemboli and new ipsilateral DWI lesions (Table 5). There was no statistical difference in the total number of new ipsilateral ischemic DWI lesions, which was 9 in the 6 patients undergoing CAS and 11 in the 2 patients undergoing CEA (P=0.301).
|
)
|
Discussion |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionConclusionsReferences |
|---|
In this study, we found that there was an association between solid and gaseous microemboli during CAS and CEA and procedure-related ipsilateral ischemic strokes or new ipsilateral cerebral ischemic DWI lesions. This suggests that both solid and gaseous emboli may cause injury to the brain during these procedures. The median numbers of gaseous and solid emboli were, respectively, 18- and 11-fold higher during CAS compared with patients undergoing CEA and those undergoing CAS had more ipsilateral ischemic strokes. These results should, however, be interpreted with considerable caution because this study was not a randomized study, which was designed to assess the efficacy or safety of CAS compared with CEA.
Studies in patients receiving a mechanical heart valve have suggested that gaseous microemboli may not cause injury to the brain because they are small and have the ability to dissolve in the blood.17,18 The theoretical diameter limits for microemboli detected in the middle cerebral artery with the multifrequency Doppler method used in this study is approximately 3 to 40 µm for gaseous and 80 to 400 µm for solid. It would seem unlikely that solid microemboli of this size, which may consist of atherosclerotic material, platelet aggregates, blood clots, or soft fat-containing particles, can pass through the brain’s microvasculature (7 to 10 µm), whereas this may be possible for much smaller gas bubbles. During CAS, however, gaseous microemboli often occur in showers within a very short time frame after injections of saline or contrast agent. Clearance of these showers of gaseous microemboli may be impaired if there is a decreased perfusion pressure distal to the stenosis.19,20 Gaseous microemboli may also occur during phases of CEA when there is hemodynamic impairment, but their much smaller numbers may reduce the risk of brain injury due to clearance problems. Previous studies have shown that the passage of gaseous or solid microemboli through the cerebral microvasculature may also cause damage to the endothelium, which causes activation of leukocytes and inflammation. This may induce secondary ischemia and delayed minor cerebrovascular symptoms or increasing neuropsychological symptoms21,22 The potential harmful effects on the brain due to solid and gaseous microemboli may therefore be more pronounced than detected during the initial clinical examinations after the procedure.
Some authors have reported an association between total MES and stroke,4–6 whereas others have concluded that cerebral microemboli are clinically silent.2,3,7,8 This lack of agreement may be due to the use of different Doppler systems, detection criteria, lack of differentiation between solid and gaseous emboli, or different definitions of clinical outcome. Clinical outcome in some of these studies included both ischemic and hemorrhagic stroke and symptoms from both the ipsilateral and contralateral cerebral hemispheres. Two previous studies have attempted to differentiate between solid and gaseous emboli during CAS using multifrequency Doppler.9,23 However, they did not use the refined automatic differentiation criteria that were used in this study13 The numbers of patients were also small and no cerebrovascular symptoms were observed. Rosenkranz et al found that 14% of all emboli during CAS in 27 patients were solid and Chen et al found that 39% of the emboli were solid in 10 patients undergoing CAS.
Automatic embolus detection and differentiation in this study was based on refined multifrequency transcranial Doppler criteria.13 This method has 3 potential limitations, ie, sensitivity when detecting small solid emboli (<80 µm, corresponding to a EBR of <7 dB), correctly differentiating small gaseous emboli (<3 µm, corresponding to a EBR of <7 dB), and counting all of the emboli when several enter the sample volume at the same time.12 It is important to stress, however, that the criteria for automatic embolus detection and differentiation using multifrequency Doppler were clearly defined before the study and therefore were exactly the same for all patients.
Finally, we found an increased number of ipsilateral solid, but not gaseous, emboli in patients with echolucent plaques and in patients who had ipsilateral DWI lesions previous to the procedures. This suggests an association between plaque morphology and solid embolization, which supports the results of previous studies in which an association was found between plaque morphology and ischemic stroke.15,24,25
|
Conclusions |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionConclusionsReferences |
|---|
We found that there was an association between ipsilateral solid and gaseous microemboli during carotid revascularization and procedure-related ipsilateral ischemic strokes or new ipsilateral DWI lesions. These results suggest that both solid and gaseous cerebral microemboli may be harmful to the brain during CAS and CEA and that their numbers should be reduced to a minimum during these procedures.
|
Acknowledgments |
|---|
Source of Funding
This work was supported by The Norwegian Foundation for Health and Rehabilitation.
Disclosures
None.
Received January 8, 2008; revision received May 6, 2008; accepted May 20, 2008.
|
References |
|---|
|
TopAbstractIntroductionSubjects and MethodsResultsDiscussionConclusionsReferences |
|---|
1. Jordan WD. Microemboli detected by transcranial Doppler monitoring in patients during carotid angioplasty versus carotid endarterectomy. Cardiovasc Surg. 1999; 7: 33–38.[CrossRef][Medline] [Order article via Infotrieve]
2. Poppert H. Differences in number, size and location of intracranial microembolic lesions after surgical versus endovascular treatment without protection device of carotid artery stenosis. J Neurol. 2004; 251: 1198–1203.[CrossRef][Medline] [Order article via Infotrieve]
3. Crawley F. Comparison of microembolism detected by transcranial Doppler and neuropsychological sequelae of carotid surgery and percutaneous transluminal angioplasty. Stroke. 2000; 31: 1329–1334.
4. Ackerstaff RGA, Moons KGM, van de Vlasakker CJW, Moll FL, Vermeulen FEE, Algra A, Spencer MP. Association of intraoperative transcranial Doppler monitoring variables with stroke from carotid endarterectomy. Stroke. 2000; 31: 1817–1823.
5. Ackerstaff RGA, Suttorp MJ, van den Berg JC, Overtoom TT, Vos JA, Bal ET, Zanen P. Prediction of early cerebral outcome by transcranial Doppler monitoring in carotid bifurcation angioplasty and stenting. J Vasc Surg. 2005; 41: 618–624.[CrossRef][Medline] [Order article via Infotrieve]
6. Muller M, Reiche W, Langenscheidt P, Hassfeld J, Hagen T. Ischemia after carotid endarterectomy: comparison between transcranial Doppler sonography and diffusion-weighted MR imaging. AJNR Am J Neuroradiol. 2000; 21: 47–54.
7. Bossema ER, Brand N, Moll FL, Ackerstaff RGA, van Doornen LJP. Perioperative microembolism is not associated with cognitive outcome three months after carotid endarterectomy. Eur J Vasc Endovasc Surg. 2005; 29: 262–268.[CrossRef][Medline] [Order article via Infotrieve]
8. van Heesewijk HPM, Vos JA, Louwerse ES, van den Berg JC, Overtoom TTC, Ernst SMPG, Mauser HW, Moll FL, Ackerstaff RGA. New brain lesions at MR imaging after carotid angioplasty and stent placement. Radiology. 2002; 224: 361–365.
9. Rosenkranz M, Fiehler J, Niesen W, Waiblinger C, Eckert B, Wittkugel O, Kucinski T, Rother J, Zeumer H, Weiller C, Sliwka U. The amount of solid cerebral microemboli during carotid stenting does not relate to the frequency of silent ischemic lesions. AJNR Am J Neuroradiol. 2006; 27: 157–161.
10. Roubin GS, Iyer S, Halkin A, Vitek J, Brennan C. Realizing the potential of carotid artery stenting: proposed paradigms for patient selection and procedural technique. Circulation. 2006; 113: 2021–2030.
11. Russell D, Brucher R. Online automatic discrimination between solid and gaseous cerebral microemboli with the first multifrequency transcranial Doppler. Stroke. 2002; 33: 1975–1980.
12. Brucher R, Russell D. Automatic online embolus detection and artifact rejection with the first multifrequency transcranial Doppler. Stroke. 2002; 33: 1969–1974.
13. Russell D, Brucher R. Embolus detection and differentiation using multifrequency transcranial Doppler. Stroke. 2005; 36: 706.
14. Grant EG, Benson CB, Moneta GL, Alexandrov AV, Baker JD, Bluth EI, Carroll BA, Eliasziw M, Gocke J, Hertzberg BS, Katanick S, Needleman L, Pellerito J, Polak JF, Rholl KS, Wooster DL, Zierler RE. Carotid artery stenosis: gray-scale and Doppler US diagnosis—Society of Radiologists in Ultrasound Consensus Conference. Radiology. 2003; 229: 340–346.
15. Mathiesen EB, Bonaa KH, Joakimsen O. Echolucent plaques are associated with high risk of ischemic cerebrovascular events in carotid stenosis: the Tromso Study. Circulation. 2001; 103: 2171–2175.
16. Anderson GB, Ashforth R, Steinke DE, Ferdinandy R, Findlay JM. CT angiography for the detection and characterization of carotid artery bifurcation disease. Stroke. 2000; 31: 2168–2174.
17. Kofidis T, Fischer S, Leyh R, Mair H, Deckert M, Haberl R, Haverich A, Reichart B. Clinical relevance of intracranial high intensity transient signals in patients following prosthetic aortic valve replacement. Eur J Cardiothorac Surg. 2002; 21: 22–26.
18. Nadareishvili ZG, Beletsky V, Black SE, Fremes SE, Freedman M, Kurzman D, Leach L, Norris JW. Is cerebral microembolism in mechanical prosthetic heart valves clinically relevant? J Neuroimaging. 2002; 12: 310–315.[Medline] [Order article via Infotrieve]
19. Caplan LR, Hennerici M. Impaired clearance of emboli (washout) is an important link between hypoperfusion, embolism, and ischemic stroke. Arch Neurol. 1998; 55: 1475–1482.
20. Orlandi G, Fanucchi S, Gallerini S, Sonnoli C, Cosottini M, Puglioli M, Sartucci F, Murri L. Impaired clearance of microemboli and cerebrovascular symptoms during carotid stenting procedures. Arch Neurol. 2005; 62: 1208–1211.
21. Muth CM, Shank ES. Gas embolism. N Engl J Med. 2000; 342: 476–482.
22. Purandare N, Burns A, Daly KJ, Hardicre J, Morris J, Macfarlane G, McCollum C. Cerebral emboli as a potential cause of Alzheimer’s disease and vascular dementia: case–control study. BMJ. 2006; 332: 1119–1124.
23. Chen C, Iguchi Y, Garami Z, Malkoff M, Smalling R, Campbell M, Alexandrov A. Analysis of emboli during carotid stenting with distal protection device. Cerebrovasc Dis. 2005; 21: 223–228.[CrossRef]
24. Biasi GM, Froio A, Diethrich EB, Deleo G, Galimberti S, Mingazzini P, Nicolaides AN, Griffin M, Raithel D, Reid DB, Valsecchi MG. Carotid plaque echolucency increases the risk of stroke in carotid stenting: the Imaging in Carotid Angioplasty and Risk of Stroke (ICAROS) Study. Circulation. 2004; 110: 756–762.
25. Mayor I, Comelli M, Vassileva E, Burkhard P, Sztajzel R. Microembolic signals and carotid plaque morphology: a study of 71 patients with moderate or high grade carotid stenosis. Acta Neurol Scand. 2003; 108: 114–117.[CrossRef][Medline] [Order article via Infotrieve]
This article has been cited by other articles:
 |
|
 |
|
  G. Cesaroni, N. Agabiti, F. Forastiere, and C. A. Perucci Socioeconomic Differences in Stroke Incidence and Prognosis Under a Universal Healthcare System Stroke, August 1, 2009; 40(8): 2812 - 2819. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||