Carotid Intima Media Thickness and Plaques Can Predict the Occurrence of Ischemic Cerebrovascular Events
Background and Purpose— The clinical usefulness of noninvasive measurement of carotid intima media thickness and plaque visualization in the general population is still uncertain.
Methods— We evaluated the age-specific incidence rates of cerebrovascular events in a cohort of 1348 subjects randomly taken from the census list of San Daniele Township and followed for a mean period of 12.7 years. The association among common carotid intima media thickness, measured at baseline, arterial risk factors, and incidence of ischemic cerebrovascular events was modeled using Poisson regression. The predictive ability of common carotid intima media thickness over arterial risk factors (summarized in the Framingham Stroke Risk Score) was evaluated by receiver operating characteristic curve analysis.
Results— During the follow-up, 115 subjects developed nonfatal ischemic stroke, transient ischemic attack, or vascular death, which were the predefined study end points. After adjustment for age and sex, hypertension, diabetes, common carotid intima media thickness above 1 mm, and carotid plaques were all independent risk factors for development of vascular events. Inclusion of carotid findings (presence of common carotid intima media thickness above 1 mm or carotid plaques) resulted in a predictive power higher than Framingham Stroke Risk Score alone only on for those subjects with a Framingham Stroke Risk Score over 20%.
Conclusions— Although common carotid intima media thickness and presence of carotid plaques are known to be risk factors for the development of vascular events and to be independent from the conventional risk factors summarized in the Framingham Stroke Risk Score, their contribution to individual risk prediction is limited. Further studies will be required to address the role of carotid ultrasonography in the primary prevention of high-risk subjects.
- asymptomatic carotid stenosis
- carotid intimal medial thickness
- carotid ultrasound
- cerebrovascular disease
- public policy
Cerebrovascular disease and the issues and disabilities related to it (transient ischemic attack, stroke, and vascular dementia) are increasing over time, particularly in blacks and the elderly,1–3 and are expected to reach fourth place in the rank order of disability-adjusted life-years by 2020.4
Because cardiovascular prevention remains the most effective tool to reduce the disability burden,5 estimation of the 10-year probability of cerebro- and cardiovascular ischemic events is the cornerstone for the individual tailoring of preventive interventions. In this regard, the Framingham Cardiovascular Risk Score and the Framingham Stroke Risk Score (FSRS) have been found to be useful tools, especially if recalibrated on the basis of risk factors of specific examined population.6,7
Ultrasonographic findings of increased carotid artery intima media thickness (C-IMT) and atherosclerotic plaques are subclinical markers of early atherosclerosis and are associated with nonmodifiable and modifiable risk factors8,9 with the occurrence of new carotid plaques10,11 and with the subsequent risk of new or recurrent stroke and myocardial infarction.12–14
However, it is still uncertain whether an abnormal carotid ultrasonography (either increased C-IMT or presence of plaques) can provide additional information over the FSRS in the prediction of cerebrovascular events. Recently, several longitudinal studies have demonstrated that an increased C-IMT can have an independent, synergistic risk prediction power for stroke and myocardial infarction.15,16
To provide additional data on the predictive value of carotid ultrasonography above and beyond conventional risk factors, we evaluated the incidence of new cerebrovascular accidents (transient ischemic attacks and stroke) in a cohort of subjects from the general population (the San Daniele Project) first examined with ultrasonography in 1990.
Materials and Methods
Details of the cohort selection and enrollment have already been reported.17 Briefly, in 1990, 1348 subjects, 630 men and 718 women aged 18 to 99 years, all living in the town of San Daniele (northeastern Italy), were randomly selected from the census list and examined.
Anthropometric and clinical chemistry measurements were performed as previously detailed.17 Subjects were considered to be hypertensive if they had systolic and diastolic blood pressure ≥160 and 95 mm Hg, respectively, and/or if they were using antihypertensive drugs. Subjects having systolic blood pressure between 140 and 159 mm Hg or diastolic blood pressure between 90 and 94 mm Hg were considered to be borderline hypertensive. Subjects with borderline hypertension were conservatively evaluated in the subgroup of normal subjects in all analyses. Low-density lipoprotein cholesterol was calculated by the Friedewald formula. Subjects were considered to be diabetic if they reported a medical history of diabetes or use of antidiabetic drugs or if they had a fasting plasma glucose level ≥126 mg/dL (7.0 mmol/L). Body mass index was considered as increased if greater than 25 kg/m2.
At baseline (1990), the C-IMT was measured offline on the far wall of the common carotid artery in a longitudinal view in a region free of plaque using a computerized system. We measured C-IMT in 3 different points of the common carotid artery and we considered normal or pathological finding if the mean maximum value was <1, 1, or >1 mm.
The plaques were measured everywhere; the site of carotid plaque was computed according to the distance from the flow divider: above flow divider up 1.5 cm, internal carotid artery; below flow divider down to 1.5 cm, bifurcation; below flow divider down to 3 cm, common carotid artery.
The plaques were considered stenotic if the percentage of lumen obstruction in the projection showing the greatest impairment was >40%. A Doppler spectrum analysis was performed and the sonograms were recorded at the site of maximal stenosis. The hemodynamic criterion adopted was a value of peak systolic velocity >120 mm/s.17
The reproducibility of the ultrasonographic procedures has been previously described.18,19
End Points and Cohort Follow-Up
All subjects enrolled in the study were followed using the regional (Friuli-Venezia Giulia) health database, an information system reporting demographic, hospital discharge, and mortality data for all resident citizens, each of whom is assigned an individual identification code. The following International Classification of Diseases, 9th Revision discharge diagnosis or mortality codes were considered as possible end points: 410.0, 414.0, 414.8, 414.9, 425.4, 427.5, 428.0, 428.1, 429.0, 429.1, 429.2, 436.0, 437.0, 437.1, 438.0, 440.9, 441.4, 441.5, 444.2, 401.9, 402.9, 415.1.
Each possible event was validated according to the MONICA standardized procedures20 by review of pertinent clinical records. To improve follow-up accuracy, a second survey on survivors of the original cohort was performed in 2001 to 2002,21 and data on nonfatal events further checked by direct investigation by one member of the team. In this second survey, all the clinical and laboratory procedures were repeated and a new ultrasonographic examination was carried out by the same trained physician to measure C-IMT and evaluate carotid plaques using a specially designed software (Intelligence in Medical Technologies, Paris, France). The ultrasonographic evaluation was performed as prescribed by the Mannheim Intima-Media Consensus.22 However, only data from the baseline intima media thickness measurement were used for the present study.
Continuous variables are expressed as either mean values (±SD) or median values (with interquartile range); the categorical variables are expressed as proportions. A composite end point formed by nonfatal ischemic stroke, transient ischemic attack, or all-causes vascular death was considered as the study outcome in the follow-up analysis. The outcome incidence (as defined by the occurrence of the composite end point) was computed in subjects with no prior history of cerebro- or cardiovascular events at the time of the first investigation and expressed as the ratio of outcomes per 1000 patient-years of observation (time to event or end of the study). Because the incidence of the chosen end point increases with age, age-adjusted incidence rates were obtained using Poisson regression and weighing the contribution of each subject to the regression in 25-year intervals.23
Subsequently, to verify if carotid findings could add anything to classical risk factors from the clinical practice point of view, traditional risk factors were collapsed into a widely used score, the FSRS. To avoid any systematic distortion of risk, we recalibrated the original Framingham function taking the β coefficients from the Framingham Study model24 and mean values from the San Daniele cohort to predict the mean incidence rates.7 This recalibration procedure performed much better than the recalibrated PROCAM score in the Italian population.25 Because electrocardiography was not available at baseline, data for atrial fibrillation and left ventricular hypertrophy were not used for all study subjects. The ability to predict the composite end point at 10 years from enrollment was evaluated by comparing the receiver operating characteristic (ROC) area of a model using the FSRS alone or FSRS and carotid findings using logistic estimates to fit the ROC model.
All computations were performed using the Stata 10 statistical package.26
At baseline, 1348 people, aged 18 to 99 years, participated in the evaluation: 630 men, mean age 46.4±16.8 years, and 718 women, mean age 49.4±18.2 years. The mean follow-up was 12.7 years (SD, 0.34). During the follow-up period, 150 subjects died (81 men and 69 women), 74 of nonvascular causes and 76 of vascular causes, whereas one man and 4 women migrated out of the region. Thus, 1193 subjects, 548 men and 645 women, were still available for reinvestigation in 2001 to 2002; among these survivors, there was a compliance rate of 72.9% and 870 subjects were duly re-examined. The study outcome occurred in 115 subjects (58 men and 57 women), who experienced nonfatal ischemic strokes (nonfatal ischemic strokes or transient ischemic attacks, 39 subjects) and 76 vascular deaths. The Table reports the baseline characteristics of this population according to the occurrence of the study outcome.
At the univariate analysis, of 1249 participants who had a validated ultrasonographic scanning, there was a significant difference in the 12-year, age-adjusted incidence of the composite end point to the carotid findings (Figure 1). Among the 955 subjects with normal carotid scan, 24 events (2.51%) occurred versus 19 events (15.45%) and 48 events (28.07%) in those, respectively, presenting a carotid intima media thickness above 1 mm and at least one plaque (P<0.001). The multivariate Poisson regression showed, after adjustment for the differences in age-specific incidence rates, presence of hypertension and diabetes, that carotid intima media thickness above 1 mm or carotid plaques were all independent risk factors of occurrence of cerebral ischemia or vascular death, representing the predefined study outcome (Figure 2).
The predictive value of the recalibrated Framingham Stroke Risk Score (rFSRS) alone was furthermore compared with rFSRS combined with carotid findings. In a first analysis, the age-adjusted incidence rate was evaluated in subjects having a rFSRS below 5% (low-risk subjects), between 5% and 20% (intermediate-risk subjects), and above 20% (high-risk subjects) stratified by carotid findings (Figure 3). There was a constant increase of the incidence rate in subjects with carotid intima media thickness above 1 mm and at least one plaque as compared with subjects with no carotid lesion for all rFSRS strata. However, the incidence rates were statistically higher only for the presence of at least one carotid plaque as compared with subjects with normal carotid findings in the intermediate- (rate 22.5 versus 9.1 per 1000 patient-years, respectively, P=0.04) and high-risk (rate 38.1 versus 10.0 per 1000 patient-years, respectively, P=0.02) rFSRS categories. Although comparison of the ROC curves of the prediction model, including the FSRS alone or the FSRS combined with carotid ultrasonography, showed no significant difference in the areas below the models (P=0.26; Figure 4). Inclusion of the carotid findings resulted in a higher predictive power in a subgroup analysis evaluating the high-risk rFRSR subgroup (ROC area 0.62 versus 0.52, P=0.02).
In this article, we evaluated a cohort of 1348 subjects, randomly taken from the general population and followed for more than 10 years, to assess the predictive value of carotid ultrasound assessment over traditional risk factors. Prediction of cerebrovascular accidents in asymptomatic subjects is usually accomplished by using the FSRS,7,24 but its use is limited because different populations can present different mean levels of cardiovascular risk factors.27 Although recalibration of the FSRS could partly overcome this problem, tools that could more precisely define the cerebrovascular risk at an individual level would be very valuable.
Ultrasound assessment of carotid arteries is a simple and noninvasive method to accurately evaluate early carotid atherosclerosis by measuring intima media thickness or by direct visualization of an atherosclerotic plaque. The C-IMT measurement may be considered a marker of early atherosclerosis, because it can predict both future occurrence of plaque11,28 and of coronary and cerebrovascular events.13,29 Therefore, we hypothesized that ultrasound assessment of carotid arteries could be useful for cerebrovascular risk stratification in individual subjects above and beyond the conventional risk factors represented by the FSRS.
Our work demonstrates that ultrasound carotid assessment performed in a general population can add some significant information to general risk estimates derived from the rFSRS alone. In particular, both at univariate and multivariate analysis, the presence of carotid abnormalities was associated with an increased risk of cerebrovascular ischemic events. In a multivariate model adjusting for age and all conventional risk factors, the relative risk (as incidence ratio) was 5.6 (95% CI, 3.2 to 10.1) and 10.4 (95% CI, 6.4 to 17.1), respectively, in subjects with a C-IMT >1 mm or with at least one plaque at baseline versus subjects with normal carotid arteries.
The absolute event rate for stroke and vascular death per 1000 patient-years in our population is 5.9 (95% CI, 4.9 to 7.1); the Atherosclerosis Risk in Communities (ARIC study) reported a stroke rate of 2.4,30 whereas the Cardiovascular Heath Study (CHS) reported an event rate of 10.2 in an older population.13 In a population sample with a mean age comparable with our study (the Rotterdam study), Bots et al found that the event rate for stroke was 11.3.12 The Malmo Diet and Cancer (MDCS) and Carotid Atherosclerosis Progression (CAPS) studies reported a stroke rate of 2.4 and 5.0, respectively.31,32 Finally, in a cohort of elderly Japanese men, Kitamura et al observed a stroke event rate of 5.9 during a follow-up of 4.5 years.33 These 6 population-based longitudinal studies in the United States, Europe, and Japan showed a consistent, positive association between common carotid artery intima media thickness and incidence of stroke16 with an adjusted hazard ratio for stroke per 1-SD increase of common carotid artery intima media thickness ranging from 1.15 (95% CI, 1.0 to 1.32) to 1.26 (95% CI, 1.16 to 1.42) in the CAPS and CHS studies, respectively.
The San Daniele population can be considered at low risk for stroke as demonstrated by the MONICA Stroke Project, which included the Friuli area, which is where San Daniele is located.34
In the San Daniele cohort, a strong correlation between an increased C-IMT thickness at baseline and the occurrence of plaque was observed.11 In accordance with the results of GENIC study,15 we observed a significant correlation among common carotid artery thickening, cardiovascular risk factors (and hence the recalibrated FSRS), and the risk of stroke. However, these studies show that a CCA-IMT >0.855/1 mm or the presence of carotid plaques significantly increase the ischemic risk, whatever the rFSRS value. In particular, Figure 3 clearly shows that in subjects with an intermediate- or high-risk rFSRS, the presence of a carotid plaque could result in a 2- to 4-fold increased risk of events as compared with normal subjects belonging to the same rFSRS category.
Although both C-IMT above 1 mm and carotid plaques were found to be associated with increased risk of cerebrovascular events, ROC analysis shows that they do not significantly add to the risk profile in the general population but could be useful only in high-risk subgroups. This finding is possibly explained by the very good predictive value of the rFSRS alone (area under the curve 0.86) with a consequent lack of improvement of the c-statistic even by a variable significant in multivariate analysis.35
Improving the predictability of rFSRS would require the addition of variables that are both strongly associated with events, but also prevalent in the population.36 This requirement is not fulfilled by carotid ultrasonography in the general population, because an abnormal ultrasound (either increased intima media thickness or plaque) was present in only 41 of 847 (4.8%) of those subjects with a favorable rFSRS (below 5%, or low-risk subjects). These subjects constituted the vast majority of the original cohort (847 of 1348 [62.8%]) and in this low-risk category, carotid ultrasonography could not be considered for prediction of cerebrovascular events.
Conversely, an abnormal carotid ultrasonography could be potentially useful in high-risk subjects such as those with a rFSRS above 20 as demonstrated by ROC analysis in this subgroup. Subjects with a high-risk rFSRS constituted 207 of 1348 (15.3%) of our population, and 3 of 4 of them had an abnormal carotid finding (156 of 207 [75.3%]).
In this category, subjects with a carotid plaque have an event rate of 38.1 per 1000 patient-years versus 10 per 1000 patient-years of those without carotid abnormalities. Thus, the combined use of the rFSRS and carotid ultrasonography would be expected to identify a subgroup of the population with a very high risk of vascular events.
Our decision to measure separately common carotid artery intima media thickness and plaque presence enables us to better address the situation of the subjects with increased common carotid artery intima media thickness or plaques (Mannheim Consensus 2004 to 2006).
Ultimately, however, the usefulness of such an approach would depend on the efficacy of preventive therapies to be adopted in this very high-risk subgroup. Even assuming that very stringent prophylactic treatments could reduce the rate of event by 50% in those with detectable carotid abnormalities and a high rFSRS, hence with an event rate around 20 per 1000 patient-years, the absolute risk reduction would be 2 per 100 patient-years, and the number of subjects needed to test to be able to prevent one event would be approximately 50.
The present study presents both limitations and strengths. One obvious limitation is that in 1992, plaque evaluation was already possible but no continuous common carotid artery intima media thickness measurement was undertaken, thus rendering with less precision the quantitative evaluation of the relation between C-IMT and the risk of ischemic events, a point interestingly made by some authors.16 Another possible limitation is that we did not include in the FSRS data regarding atrial fibrillation and left ventricular hypertrophy. However, it is biologically implausible that carotid echography could predict cardiogenic strokes, and therefore we do not believe that this limitation could significantly lower the predictability of carotid echography. On the other hand, this longitudinal observation of the general population has a long follow-up period. The ARIC study examined a large cohort of men and women aged 45 to 64 years for a follow-up of 10.7 years30; the CHS followed for 6.3 years a cohort of subjects older than 65 years without previous stroke or myocardial infarction13; the MDCS studied a 46- to 68-year-old sample for 7 years31; and the CAPS study32 observed the myocardial infarction and stroke end points in a general population 19 to 90 years of age for 4.2 years.
In conclusion, this study demonstrates that the presence of an increased C-IMT and/or plaques on carotid ultrasonography is associated with the occurrence of cerebrovascular ischemic events, independently from the rFSRS. The study furthermore shows that at present, a carotid ultrasonography screening policy is unwarranted in the general population, but could be considered in subjects with intermediate/high rFSRS to better stratify their actual risk. However, due to the relatively high number of subjects needed to be tested and the efficacy of the currently available prophylactic treatments in subjects identified by carotid ultrasonography, further testing of such a screening policy is required with appropriate intervention trials.
The authors wish to thank the non-profit Association against stroke of Udine (ALICE-UD Associazione per la Lotta contro l'lctus Cerebrale) for the help and support given to this research project.
- Received December 2, 2007.
- Revision received January 22, 2008.
- Accepted February 8, 2008.
Kissela B, Schneider A, Kleindorfer D, Khoury J, Miller R, Alwell K, Woo D, Szaflarski J, Gebel J, Moomaw C, Pancioli A, Jauch E, Shukla R, Broderick J. Stroke in a biracial population: the excess burden of stroke among blacks. Stroke. 2004; 35: 426–431.
Marini C, Baldassarre M, Russo T, De Santis F, Sacco S, Ciancarelli I, Carolei A. Burden of first-ever ischemic stroke in the oldest old: evidence from a population-based study. Neurology. 2004; 62: 77–81.
Goldstein LB, Adams R, Alberts MJ, Appel LJ, Brass LM, Bushnell CD, Culebras A, Degraba TJ, Gorelick PB, Guyton JR, Hart RG, Howard G, Kelly-Hayes M, Nixon JV, Sacco RL; American Heart Association/American Stroke Association Stroke Council; Atherosclerotic Peripheral Vascular Disease Interdisciplinary Working Group; Cardiovascular Nursing Council; Clinical Cardiology Council; Nutrition, Physical Activity, and Metabolism Council; Quality of Care and Outcomes Research Interdisciplinary Working Group; American Academy of Neurology. Primary prevention of ischemic stroke: a guideline from the American Heart Association/American Stroke Association Stroke Council: cosponsored by the Atherosclerotic Peripheral Vascular Disease Interdisciplinary Working Group; Cardiovascular Nursing Council; Clinical Cardiology Council; Nutrition, Physical Activity, and Metabolism Council; and the Quality of Care and Outcomes Research Interdisciplinary Working Group: the American Academy of Neurology affirms the value of this guideline. Stroke. 2006; 37: 1583–1633.
Coppola WG, Whincup PH, Papacosta O, Walker M, Ebrahim S. Scoring system to identify men at high risk of stroke: a strategy for general practice. Br J Gen Pract. 1995; 45: 185–189.
Chambless LE, Heiss G, Folsom AR, Rosamond W, Szklo M, Sharrett AR, Clegg LX. Association of coronary heart disease incidence with carotid arterial wall thickness and major risk factors: the Atherosclerosis Risk in Communities (ARIC) Study, 1987–1993. Am J Epidemiol. 1997; 146: 483–494.
Ebrahim S, Papacosta O, Whincup P, Wannamethee G, Walker M, Nicolaides AN, Dhanjil S, Griffin M, Belcaro G, Rumley A, Lowe GD. Carotid plaque, intima media thickness, cardiovascular risk factors, and prevalent cardiovascular disease in men and women: the British Regional Heart Study. Stroke. 1999; 30: 841–850.
Zureik M, Touboul PJ, Bonithon-Kopp C, Courbon D, Berr C, Leroux C, Ducimetiere P. Cross-sectional and 4-year longitudinal associations between brachial pulse pressure and common carotid intima-media thickness in a general population. The EVA study. Stroke. 1999; 30: 550–555.
Bots ML, Hoes AW, Koudstaal PJ, Hofman A, Grobbee DE. Common carotid intima-media thickness and risk of stroke and myocardial infarction: the Rotterdam Study. Circulation. 1997; 96: 1432–1437.
Tsivgoulis G, Vemmos K, Papamichael C, Spengos K, Manios E, Stamatelopoulos K, Vassilopoulos D, Zakopoulos N. Common carotid artery intima-media thickness and the risk of stroke recurrence. Stroke. 2006; 37: 1913–1916.
Touboul PJ, Labreuche J, Vicaut E, Amarenco P; GENIC Investigators. Carotid intima-media thickness, plaques, and Framingham risk score as independent determinants of stroke risk. Stroke. 2005; 36: 1741–1745.
Lorenz MW, Markus HS, Bots ML, Rosvall M, Sitzer M. Prediction of clinical cardiovascular events with carotid intima-media thickness: a systematic review and meta-analysis. Circulation. 2007; 115: 459–467.
Prati P, Vanuzzo D, Casaroli M, Di Chiara A, De Biasi F, Feruglio GA, Touboul PJ. Prevalence and determinants of carotid atherosclerosis in a general population. Stroke. 1992; 23: 1705–1711.
Project WHO MONICA Manual (1998–1999).
Touboul PJ, Hennerici MG, Meairs S, Adams H, Amarenco P, Bornstein N, Csiba L, Desvarieux M, Ebrahim S, Fatar M, Hernandez Hernandez R, Jaff M, Kownator S, Prati P, Rundek T, Sitzer M, Schminke U, Tardif JC. Mannheim carotid intima-media thickness consensus (2004–2006). An update on behalf of the Advisory Board of the III and IV Watching the Risk Symposium, 13th and 15th European Stroke Conferences, Mannheim, Germany, 2004, and Brussels, Belgium, 2006. Cerebrovasc Dis. 2007; 23: 75–80.
Clayton DG, Hills M. Statistical Models in Epidemiology. Oxford: Oxford University Press; 1993.
Wolf PA, D'Agostino RB, Belanger AJ, Kannel WB. Probability of stroke: a risk profile from the Framingham Study. Stroke. 1991; 22: 312–318.
Ferrario M, Chiodini P, Chambless LE, Cesana G, Vanuzzo D, Panico S, Sega R, Pilotto L, Palmieri L, Giampaoli S; CUORE Project Research Group. Prediction of coronary events in a low incidence population. Assessing accuracy of the CUORE Cohort Study prediction equation. Int J Epidemiol. 2005; 34: 413–421.
StataCorp. Stata Statistical Software: Release 10.0. 2007.
Brindle P, Emberson J, Lampe F, Walker M, Whincup P, Fahey T, Ebrahim S. Predictive accuracy of the Framingham coronary risk score in British men: prospective cohort study. BMJ. 2003; 327: 1267.
Zureik M, Ducimetiere P, Touboul PJ, Courbon D, Bonithon-Kopp C, Berr C, Magne C. Common carotid intima-media thickness predicts occurrence of carotid atherosclerotic plaques: longitudinal results from the Aging Vascular Study (EVA) study. Arterioscler Thromb Vasc Biol. 2000; 20: 1622–1629.
Chambless LE, Folsom AR, Clegg LX, Sharrett AR, Shahar E, Nieto FJ, Rosamond WD, Evans G. Carotid wall thickness is predictive of incident clinical stroke: the Atherosclerosis Risk in Communities (ARIC) study. Am J Epidemiol. 2000; 151: 478–487.
Lorenz MW, von Kegler S, Steinmetz H, Markus HS, Sitzer M. Carotid intima-media thickening indicates a higher vascular risk across a wide age range: prospective data from the Carotid Atherosclerosis Progression Study (CAPS). Stroke. 2006; 37: 87–92.
Kitamura A, Iso H, Imano H, Ohira T, Okada T, Sato S, Kiyama M, Tanigawa T, Yamagishi K, Shimamoto T. Carotid intima-media thickness and plaque characteristics as a risk factor for stroke in Japanese elderly men. Stroke. 2004; 35: 2788–2794.
Tolonen H, Mähönen M, Asplund K, Rastenyte D, Kuulasmaa K, Vanuzzo D, Tuomilehto J. Do trends in population levels of blood pressure and other cardiovascular risk factors explain trends in stroke event rates? Comparisons of 15 populations in 9 countries within the WHO MONICA Stroke Project. World Health Organization Monitoring of Trends and Determinants in Cardiovascular Disease. Stroke. 2002; 33: 2367–2375.
Cook NR. Use and misuse of the receiver operating characteristic curve in risk prediction. Circulation. 2007; 115: 928–935.
Wald NJ, Hackshaw AK, Frost CD. When can a risk factor be used as a worthwhile screening test? BMJ. 1999; 319: 1562–1565.