How Does Self-Reported History of Stroke Compare to Hospitalization Data in a Population-Based Survey in New Zealand?
Background and Purpose—There is mixed evidence concerning the validity of self-reported history of stroke in population-based studies. We aimed to examine the validity of self-reported stroke using hospitalization with a primary diagnosis of stroke as the reference group.
Methods—Self-reported history of stroke was taken from the Survey of Families, Income, and Employment (N=18 950; 2004–2005) and defined as a respondent answering yes to the question, “Have you ever been told by a doctor that you have had a stroke?”. Survey of Families, Income, and Employment respondents consented to link their data to the New Zealand Health Information Service records of publically funded hospitalizations between 1990 and 2006. We calculated positive predictive value, sensitivity, and specificity of self-reported stroke against hospitalization for stroke.
Results—Approximately 2% of the adult Survey of Families, Income, and Employment population reported they had been told by a doctor that they had a stroke. Only 1% had evidence of hospitalization for stroke since 1990. The sensitivity of self-reported stroke was 73% and specificity was 98%. However, the positive predictive value, people who reported having a stroke with confirmation of hospitalization for stroke, was low at 29%.
Conclusions—The use of self-reported stroke will most likely overestimate the prevalence of stroke. A combination of methods is required to determine prevalence in population-based studies.
Self-administered questionnaires are often used to obtain information about history of stroke in population-based surveys.1–5 However, evidence for the validity of this approach is unclear. Self-reported history of stroke has low sensitivity and specificity when MRI is used as the gold standard,5 but it is not feasible to use brain imaging to determine past stroke in most population-based surveys. The aim of this study was to examine the validity of self-reported stroke in a general population-based survey against electronic records of hospitalization for stroke.
Materials and Methods
This study is a cross-sectional analysis utilizing data from the Survey of Families, Income, and Employment (SoFIE; wave 1 to 4 data, version 6).6 Briefly, SoFIE is a representative, longitudinal survey of the usually resident population in New Zealand that is conducted by the government agency Statistics New Zealand (2002–2010). Annual interviews are used to collect demographic data and information concerning household and family status, education level, employment status, income levels, and health. In the third year of SoFIE (wave 3, 2004–2005), a health module collected information on specific illnesses such as stroke, lifestyle factors (smoking status), and other health measures.
The initial SoFIE sample comprised ≈11 500 responding private households (a response rate of 77%). In wave 1, 22 165 adults aged 15 years or older responded, but only 18 945 responded in wave 3 (83% of wave 1), which is the current analysis sample. Demographic and socioeconomic factors were taken from the wave 3 interview. Household income, adjusted to the Consumer Price Index, and household size were categorized into quintiles. The New Zealand Deprivation (2001) index provides a neighborhood-level deprivation score grouped into quintiles.
Self-Reported History of Stroke
In the health module, respondents were asked if they had ever been told by a doctor that they had specific chronic conditions. Self-reported history of stroke was defined as a respondent answering yes to the question, “Have you ever been told by a doctor that you have had a stroke?”. An estimated age of stroke was provided.
Hospitalization for Stroke
In wave 3, 14 490 of 18 945 (76%) of the participants consented to have their results from the survey electronically linked with information collected by the New Zealand Health Information Service (NZHIS), including hospitalizations in publically funded hospitals, cancer registrations, and mortality data.7 All consenting participants were linked to their hospitalization information from 1990 to 2005 by NZHIS so that any hospitalization for a primary diagnosis of stroke could be identified on SoFIE respondents.
The validity of self-reported stroke was cross-classified assuming primary diagnosis for hospitalization for stroke as the reference group. Sensitivity, specificity, positive predictive values (PPV), and negative predictive values were calculated. The PPV was calculated by dividing the number of confirmed strokes by the total number of self-reported strokes. The numbers of respondents are random-rounded to the nearest multiple of 5, with a minimum value of 10, as per Statistics New Zealand confidentiality protocol.
In the wave 3 SoFIE population, 450 of 18 945 (2%) reported they had previously been told by a doctor that they had experienced a stroke. Of the 14 490 respondents who consented to having their data linked, 385 (3%) reported they had previously had a stroke. However, only 150 (1%) respondents actually had NZHIS evidence of hospitalization with a primary diagnosis of stroke since 1990.
The Table presents simple 2-way cross-tabulations of self-reported history of stroke compared to hospitalization with a primary diagnosis of stroke for the consenting population and those aged 65 years and older. The sensitivity (the proportion of people with evidence of hospitalization for stroke and who reported having a stroke) was 73% and the specificity was 98%. The PPV (the probability of a person reporting having experienced a stroke and actually having one using the NZHIS data) was low at 29%. Similar results were found when the population was restricted to those aged 65 years or older.
This study has shown moderate sensitivity and high specificity of self-reported history of stroke using hospitalization with a primary diagnosis of stroke as the reference. This finding supports the use of a general question such as, “have you ever been told by a doctor that you had a stroke?” to measure the prevalence of stroke in population-based surveys. The prevalence of self-reported history of stroke was similar to the 1.8% shown in the recent New Zealand Health survey (2006–2007).8 However, the PPV of self-reported stroke was low, so that 71% of the respondents who reported they had a stroke had no evidence of a hospitalization for stroke.
These findings are in contrast to that of Reitz et al,5 in which there was poor sensitivity (32%) and specificity (79%) of self-reported stroke compared to diagnosis of stroke on MRI. Earlier studies also found higher PPV (≈60%) for self-reported stroke.1–2,4,9 We had wondered if the discrepancy in PPV was explained by the older populations in the earlier studies, but our results were the same when the analysis was restricted to those aged 65 years or older.
There are a number of reasons that could potentially explain lack of concordance between self-reported stroke and hospitalization for stroke. The NZHIS linked data only goes back to 1990, so any hospitalizations for stroke before this time would be missed. However, only 16% of self-reported strokes occurred before 1990, and similar results were found in the post-1990 population. Also, most (92%) people with stroke in New Zealand are admitted to a public hospital, so a small number may be missed in the NZHIS data.10 There is also a possibility that NZHIS coding may not be completely accurate; however, we have no evidence to support or refute this because of lack of audit in this area. Finally, we were only able to link to the primary International Classification of Diseases 10th revision diagnosis for stroke, so we may have missed some people with stroke who were primarily admitted to hospital for other reasons. This may have led to increased sensitivity and PPV.11
Assuming the “true” prevalence of stroke in this population is somewhere between the values from self-reported and hospitalization for stroke, use of either value in population-based studies has potential limitations. Hospitalization data are a likely underestimate of the total population prevalence for stroke, whereas self-report may overestimate stroke prevalence. Furthermore, neither self-reports nor hospitalization data take into account those who die from stroke before hospital admission or the small number of patients who are not admitted to hospital. Therefore, prevalence data only highlight part of the true description of stroke in a population.
This study has shown that calculating the prevalence of stroke from data obtained by a question such as, “have you ever been told that you have had a stroke by a doctor?” may overestimate the prevalence of stroke. In contrast, hospital registry data may underestimate the prevalence of stroke. A combination of methods is best to estimate the prevalence of stroke in the general population.
The authors thank Professor Tony Blakely for comments on the final draft of this paper.
Sources of Funding
SoFIE Health is primarily funded by the Health Research Council of New Zealand as part of the Health Inequalities Research Programme.
Statistics New Zealand Security Statement: Access to the data used in this study was provided by Statistics New Zealand in a secure environment designed to give effect to the confidentiality provisions of the Statistics Act, 1975. The results in this study and any errors contained therein are those of the author, not Statistics New Zealand.
- Received July 26, 2010.
- Accepted August 18, 2010.
Bots ML, Looman SJ, Koudstaal PJ, Hofman A, Hoes AW, Grobbee DE. Prevalence of stroke in the general population: The Rotterdam Study. Stroke. 1996; 27: 1499–1501.
Engstad T, Bonaa KH, Viitanen M. Validity of self-reported stroke: The Tromso study. Stroke. 2000; 31: 1602–1607.
Glymour MM, Avendano M. Can self-reported strokes be used to study stroke incidence and risk factors?: evidence from the health and retirement study. Stroke. 2009; 40: 873–879.
O'Mahony PG, Dobson R, Rodgers H, James OF, Thomson RG. Validation of a population screening questionnaire to assess prevalence of stroke. Stroke. 1995; 26: 1334–1337.
Carter KN, Cronin M, Blakely T, Hayward M, Richardson K. Cohort profile: Survey of Families, Income and Employment (SoFIE) and Health Extension (SoFIE-Health). Int J Epidemiol. 2010; 39: 653–659.
Carter K, Shaw C, Hayward M, Blakely T. Understanding the determinants of consent for linkage of administrative health data with a longitudinal survey. Kōtuitui. NZ J Social Sci. 2010;(in press).
Ministry of Health. A Portrait of Health. Key results of the 2006/07 New Zealand Health Survey. Wellington: Ministry of Health 2008.
Bergmann MM, Byers T, Freedman DS, Mokdad A. Validity of self-reported diagnoses leading to hospitalization: A comparison of self-reports with hospital records in a prospective study of American adults. Am J Epidemiol. 1998; 147: 969–977.
Anderson CS, Carter KN, Hackett ML, Feigin V, Barber PA, Broad JB, Bonita R, Auckland Regional Community Stroke (ARCOS) Study Group. Trends in stroke incidence in Auckland, New Zealand, during 1981 to 2003. Stroke. 2005; 36: 2087–2093.
Benesch C, Witter DM Jr, Wilder AL, Duncan PW, Samsa GP, Matchar DB. Inaccuracy of the International Classification of Diseases (ICD-9-CM) in identifying the diagnosis of ischemic cerebrovascular disease. Neurology. 1997; 49: 660–664.