Multilingual Validation of the Questionnaire for Verifying Stroke-Free Status in West Africa
Background and Purpose—The Questionnaire for Verifying Stroke-Free Status (QVSFS), a method for verifying stroke-free status in participants of clinical, epidemiological, and genetic studies, has not been validated in low-income settings where populations have limited knowledge of stroke symptoms. We aimed to validate QVSFS in 3 languages, Yoruba, Hausa and Akan, for ascertainment of stroke-free status of control subjects enrolled in an on-going stroke epidemiological study in West Africa.
Methods—Data were collected using a cross-sectional study design where 384 participants were consecutively recruited from neurology and general medicine clinics of 5 tertiary referral hospitals in Nigeria and Ghana. Ascertainment of stroke status was by neurologists using structured neurological examination, review of case records, and neuroimaging (gold standard). Relative performance of QVSFS without and with pictures of stroke symptoms (pictograms) was assessed using sensitivity, specificity, positive predictive value, and negative predictive value.
Results—The overall median age of the study participants was 54 years and 48.4% were males. Of 165 stroke cases identified by gold standard, 98% were determined to have had stroke, whereas of 219 without stroke 87% were determined to be stroke-free by QVSFS. Negative predictive value of the QVSFS across the 3 languages was 0.97 (range, 0.93–1.00), sensitivity, specificity, and positive predictive value were 0.98, 0.82, and 0.80, respectively. Agreement between the questionnaire with and without the pictogram was excellent/strong with Cohen k=0.92.
Conclusions—QVSFS is a valid tool for verifying stroke-free status across culturally diverse populations in West Africa.
The 8-item Questionnaire for Verifying Stroke-Free Status (QVSFS) was developed as a tool for verifying stroke-free phenotype in participants of clinical, epidemiological, and genetic studies.1 Among English-speaking Western populations, the QVSFS demonstrated excellent diagnostic performance2,3 and has been translated into Spanish for use among Hispanic populations.4 The QVSFS is also useful for screening undiagnosed stroke. The sensitivity and specificity of having any of the 6 symptom questions for stroke detection was 0.82 and 0.62, respectively.5 This tool has been deployed in the recruitment of controls involved in genetic studies of ischemic stroke6,7 to screen for cerebrovascular end points in the Carotid Revascularisation Endarterectomy Versus Stent Trial (CREST)8 and in the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study.9
The QVSFS could have tremendous potential in developing countries where stroke prevalence is escalating, awareness of stroke symptoms is low, and neuroimaging facilities are lacking.10 Validation of the tool is thus needed in developing countries where low levels of literacy is rife and local vernacular for stroke symptoms is challenged. Our aim was to assess the performance of the QVSFS with and without stroke symptom pictograms in 3 major languages spoken across the West African subcontinent in 5 sites involved in the on-going National Institute of Health supported Stroke Investigative Research Educational Networks (SIREN) study. SIREN has an overarching objective of identifying novel and traditional vascular risk factors as well as genetic markers associated with stroke in sub-Saharan Africa.
Participants were recruited from 5 tertiary referral medical centers in West Africa namely the Komfo Anokye Teaching Hospital in Kumasi, Ghana; the University College Hospital, Ibadan; Federal Medical Centre, Abeokuta; Ahmadu Bello University Hospital, Zaria; and Aminu Kano Teaching Hospital, Kano, Nigeria. Kumasi where Komfo Anokye Teaching Hospital is situated is the principal city of the Akan tribe in central Ghana and serves a population of 4 million. University College Hospital and Federal Medical Centre situated in Ibadan and Abeokuta, respectively, are located in Southern Nigeria and serve a combined population of 4 million Nigerians of predominantly Yoruba ethnicity. Ahmadu Bello University Hospital and Aminu Kano Teaching Hospital in Zaria and Kano provide healthcare services to 15 million Nigerians of mainly Hausa and Fulani descent. All these facilities have consultant neurologists. Ethical approval for the study was sought from the ethics committees of the 5 participating sites.
Consecutive patients aged 18+ years presenting at the General Medicine and Neurology outpatient clinics were recruited after obtaining informed consent. Patients were excluded if they had severe dysphasia, cognitive or hearing impairment.
Baseline demographic characteristics were sought and the questionnaire administered in the local dialects. The original questionnaire developed by Meschia et al1 was translated into 3 local languages (Yoruba spoken at Ibadan and Abeokuta sites in Southern Nigeria, Hausa spoken at Kano and Zaria sites Northern Nigeria, and Akan spoken at Kumasi site in middle and lower belts of Ghana), pretested and back-translated into English language to establish semantic equivalence. At each of the study sites, a panel comprising a neurologist, 3 to 5 doctors and nurses, and public health practitioners translated the questionnaire into the local dialect by consensus. Another version of the questionnaire was developed where question items 3 to 8 had pictures of the neurological symptoms being elicited by the interviewer (online-only Data Supplement). At each of the study sites, the questionnaire was administered by a medicine resident in the local dialect and the answers recorded as yes, no, or don’t know to each question item.
Participants were subsequently reviewed by neurologists, who were blinded to responses of the QVSFS. The neurologist reviewed the medical records of all participants for documentary evidence of clinically or radiologically confirmed stroke, followed by a structured neurological examination to elicit the presence of hemiparesis, hemianesthesia, visual field defects, and aphasia. Findings by the neurologist were documented in a questionnaire as our gold standard.
Sample Size Calculation
The sample size for the study (n=384) was estimated under the assumption that the QVSFS test will have a 10% to 15% difference in accuracy as measured by the area under the receiver-operating characteristic curve, a false-positive rate of ≤10% and a 0.5 ratio of the SD of the responses in the control group to those in the case group as suggested by Obuchowski and McClish.11 To detect this difference, a 2-sided z test at a significance level of 0.05 achieved at least 80% power under the same conditions.
Baseline demographic characteristics of participants were compared using either χ2 test (categorical variables) or Mann–Whitney U test for medians, respectively (continuous variables). In our primary analysis, participants who scored 0 of 8 (no to all question items) were classified as stroke free. We estimated the sensitivity, specificity, positive, and negative predicted values and corresponding likelihood ratios for the performance of the questionnaire using Proc Logistic in SAS 9.4. Area under the receiver-operating characteristic curve was used to estimate the accuracy of the questionnaire across the 3 linguistic groups. The 95% confidence interval for each of the validation statistics was computed using the Clopper–Pearson exact approach in SAS PROC FREQ.12 Furthermore, the performance of the questionnaire with and without pictogram was compared using κ statistics. There were 6 participants who responded “don’t know” in the questionnaire and analysis was done assuming they were stroke free and with a sensitivity analysis assuming they had stroke. There was no difference in the analysis based on stroke-free assumption (not shown). In all analyses, statistical significance was attained if a 2-tailed P<0.05 was reached with no adjustments for multiple comparisons.
Table 1 shows the demographic characteristics of 384 study participants according to the 3 linguistic groups in which the QVSFS was validated. The overall median age of study participants was 54 years, 48.4% were males, nearly 30% had no or primary education and 5% were resident in rural communities. One hundred sixty-five (43%) participants were identified as having had stroke by neurologists involved in the study after a thorough evaluation. Participants with stroke phenotype were significantly older, more likely to be males and residents in rural domiciles compared with those without stroke with no significant differences in highest educational level achieved and employment status (Table 2). Patients with stroke were significantly more likely to report having hypertension, diabetes mellitus, hypercholesterolemia, cigarette smoking, and a family history of stroke compared with stroke-free participants. Overall, 72 (43.6%) patients with stroke versus 24 (11.0%) of stroke-free individuals were recruited from neurology clinics.
Performance of the 8-Item QVSFS
Of the 165 cases with stroke identified by our gold standard, 161 (98%) were determined by the questionnaire to have had a stroke, whereas of the 219 cases without stroke 179 (82%) were determined not have stroke by the questionnaire without the pictogram. Using the questionnaire with the pictogram, 160 of 165 (97%) participants with stroke and 190 of 219 (87%) stroke-free participants were identified. The overall negative predictive values (NPVs) of the tool for identifying stroke-free status with and without the pictogram across the 3 languages were 0.97 (range, 0.93–1.00) and 0.98 (range, 0.95–1.00), respectively. The sensitivity (proportion of participants with stroke correctly identified by the tool), specificity (proportion of participants without stroke correctly identified by the tool), and positive predictive values (PPVs; the probability of correctly identifying a patient with stroke) of the questionnaire without versus with pictogram were 0.98 versus 0.98 (P=1.00), 0.82 versus 0.85 (P=0.06), and 0.80 versus 0.86 (P=0.07). Furthermore, the positive likelihood ratio (the odds of a positive result from the questionnaire in an individual with stroke) and negative likelihood ratio (the odds of a negative result from the questionnaire in an individual with stroke) were 5.38 and 0.02, respectively.
The specificity and PPV of the QVSFS was affected by educational status being 0.67 and 0.75, respectively, among those with primary/no formal education versus 0.86 and 0.83 for those who have attained higher than primary education. The use of the pictogram significantly improved the specificity of the QVSFS from 0.67 to 0.79 (P=0.03) among participants with educational levels below primary education but not among those with higher than primary education from 0.86 to 0.89. Overall agreement between the questionnaire with and without the pictogram was excellent with Cohen k=0.92 (range, 0.90–0.96 across linguistic groups) and differences in the performance characteristics were not significant as shown in Tables 3 and 4. The area under the receiver-operating characteristic curve of QVSFS overall compared with the gold standard was 0.92 (95% confidence interval, 0.90–0.95) indicating a very good accuracy.
Shown in Table 5 is the overall performance of the tool compared with each of the individual items on the questionnaire. The sensitivity of the items on the questionnaire ranged from 0.18 for item 6 to 0.90 for item 1 with none of the individual items surpassing the 8-item QVSFS in sensitivity. The question items assessing visual field symptoms (questions 5 and 6) were the least sensitive, 0.28 and 0.18, respectively. Question items 1 and 2 that assessed history of previous stroke and transient ischemic attack diagnoses had highest specificity—0.98 and 0.97, respectively, for stroke-free phenotype compared with a global specificity of 0.82. Exclusion of question items 1 and 2 led to a drop in sensitivity and NPV to 0.92 and 0.93 but specificity and PPV were maintained at 0.82 and 0.79, respectively, compared with the 8-item QVSFS results overall. Analysis of subjects perception of stroke symptoms in the presence of clinically demonstrable neurological deficits revealed that 89.3% and 62.5% with hemiparesis and hemianesthesia, respectively, responded positively to question items 3 and 4; only 24.4% and 17.8% with mono-ocular visual loss and hemianopia responded positively to questions 5 and 6; and 26.9% and 57.7% with receptive aphasia and expressive aphasia responded “yes” to questions 7 and 8.
This is the first transnational, multicenter, and multilingual validation of the 8-item QVSFS among culturally diverse participants in the West African subcontinent. Yoruba, Hausa, and Akan are among the most commonly spoken languages in West Africa. As a tool for identifying stroke-free phenotype, we show that the 8-item QVSFS has an NPV of 0.98 and negative likelihood ratio of 0.02. Conversely, the questionnaire was able to identify stroke patients with a sensitivity of 0.98, specificity of 0.82, and PPV of 0.80. Among the 3 languages, the performance characteristics of the questionnaire were maintained with minor differences in its diagnostic performance. This confirms the excellent diagnostic accuracy of the QVSFS as a stroke symptom questionnaire and concurs largely with previous studies conducted under similar clinical settings.1–3,5
We directly compared the performance of the tool with and without pictograms (question items 3–8) for each of the participants enrolled into this study. As shown in Table 4, although the sensitivity of the 2 versions of the tool was equal, the version with the pictogram had a slightly improved specificity (0.85 versus 0.82, P=0.06) and PPV (0.86 versus 0.80, P=0.07) compared with the version without pictograms. Educational status had a significant effect on the specificity of the QVSFS with a 19% difference between those with formal/primary education compared with those with higher educational status. The discriminatory value of adding a pictogram was evidenced by an improvement in specificity of the tool from 67% to 79% (P=0.03) among participants with low educational status. These findings suggest that the performance of the questionnaire could be enhanced further by the introduction of culturally acceptable pictures of stroke symptoms. Khan et al13 have recently reported on a translation of the 8-item QVSFS into Urdu with the administration of the questionnaire among 322 community-dwelling Pakistani subjects using ecologically valid pictures and found a sensitivity, specificity, and NPV for detection of stroke of 71%, 85.8%, and 95%, respectively. However, in that study comparison of the questionnaire with and without the pictogram was not reported. In this study, pictograms were in line diagrams with a strong agreement between the 2 versions of the questionnaire.
It has been proposed that the QVSFS is an excellent tool for the selection of proper controls for epidemiological and genetic research in stroke.1,2 We found a low negative likelihood ratio of 0.02, meaning that 2 of 100 individuals with stroke/transient ischemic attack are likely to be labeled stroke free by the QVSFS. As a multiple-item questionnaire, the QVSFS has a higher accuracy than single-item questions on stroke symptoms or previous physician diagnosis of stroke.14,15 We found that none of the individual question items in the QVSFS had superior sensitivity or NPV compared with the QVSFS overall (Table 5). Question 1 that assessed previous diagnosis of stroke had the highest sensitivity (0.90), specificity (0.98), PPV (0.97), NPV (0.93), and consequently a high positive likelihood ratio of 39.29 and a negative likelihood ratio of 0.11. The implication is that even though question item 1 would be useful in providing support for the diagnosis of stroke by virtue of its high positive likelihood ratio, its usefulness is attenuated by its high negative likelihood ratio, which does not allow us to rule out the diagnosis of stroke with confidence.
In general, among the question items that evaluated stroke symptoms (questions 3–8), item 3 that assessed the presence of hemiparesis had the highest sensitivity (0.85) and NPV (0.89), whereas items 5 and 6 that evaluated mono-ocular vision loss and hemianopia, respectively, had the lowest sensitivity and NPV in agreement with previous studies.2 Indeed, we found that higher proportions of participants with stroke diagnosis by gold standard, responded yes to having symptoms of hemiparesis and hemianesthesia in the presence of clinically demonstrable neurological deficits, whereas visual and speech symptoms were poorly correlated with corresponding neurological deficits. The implications is that patients may not associate visual and language symptoms with stroke in our settings, which in turn explains the low sensitivity of question items evaluating these deficits. Sung et al5 found that the sensitivity and specificity for stroke diagnosis using questions 3 to 8 were 0.82 and 0.62 compared with 0.92 and 0.82 in this study.
The gold standard for verification of stroke status in previous studies has relied on either review of medical records or history and examination of patients by neurologists. This approach may be problematic in settings where medical records are kept under less than optimal circumstances and where patients may not always seek health services for ailments from medical facilities but from herbalists, chemical shops, and faith healers. Thus in this study, a 3-stage validation process was followed: a structured history and physical examination by neurologists at all sites, a review of medical records of all participants enrolled and then a review of cranial computed tomographic scan results where available. In addition, internal medicine residents were trained to administer the questionnaire at each site to ensure precision of questioning. Data on how many times questions were rephrased for clarification were not recorded. Although recall bias might limit the sensitivity and specificity of the questionnaire, the rigorous procedures put in place for case ascertainment would be expected to mitigate the impact of recall bias. Certainly, our results should be interpreted in the context of recruitment of participants from hospital settings. However, as previously stated, the QVSFS has been used largely in the setting of stroke research, in particular, for identifying stroke-free individuals as controls. Its applicability outside this context for stroke surveillance within the community remains a promising prospect for further evaluation in sub-Saharan Africa.
In conclusion, we have shown that the 8-item QVSFS is a simple, accurate, and cheap tool for identifying stroke-free individuals with a high degree of certainty and has a tremendous potential for ruling out stroke diagnosis in settings where computed tomographic scan may not be routinely available.
Sources of Funding
The Stroke Investigative Research Educational Networks (SIREN) study is supported by U01 NS079179, and U54 HG007479 from the National Institutes of Health as part of the H3 Africa Consortium.
Guest Editor for this article was Natan M. Bornstein, MD.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.115.010374/-/DC1.
- Received June 12, 2015.
- Revision received September 14, 2015.
- Accepted October 19, 2015.
- © 2015 American Heart Association, Inc.
- Meschia JF,
- Brott TG,
- Chukwudelunzu FE,
- Hardy J,
- Brown RD Jr,
- Meissner I,
- et al
- Jones WJ,
- Williams LS,
- Meschia JF.
- Howard G,
- Cushman M,
- Gomez CR,
- Graham A,
- Howard VJ,
- Kirk KA,
- et al
- Clopper CJ,
- Pearson ES.
- Khan M,
- Kamal AK,
- Islam M,
- Azam I,
- Virk A,
- Nasir A,
- et al
- O’Mahony PG,
- Dobson R,
- Rodgers H,
- James OF,
- Thomson RG.