Montreal Cognitive Assessment 5-Minute Protocol Is a Brief, Valid, Reliable, and Feasible Cognitive Screen for Telephone Administration
Background and Purpose—The National Institute of Neurological Disorders and Stroke-Canadian Stroke Network Vascular Cognitive Impairment Harmonization working group proposed a brief cognitive protocol for screening of vascular cognitive impairment. We investigated the validity, reliability, and feasibility of the Montreal Cognitive Assessment 5-minute protocol (MoCA 5-minute protocol) administered over the telephone.
Methods—Four items examining attention, verbal learning and memory, executive functions/language, and orientation were extracted from the MoCA to form the MoCA 5-minute protocol. One hundred four patients with stroke or transient ischemic attack, including 53 with normal cognition (Clinical Dementia Rating, 0) and 51 with cognitive impairment (Clinical Dementia Rating, 0.5 or 1), were administered the MoCA in clinic and a month later, the MoCA 5-minute protocol over the telephone.
Results—Administration of the MoCA 5-minute protocol took 5 minutes over the telephone. Total score of the MoCA 5-minute protocol correlated negatively with age (r=−0.36; P<0.001) and positively with years of education (r=0.41; P<0.001) but not with sex (ρ=0.03; P=0.773). Total scores of the MoCA and MoCA 5-minute protocol were highly correlated (r=0.87; P<0.001). The MoCA 5-minute protocol performed equally well as the MoCA in differentiating patients with cognitive impairment from those without (areas under receiver operating characteristics curve for MoCA 5-minute protocol, 0.78; MoCA=0.74; P>0.05 for difference; Cohen d for group difference, 0.80–1.13). It differentiated cognitively impaired patients with executive domain impairment from those without (areas under receiver operating characteristics curve, 0.89; P<0.001; Cohen d=1.7 for group difference). Thirty-day test–retest reliability was excellent (intraclass correlation coefficient, 0.89).
Conclusions—The MoCA 5-minute protocol is a free, valid, and reliable cognitive screen for stroke and transient ischemic attack. It is brief and highly feasible for telephone administration.
Vascular cognitive impairment (VCI) refers to cognitive dysfunction with an underlying vascular cause. It is common in the aging population and is particularly prevalent in patients with stroke.1,2 Cognitive assessment is integral in the diagnosis and management of VCI. Screening instruments for VCI should be sensitive to mild impairment and neuropsychological features of VCI. In 2005, the National Institute of Neurological Disorders and Stroke-Canadian Stroke Network (NINDS-CSN) VCI Harmonization Working Group recommended a set of 3 neuropsychological protocols to serve different purposes in VCI assessment. The 60- and the 30-minute protocols were designed for detailed cognitive profiling and clinical screening, whereas a brief protocol (≈5 minutes) was proposed to serve as a screen at bedside, busy clinics, and over the telephone.1 The latter protocol aims to increase access for cognitive assessment for patients living in remote areas or for those not able to attend clinical follow-up for various reasons (eg, homebound) and to support telemedicine stroke service and large epidemiological and clinical research.
The Montreal Cognitive Assessment (MoCA) has been recommended as a clinical screening instrument for VCI. It is valid and reliable in the patients with VCI, including stroke, subarachnoid hemorrhage, and stroke-free patients with vascular risk factors.3–6 However, its paper-and-pencil test format requires the examinee be physically present for examination. It is also too lengthy (≈15 minutes) to be used as a brief screen. More than 3 abbreviations of the MoCA have been developed and examined in patients with stroke (Table 1).4,7 In accordance with the structure of the 5-minute protocol proposed by the NINDS-CNS Harmonization workshop, we constructed the MoCA 5-minute protocol by extracting 4 subtests from the MoCA. The objective of this study is to examine the validity and reliability of the MoCA 5-minute protocol administered over the telephone.
Participants are patients with stroke or transient ischemic attack (TIA) recruited in the ongoing STroke Registry Investigating Cognitive DEcline (STRIDE) study,8 which is an ongoing 5-year longitudinal study to evaluate the rate and predictors of delayed cognitive decline in a consecutive cohort of 1013 patients (mean age, 69.6 [SD=11.7] years; 44% women) admitted to a major regional hospital in Hong Kong for stroke and TIA between 2009 and 2010. Patients with prestroke dementia were excluded from the STRIDE study. In the current substudy, participants were selected through stratified random sampling by cognitive status based on Clinical Dementia Rating (CDR).9 Patients with moderate or severe dementia, as defined by CDR≥2, were excluded. Patients had adequate sensorimotor and language capacity to complete cognitive testing, as determined at entry into the STRIDE study. Approval from the Chinese University of Hong Kong-New Territories East Cluster Ethics Committee was obtained for the STRIDE study, and all patients gave written informed consent for participation. Proxy consents (adult children for 4 patients, spouse for 1) were obtained in patients with dementia who lacked capacity to give informed consent. Study assessments were administered in the Cantonese language.
Description of the MoCA 5-Minute Protocol
The MoCA 5-minute protocol consists of 4 subtests examining 5 cognitive domains, including attention, verbal learning and memory, executive functions/language, and orientation. In this study, the MoCA 5-minute protocol was derived from the Hong Kong version of the MoCA.5 These items are selected for domain specificity for VCI, brevity, and feasibility for administration over the telephone. The stimuli of 4 test items remained the same in the MoCA 5-minute protocol. Several modifications were made with regards to the scoring of these items. First, the number of words recalled in the first immediate recall trial of the 5-word learning was adopted as a measure of immediate auditory attention span. Second, outputs in verbal fluency test were scored incrementally instead of categorically using a single predetermined cut off. Note that while phonemic fluency was proposed in the English version,1 semantic fluency (animal) was used in the study because the Cantonese language is based on a nonalphabetic system.3 Third, different weightings were given to free recall and cued recall or recognition to reflect different types of memory failure (ie, encoding versus retrieval deficits). Total scores of the MoCA 5-minute protocol range between 0 and 30. Table 2 shows a description of the MoCA 5-minute protocol.
Clinical Assessment of Cognitive Functions
As a part of their annual clinical assessment in the STRIDE study, participants were administered the Hong Kong version of the MoCA5 and the Cantonese Mini-Mental State Examination10 for psychometric evaluation of cognitive functions. The CDR was assessed using a semistructure interview format with informants to provide a global rating of cognitive status based on clinical history.9 On the basis of CDR score, we classified participants as cognitively normal (CDR=0) or impaired (CDR=0.5 or CDR=1).
Administration of MoCA 5-Minute Protocol Over Telephone
Patients who agreed to participate were administered the MoCA 5-minute protocol over the telephone 1 month post clinical visit. Test order was the same for all participants. Before the administration on the MoCA 5-minute protocol, participants were advised to turn off the radio and television and to go to a quiet area for the test. Whenever possible, family members were explained the purpose of the test and were encouraged to remove any distractions and aids (eg, wall calendars) before the test administration on the phone. Performance on the MoCA 5-minute protocol was not considered in the rating of the CDR.
Bland–Altman analysis was used to calculate the mean difference and 95% limits of agreement between the total scores of the MoCA and MoCA 5-minute protocol. Interscale agreement was indexed by the mean difference and limits of agreement (ie, the smaller the better agreement). Pearson correlation coefficient (r) was also used to examine the correlations between the total scores of the MoCA 5-minute protocol with age, education, and total MoCA scores. Relationship between MoCA 5-minute protocol and sex was examined with Spearman correlation coefficient (ρ). ANCOVA was used to compare scores of the Mini-Mental State Examination, MoCA, and MoCA 5-minute protocol between the cognitively normal and impaired groups with age and education adjusted. The Cohen d statistic was used as a measure of effect size of group difference. Cohen d ≥0.8 is classified as large effect size.11 To examine the external validity of the MoCA 5-minute protocol in detecting cognitive impairment, receiver operating characteristics curve was used to examine the ability of the 2 tests to differentiate the cognitively impaired group from the cognitively normal group. Areas under receiver operating characteristics curve of total scores of the MoCA and MoCA 5-minute protocol were compared.12
We also calculated the executive and memory domain scores of the MoCA according to the method by Lam et al.13 To determine the ability of the 5-minute protocol in screening for executive impairment, cognitively impaired patients (ie, CDR≥0.5) was further categorized based on the presence of executive impairment, which is defined as 1.5 SD below the mean of MoCA executive domain score in 50 healthy controls with comparable demographics (mean age, 69.2 years; 50% women; education, 7.3 years) in our previous study.5 We compared the 5-minute protocol score between cognitively impaired patients with and without executive impairment using ANCOVA with age and education adjusted. Furthermore, we performed a receiver operating characteristics curve analysis to examine the ability of the 5-minute protocol to differentiate patients with executive impairment from those without. Moreover, to evaluate the advantage of the weighted scoring for delayed memory, Pearson correlation was calculated between the delayed memory score and spontaneous recall (without adding score for correct cued recall and recognition) of the MoCA 5-minute protocol with the memory domain score of the MoCA.13
Thirty participants completed a second MoCA 5-minute protocol over the telephone 30 days after the first administration to assess the test–retest reliability indexed by the intraclass correlation coefficient. Internal consistency of the 4 items of the MoCA 5-minute protocol was measured by the Cronbach α.14 Statistical analyses were performed using SPSS Statistics version 21.
One hundred four patients with ischemic stroke or TIA participated in the study. Mean number of days between index hospital admission and clinical assessment was 39.4 (SD=7.6) months. Table 3 shows the demographic and clinical features and cognitive performance of the participants. Approximately half of the participants (n=51; 49%) had cognitive impairment. Cognitively normal and impaired groups were comparable in age, years of education, sex, and stroke severity, as measured by the National Institute of Health Stroke Scale.
In general, administration of the MoCA 5-minute protocol and the MoCA took 5 and 12 minutes, respectively. Total score of the MoCA 5-minute protocol negatively correlated with age (r=−0.36; P<0.001) and positively with years of education (r=0.41; P<0.001), but not with sex (ρ=0.03; P=0.773). Bland–Altman analysis showed that the total scores were on average ≈1.8 points higher on the MoCA 5-minute protocol than on the MoCA (Figure 1). The 95% limits of agreement between the MoCA and MoCA 5-minute protocol (calculated as MoCA total−MoCA 5-minute protocol total) were between −7.4 and 3.8. Figure 2 shows a scatter plot of total scores of MoCA and 5-minute protocol for the whole sample. The total scores of the 2 tests were highly correlated (r=0.87; P<0.001). When compared with the cognitively normal group, the cognitively impaired group performed more poorly in the MoCA and MoCA 5-minute protocol, as well as in all items scores of the MoCA 5-minute protocol (P<0.001 for all). Cohen d ranged between 0.80 and 1.13 for the total and item scores of the MoCA 5-minute protocol and 0.97 for the MoCA total score. Areas under receiver operating characteristics curves for the MoCA and MoCA 5-minute protocol were 0.74 (SE=0.05) and 0.78 (SE=0.05), respectively, and not significantly different from each other (z=0.64; P>0.05). Excluding patients with dementia (CDR=1; n=5) resulted in a slightly lower areas under receiver operating characteristics curve (0.75 [0.05]). Eighteen (35%) of 51 patients with cognitive impairment had impaired executive domain score on the MoCA. Those with executive domain impairment on the MoCA had significantly lower scores on the MoCA 5-minute protocol (MoCA 5-minute protocol mean=15.6 [SD=5.2]) than those without (23.6 [SD=4.2]; P=0.001; Cohen’s d=1.7). Areas under receiver operating characteristics curve for the MoCA 5-minute protocol to differentiate the group with executive domain impairment was 0.89 (SE=0.045; P<0.001). The delayed memory score of the MoCA 5-minute protocol had slightly higher correlation (r=0.73; P<0.001) than the spontaneous recall score (r=0.68; P<0.001) with the MoCA memory domain score.
Thirty participants completed a second MoCA 5-minute protocol 31.7 (6.7) days after the first administration of the MoCA 5-minute protocol. Test–retest reliability was excellent (intraclass correlation coefficient, 0.89; 95% confidence interval, 0.77–0.95; P<0.001). Cronbach α measuring internal consistency of the 4 items of the MoCA 5-minute protocol was 0.79.
In this study, we showed that the MoCA 5-minute protocol administered over telephone was equally useful as the clinically administered MoCA in detecting cognitive impairment in patients with stroke or TIA. Excluding patients with mild dementia (n=5) only resulted in a minimal reduction in diagnostic accuracy. Interscale agreement between the 2 tests was also good. Effect size between the cognitively normal and impaired groups was even slightly larger for the MoCA 5-minute protocol than for the MoCA (Cohen d=1.15 [MoCA 5-minute protocol] versus 0.97 [MoCA]). In addition, although it seems that the MoCA 5-minute is weighted toward memory and orientation, we showed that it is highly accurate in detecting patients with executive impairment. The MoCA 5-minute protocol also exhibited excellent test–retest reliability and acceptable internal consistency.
Telephone-based cognitive tests serve to increase accessibility for testing and minimize missing data because of inability to attend clinical visits, thus offering multiple benefits for clinical and research purposes. Many telephone-based tests have been developed for clinical use and epidemiological studies.4,15,16 The MoCA was recommended by the NINDS-CSN VCI working group as the choice of test for VCI screening. It has demonstrated good psychometric properties across a wide range of VCI-related conditions and is free for clinical and research use (at the time of this writing). The MoCA 5-minute protocol was derived from the MoCA by extracting items with domain specificity for VCI, brevity and feasibility for telephone administration. Several abbreviated versions of the MoCA have been developed (Table 1). For example, Mai et al7 showed that a combination of clock drawing, delayed recall, and abstraction items performed better than the combination proposed by the NINDS-CSN working group in predicting impaired MoCA performance. However, their version precludes the possibility for telephone administration as drawing is required. Pendlebury et al4 developed 2 telephone versions of MoCA of different lengths, including a short version that is composed of the set of items recommended by NINDS-CSN working group. This short version included phonemic fluency, word recall, and orientation with a maximum score of 12. It was shown to be valid in detecting patients with stroke or TIA and mild cognitive dysfunction. The MoCA 5-minute protocol exhibits several strengths over other abbreviated MoCA versions. First, the MoCA 5-minute protocol capitalizes on the performance of immediate word recall as an extra measure of auditory attention span. Second, verbal fluency performance in the MoCA 5-minute protocol is scored incrementally instead of dichotomously, which enables a continuous and more representative weighting of this item in the overall score. Despite the fact that executive functioning was assessed based on 1 item, namely category fluency, it is a sensitive test to frontal lesions17 and our data showed that the MoCA 5-minute protocol is effective in detecting executive impairment. Third, by retaining category cueing and recognition in the delayed memory, the MoCA 5-minute protocol offers important information on the type of memory failure that may help to differentiate subtypes of VCI. We showed that when compared with the spontaneous recall score,4,7 this modified scoring method correlated better with the MoCA memory domain score. Finally, the MoCA 5-minute protocol retained a 30-point scoring, which allows a broader range of scores for more precise clinical measurement and for statistical analysis. This 30-point scoring is also in line with other common cognitive screens, such as the MoCA and Mini-Mental State Examination. Nevertheless, it important to keep in mind that scores between the MoCA 5-minute protocol and the MoCA are not equivalent for interpretative purpose because of differences in test makeup and scoring methods. The telephone versions of MoCA of Pendlebury et al4 take longer to administer but might offer an advantage over the MoCA 5-minute protocol in longitudinal studies, where direct comparison of item performance can be made with the paper-and-pencil MoCA. The MoCA 5-minute protocol best serves as a brief clinical screen that offers additional clinical information, such as type of memory failure (ie, encoding versus retrieval), that may help more accurate diagnosis.
The MoCA 5-minute protocol is highly feasible for telephone administration. From our experience, it is important to keep administration time as brief as possible to minimize environmental factors, such as distractions from television and family members, that may invalidate test performance. We found it useful to explain the purpose of the test and to seek their cooperation in test preparation to family members, for example, to remove small children from the room and calendars from the wall. Family members are also asked not to help the patient in answering the test questions. As a limitation inherent to all telephone-based tests, individuals with hearing impairment may not be suitable for testing with the MoCA 5-minute protocol, and hence screening of hearing problem is suggested before administering the test.
There are several limitations in our study. First, participants’ performance on the MoCA 5-minute protocol may be biased by the previous exposure to the MoCA, and the order of the MoCA and MoCA 5-minute protocol was not counterbalanced. Yet, we tried to minimize this influence using a longer interval of 30 days between the administrations of MoCA and the MoCA 5-minute protocol. Second, the inclusion of a few participants with mild dementia (n=5) may have slightly inflated the performance of the MoCA 5-minute protocol in a group comparison and receiver operating characteristics curve analysis.18 However, we showed that the reduction in diagnostic accuracy was only minimal when patients with dementia were excluded. Likewise, we did not include patients with more severe dementia; therefore, the psychometric properties of the MoCA 5-minute protocol toward this end of cognitive spectrum and in other causes would need to be further investigated. Third, undetected subtle hearing problem might have lowered the MoCA 5-minute score and contributed to reduced concordance between performance in MoCA and MoCA 5-minute protocol in our sample. Fourth, in this study, we did not derive any single cutoff score or education adjustment for the total score of the MoCA 5-minute protocol because a much larger sample is needed to generate age- and education-corrected norms. Additional studies are needed for standardization of norms for the MoCA 5-minute protocol, and to compare its performance with other telephone-based cognitive tests such as the Telephone Interview for Cognitive Status.15 Last but not least, despite the participants in study were recruited from a large cohort of consecutive patients admitted for stroke and TIA, results of this study may not generalize to patients with different cultural and language backgrounds.
In conclusion, our study shows that the MoCA 5-minute protocol is a brief, valid, and reliable cognitive test that performed equally well as the MoCA in screening patients with stroke or TIA having cognitive impairment. It is highly feasible for administration over the telephone.
Sources of Funding
This study was supported by funding from Chinese University of Hong Kong Neurology Research and National Institute of Health and Ageing (reference number 1R01 NS057514).
Dr Nasreddine owns the copyright of the Montreal Cognitive Assessment (MoCA) and the MoCA 5-minute Protocol. The other authors report no conflicts.
Guest Editor for this article was Costantino Iadecola, MD.
- Received August 27, 2014.
- Revision received January 5, 2015.
- Accepted January 26, 2015.
- © 2015 American Heart Association, Inc.
- Hachinski V,
- Iadecola C,
- Petersen RC,
- Breteler MM,
- Nyenhuis DL,
- Black SE,
- et al
- Wong A,
- Xiong YY,
- Wang D,
- Lin S,
- Chu WW,
- Kwan PW,
- et al
- Pendlebury ST,
- Welch SJ,
- Cuthbertson FC,
- Mariz J,
- Mehta Z,
- Rothwell PM
- Wong GK,
- Lam S,
- Ngai K,
- Wong A,
- Mok V,
- Poon WS
- Chiu HFK,
- Kee HC,
- Chung WS,
- Kwong PK
- Cohen J
- Bland JM,
- Altman DG
- Knopman DS,
- Roberts RO,
- Geda YE,
- Pankratz VS,
- Christianson TJ,
- Petersen RC,
- et al
- Wilson RS,
- Leurgans SE,
- Foroud TM,
- Sweet RA,
- Graff-Radford N,
- Mayeux R,
- et al