MoCA, ACE-R, and MMSE Versus the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Vascular Cognitive Impairment Harmonization Standards Neuropsychological Battery After TIA and Stroke
Background and Purpose—The Montreal Cognitive Assessment (MoCA) and Addenbrooke's Cognitive Examination–Revised (ACE-R) are proposed as short cognitive tests for use after stroke, but there are few published validations against a neuropsychological battery. We studied the relationship between MoCA, ACE-R, Mini-Mental State Examination (MMSE) and mild cognitive impairment (MCI) in patients with cerebrovascular disease and mild cognitive impairment (MCI).
Methods—One hundred consecutive non-institutionalized patients had the MMSE, MoCA, ACE-R, and National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Vascular Cognitive Impairment Harmonization Standards Neuropsychological Battery ≥1 year after transient ischemic attack or stroke in a population-based study. MCI was diagnosed using modified Petersen criteria in which subjective cognitive complaint is not required (equivalent to cognitive impairment–no dementia) and subtyped by number and type of cognitive domains affected.
Results—Among 91 nondemented subjects completing neuropsychological testing (mean/SD age, 73.4/11.6 years; 44% female; 56% stroke), 39 (42%) had MCI (amnestic multiple domain=10, nonamnestic multiple domain=9, nonamnestic single domain=19, amnestic single domain=1). Sensitivity and specificity for MCI were optimal with MoCA <25 (sensitivity=77%, specificity=83%) and ACE-R <94 (sensitivity=83%, specificity=73%). Both tests detected amnestic MCI better than nonamnestic single-domain impairment. MMSE only achieved sensitivity >70% at a cutoff of <29, mainly due to relative insensitivity to single-domain impairment.
Conclusions—The MoCA and ACE-R had good sensitivity and specificity for MCI defined using the Neurological Disorders and Stroke–Canadian Stroke Network Vascular Cognitive Impairment Battery ≥1 year after transient ischemic attack and stroke, whereas the MMSE showed a ceiling effect. However, optimal cutoffs will depend on use for screening (high sensitivity) or diagnosis (high specificity). Lack of timed measures of processing speed may explain the relative insensitivity of the MoCA and ACE-R to single nonmemory domain impairment.
Stroke doubles the risk of dementia in epidemiological studies and rates of dementia in the first year after stroke are high particularly after recurrent stroke.1,2 Mild cognitive impairment (MCI) is also common after stroke and is associated with increased risk of dementia.1,3,4 However, lengthy neuropsychological batteries are often not feasible in routine practice or large-scale studies and there is thus a need for short tests of cognition that are sensitive to MCI and to the frontal/executive deficits that are prominent in vascular cognitive impairment.
Two recently developed short tests of cognition, the Montreal Cognitive Assessment (MoCA 30 point test)5 and the Addenbrooke's Cognitive Examination–Revised6 (ACE-R 100-point test in which the Mini-Mental State Examination [MMSE]7 is embedded) include tests of frontal lobe function (executive and attentional tasks) and were designed to be sensitive to MCI in a nonvascular setting. Preliminary studies in stable cerebrovascular disease suggest that the MoCA is more sensitive to MCI than the MMSE8 but the MoCA cutoff of <26 out of 30 for MCI derived from a memory clinic population5 may not be appropriate in a population with cerebrovascular disease. The ACE-R has also been proposed as a useful cognitive outcome measure in stroke but there are no published data.
There is no consensus on the definition or the criteria for MCI because there are uncertainties in delineating the boundaries between normal cognitive function and MCI and between MCI and dementia with at least 18 different terms and definitions in current use.9–12 We used the modified Petersen criteria9,11 (in which subjective memory complaint is not required as in cognitive impairment–no dementia1,3,4), which allow subtyping of MCI by number and types of cognitive domains affected.
Our hypothesis was that the MoCA and the ACE-R would be more sensitive for MCI than the MMSE but that the MoCA and ACE-R would perform similarly in view of the cognitive domains covered and level of detail. We therefore aimed to determine the sensitivities and specificities of the MoCA, ACE-R, and MMSE at ≥1 year after transient ischemic attack (TIA) or stroke for detection of MCI identified with the neuropsychological battery recommended in the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Vascular Cognitive Impairment Harmonization Standards working group.13
Patients were participants in the Oxford Vascular Study (OXVASC 2002), a prospective population-based cohort study of all acute vascular events occurring within a defined population of approximately 91 000.14,15 The study was approved by the local ethics committee and informed consent was obtained. Between August 2009 and November 2010, consecutive patients attending the OXVASC clinic were invited at their routine 1- or 5-year follow-up to undergo further cognitive testing with the ACE-R6 and the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Harmonization Standards Neuropsychological Battery13 in addition to the MMSE,7 MoCA,5 Barthel,16 and modified Rankin Scale score,17 which were done routinely at the follow-up appointment. Further cognitive testing was performed by investigators (S.T.P., J.M., L.B.) who did not undertake the routine follow-up and were blinded to the MMSE and MoCA results. Nursing home residents and patients who had problems that interfered with testing such as poor vision, severe hearing impairment, inability to use the right arm, dysphasia, poor English, or acute illness were excluded.
The neuropsychological battery tests frontal/executive, attentional, language, visuospatial, and memory domains and took approximately 50 to 60 minutes to administer. Specific tests were: (1) Trail Test (Parts A and B)18,19; (2) Symbol Digit Modalities Test20; (3) Boston Naming Test (30-item version)19,21; (4) Rey-Osterrieth complex Figure copy22,23; (5) Hopkins Verbal Learning Test-Revised24,25; and (6) Letter (Controlled Oral Word Association Test)19,26 and category (animals) fluency.27
Depression was assessed using the short-form Geriatric Depression Score.28
For MCI diagnosis,9,11 the subject had to be impaired (≥1.5 SD) on at least 1 cognitive domain compared with age- and education-matched published norms,19,20,23,25 had no impairment of basic functional activities of daily living as measured by the Barthel Index, and did not fulfil the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition dementia diagnostic criteria.29 Four subtypes of MCI were distinguished11: (1) amnestic single-domain: objective impairment of memory only; (2) amnestic multiple-domain: memory and at least 1 other cognitive domain impaired; (3) nonamnestic single-domain: 1 single domain other than memory impaired; and (4) nonamnestic multiple-domain: at least 2 cognitive domains impaired but not memory.
Rates of MCI were also determined using the original Petersen criteria in which presence of a subjective memory complaint is required, using the question: “Do you think you have more problems with your memory than most?” The original Petersen criteria are widely used in memory clinics and Alzheimer disease in which memory impairment is prominent but are likely to underestimate cognitive impairment in which nonmemory domains are preferentially affected.9,10
For MoCA, ACE-R, and MMSE, mean raw and percent (as percent of the maximum possible score) subtest scores were calculated and z scores were derived by converting the mean raw score and SD to the standard normal distribution with mean 0 and SD 1 (lower z scores indicating greater discrimination between subjects).8
Years of education, and modified Rankin Scale scores were dichotomized as follows: < versus ≥12 years of education, and modified Rankin Scale <3 (nondependent) versus ≥3 (dependent).
Level of agreement between MCI and MoCA, ACE-R, and MMSE and scores were assessed using the area under the receiver operating characteristic curves (c statistic). Sensitivities, specificities, positive predictive values, and negative predictive values for various MoCA and ACE-R cutoffs for identifying MCI were determined. Significance levels for ORs were calculated using the χ2 test.
Among 100 consecutive patients (mean age, 73.4/11.6 years; 44% female), 9 subjects had incomplete neuropsychology data (5=dementia, 2=poor vision, 1=severe residual hemiparesis, 1=declined part of testing) and a further 3 did not have the ACE-R. Of the 91 subjects with complete neuropsychology data, there were 63% (n=57) with <12 years education, 56% (n=51) stroke (40 first and 11 recurrent), 52% (n=47) at 1-year follow-up, and 89% (n=81) with modified Rankin Scale <3 with mean MMSE 27/3.3 and mean MoCA 22.7/4.9.
MMSE and ACE-R scores were skewed toward higher values (median and interquartile range, 28 [26–29] and 93 [86–96]), whereas MoCA scores were normally distributed (23 [20–26]). The MoCA and ACE-R were strongly correlated (Spearman r2=0.76, P<0.0001; Figure). Individual subtests of the MoCA and ACE-R are described in Table 1 together with the mean and z scores. All MoCA subtests and most ACE-R subtests discriminated well between subjects.
In the neuropsychological battery, more patients performed below the cutoff (≥1.5 SD below published norms) on visuospatial and executive/attentional tasks than on memory, language, and naming tasks (Table 2). Patients with TIA and stroke were similar in age, education level, and sex distribution but compared with those with TIA and patients with stroke had lower mean MMSE, MoCA, ACE-R, and memory (Hopkins Verbal Learning Test) scores with a trend to worse performance on the Symbol Digit Modalities Test, Trails B, and verbal fluency (Table 2).
Thirty-nine of 91 (43%) nondemented subjects who completed neuropsychological testing had a diagnosis of MCI. Half of the MCI cases (20 of 39) had single-domain impairment, the vast majority of which was in a nonmemory domain: nonamnestic single-domain=19, amnestic single-domain=1, nonamnestic multiple-domain=9, amnestic multiple-domain=10. C statistics for MCI were: MoCA= 0.85 (95% CI, 0.78–0.93), ACE-R=0.90 (0.83–0.96), and MMSE=0.83 (0.75–0.92).
Optimal sensitivities and specificities for MCI were achieved for MoCA cutoffs of approximately 25 to 26 (MoCA <25, sensitivity=77%, specificity=83%; MoCA <26, sensitivity=87%, specificity=63%) and ACE-R cutoffs between 92 to 94 (ACE <92, sensitivity=72%, specificity=79%; ACE-R<94, sensitivity=83%, specificity=73%; Table 3). Sensitivity of the MMSE for MCI was relatively low, being >70% only at a cutoff of <29. Restricting analyses to multiple-domain MCI showed similar predictive values for the MoCA, ACE-R, and MMSE: MoCA <25, sensitivity=89%, 95% CI, 67 to 99, specificity=69 (57–80), positive predictive value=0.44 (0.28–0.60), negative predictive value=0.96 (0.88–1.0); ACE-R <92, sensitivity=88 (64–99), specificity=69 (56–79), positive predictive value=0.42 (0.26–0.59), negative predictive value=0.96 (0.86–0.99); MMSE <28, sensitivity=79 (54–94), specificity=78 (66–87), positive predictive value=0.48 (0.30–0.67), negative predictive value=0.93 (0.84–0.98).
The MoCA detected all cases of amnestic MCI but missed 9 and 5 cases of nonamnestic MCI at cutoffs of <25 and <26, respectively (Table 4). Most cases of nonamnestic MCI that were missed by the MoCA were of single- rather than multiple-domain impairment. Results were qualitatively similar for ACE-R <94 and <92 (Table 4). Nine MCI subjects had MMSE ≥29 and 14 had MMSE ≥28 of whom 4 had multiple-domain impairment (Table 4).
Thirteen of 86 (16%) subjects completing the Geriatric Depression Scale had abnormal scores indicating possible depression. There was a trend toward increased likelihood of abnormal Geriatric Depression Scale in subjects with MCI versus those without: 8 of 34 versus 5 of 52 (OR, 2.89; 0.87–9.76; P=0.087). MCI associated with high Geriatric Depression Scale score was nearly always multiple-domain (7 of 8; Table 4).
Agreement between objective cognitive impairment and subjective memory complaint was poor (κ=0.27; 0.08–0.47; P=0.006): only 17 of the 39 patients with objective cognitive impairment had a subjective memory complaint and therefore met criteria for MCI by the original Petersen criteria (Table 5). Nine subjects with a subjective memory complaint did not have any objective cognitive impairment.
The National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Harmonization Standards Neuropsychological Battery was feasible in the majority of OXVASC community-dwelling patients tested at least 1 year after TIA or stroke, although approximately 10% were unable to complete all tests. Rates of MCI defined using modified Petersen criteria (subjective memory impairment not required) were high with nonamnestic single-domain and multiple-domain impairment most common. Both the MoCA and the ACE-R had good sensitivity and specificity for MCI thus defined.
The pattern of cognitive deficits in our study with rarity of isolated memory impairment and prominence of slowed processing speed and visuoexecutive deficits is characteristic of vascular cognitive impairment3 and suggests that the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Neuropsychological Battery, although relatively short, covers the relevant cognitive domains effectively. Cognitive profiles were qualitatively similar in patients with TIA and stroke, although more abnormalities were seen after stroke, particularly in memory. There was a trend toward more abnormal depression scores in the MCI group, particularly in those with multidomain impairment, consistent with the recognized association between depression and vascular cognitive impairment.30
It is important to note that although neuropsychological testing is considered the “gold standard” for identifying cognitive impairment, there is no consensus on how such data are used to diagnose MCI.9–12 The sensitivities and specificities obtained for different MoCA, ACE-R, and MMSE cutoffs will therefore be highly dependent on how MCI is defined as well as on other factors such as case-mix. The Petersen criteria class single-domain impairment as MCI, whereas other methods require at least 2 domains to be impaired and the threshold for abnormal cognitive domain function ranges from ≥1 to ≥2 SD below normal. Furthermore, the requirement for subjective memory decline affects estimates9,10 as seen also in our study; less than half of those with objective deficits had subjective memory complaint and were thus classed as MCI by the standard Petersen criteria. Finally, in distinguishing MCI from dementia, the Barthel Index may underestimate functional impairment owing to ceiling effects and thus some patients with MCI in our study may have been classed as having dementia using alternative functional criteria. Distinguishing MCI from dementia is of less relevance in which the aim is to detect any cognitive impairment regardless of severity.
Our data confirm the previously observed ceiling effect for the MMSE31 that only cutoffs of <29 or greater had sensitivities for MCI of >70% with MMSE <27 having a sensitivity of only 50%, although MMSE sensitivity was greater for multidomain impairment. Both the MoCA and ACE-R performed well in detecting MCI including single-domain impairment. The optimal MoCA cutoff was lower than seen in our study when measured at a mean of 6 days after stroke32 against neuropsychological testing performed 2 to 3 weeks later, probably due in part to different definitions of cognitive impairment and/or to effects of delirium and acute illness. Our study used similar criteria for MCI (Petersen criteria) to the original MoCA study on a memory clinic cohort5 and our results are broadly consistent despite the different clinical characteristics of the patients.
Our results suggest that the MoCA and the ACE-R are similarly useful in measuring cognitive outcomes in stable cerebrovascular disease; most items on both tests discriminated well between subjects, although our study was not powered to detect small differences in performance. Our data should inform such calculations in future studies. Although both the MoCA and ACE-R had excellent sensitivity for amnestic impairment, sensitivity to single-domain nonamnestic impairment was less good, possibly because of the lack of timed tasks needed to measure reduced information processing speed. Both the MoCA and ACE-R contain similar visuoexecutive tasks, although the MoCA also includes abstraction and has more attentional tests, whereas the ACE-R contains more language and memory items. The ACE-R takes a few minutes longer to administer than the MoCA alone but includes the MMSE within it. Choice of cutoff will depend on whether the test is being used as a screen (high sensitivity required) or as a diagnostic tool (high specificity required).
In conclusion, the MoCA and the ACE-R had good sensitivity and specificity for MCI defined using modified Petersen criteria with the National Institute of Neurological Disorders and Stroke–Canadian Stroke Network Harmonization Standards Neuropsychological Battery in patients with stable cerebrovascular disease, but the MMSE had lower sensitivity for single-domain MCI. However, all 3 tests performed similarly in detecting multidomain impairments within the limits imposed by the relatively small numbers in our study. Both the MoCA and ACE-R are short, feasible tests suitable for routine clinical practice and for large studies of stroke outcome. Further longitudinal studies are required to determine the prognostic value of the MoCA and ACE-R for the development of dementia after TIA and stroke.
Sources of Funding
The Oxford Vascular Study is funded by the UK Stroke Association, the Dunhill Medical Trust, the National Institute of Health Research (NIHR), the Medical Research Council, the NIHR Biomedical Research Centre, Oxford, and the Wellcome Trust. S.T.P. is supported by the NIHR Biomedical Research Centre, Oxford. P.M.R. is an NIHR Senior investigator and a Wellcome Trust Senior Investigator.
The Oxford Vascular Study was approved by the Oxfordshire clinical research ethics committee (CO.043).
- Received July 22, 2011.
- Revision received September 23, 2011.
- Accepted October 14, 2011.
- © 2012 American Heart Association, Inc.
- Narasimhalu K,
- Ang S,
- De Silva DA,
- Wong MC,
- Chang HM,
- Chia KS,
- et al
- Narasimhalu K,
- Ang S,
- De Silva DA,
- Wong MC,
- Chang HM,
- Chia KS,
- et al
- Pendlebury ST,
- Cuthbertson FC,
- Welch SJ,
- Mehta Z,
- Rothwell PM
- Fisk JD,
- Merry HR,
- Rockwood K
- Hachinski V,
- Iadecola C,
- Petersen RC,
- Breteler MM,
- Nyenhuis DL,
- Black SE,
- et al
- Rothwell PM,
- Coull AJ,
- Silver LE,
- Fairhead JF,
- Giles MF,
- Lovelock CE,
- et al
- Loewen SC,
- Anderson BA
- Smith A
- Fasteneau PS,
- Denburg NL,
- Hufford BJ
- Benton AL,
- Hamsher K
- Isaacs B,
- Kennie AT
- Brink TL
- Sheikh JA,
- Yesavage JA
American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-IV), ed 4. Washington, DC: American Psychiatric Association, 24th International Standard Classification;1994.
- Godefroy O,
- Fickl A,
- Roussel M,
- Auribault C,
- Bugnicourt JM,
- Lamy C,
- et al