Comparison Between a Standardized Questionnaire and Expert Clinicians for Capacity Assessment in Stroke Clinical Trials
Research into treatments for acute strokes has dramatically increased in the last decade. Accordingly, the need for testing through randomized clinical trials has also increased. Because of the unique combination of factors that are common in acute stroke–related research, including narrow treatment windows, ethical concerns regarding research with acute stroke populations, and capacity for informed consent, stroke clinical trial enrollment levels have remained stagnant. Given the devastating consequences of acute stroke, researchers are intensifying their efforts to recruit and enroll larger sample sizes in clinical trials.1
Capacity to consent to medical treatment or medical research is closely related to cognitive functioning, which is frequently impaired in stroke.2 Medical decision-making capacity (MDC) requires the ability to understand and appreciate diagnostic, treatment, and research information and risks and ability to express a choice that is based on adequate reasoning. The treating physician is responsible for the assessment of a patient’s decision-making capacity and clinically estimate their patient’s ability to provide informed consent.3 Many physicians request additional consultative assistance to assess cognitive capacity for consent from psychiatry or neuropsychology, which are considered to be the clinical gold standards4 or they perform standardized capacity questionnaires to aid the assessment of capacity.5,6
One such tool, the aid-to-capacity evaluation (ACE) had 80% to 89% agreement with expert clinicians in 1 study and inter-rater reliability reported as ĸ=0.79 for medical hospitalized patients.7,8 The ACE has been validated and found to be one of the best available instruments to assist clinicians in making judgments on MDC.4 It is designed to be used by trained nonclinicians and takes ≈10 minutes to perform.4 For patients who lack capacity medical decisions are deferred to a surrogate decision maker. Surrogate decision makers in acute stroke are able to accurately predict patient treatment preferences, but are significantly less accurate in predicting patient preferences for research participation.9
The development of a standardized validated tool to determine MDC has been instrumental in identifying patients who lack capacity in hospitalized nonstroke inpatients (see online-only Data Supplement). Although the ACE has been adapted for many medical conditions, it has yet to be extensively evaluated in the acute stroke population.
Presently, there is no standardized tool to evaluate decision-making capacity in stroke patients.10 The introduction of a standardized tool to rapidly determine MDC of acute stroke patients in the emergency setting would be a significant advancement and would likely increase randomized clinical trial enrollment, especially when the clinician or study team is uncertain of the patient’s cognitive status in the absence of a legal surrogate decision maker.
We hypothesized that a modified, stroke-specific, version of the ACE would show similar agreement with clinician judgment and would therefore be appropriate for rapid bedside screening for MDC.
This was a prospective, pilot study of 3 different capacity evaluations (ACE performed by a trained rater, psychiatrist assessment, and neuropsychologist assessment). All assessments were performed independently within the same day or within ±24 hours. All patients were hospitalized patients in stroke, rehabilitation, or neurological intensive care units of a single tertiary-care university medical center. The study was approved by the Institutional Committee for Protection of Human Subjects.
Patients were English speaking, ≥18-years-old, and diagnosed with either an ischemic or hemorrhagic stroke (National Institutes of Health Stroke Scale [NIHSS] score ≥1) within 10 days of symptom onset. Patients unable to hear despite assisted devices, declared legally incompetent, encephalopathic, severely lethargic or obtunded, diagnosed with dementia or severe cognitive decline, or had a current psychiatric diagnosis (schizophrenia, major depression) that would interfere with study assessment were excluded.
Patients underwent 3 independent capacity assessments: ACE, psychiatrist, and neuropsychologist. Attempts were made to perform all capacity evaluations while the patient was mentally alert to reduce the variation within the patient’s responses between examiners.
To minimize discrepancy among individual examiners, every effort was undertaken to ensure that all examiners remained consistent with their technique throughout the study duration. The clinicians and the trained rater were blinded to the results of the other’s assessments as well as methods used.
A research assistant (K.L. Feng) was trained on how to conduct the ACE and performed 3 practice evaluations under the supervision of the principal investigator (A.D. Barreto) and experienced research coordinators. The ACE questions were modified for stroke (see Figure [A] and online-only Data Supplement for further detail) by using a hypothetical scenario (Figure [B]) followed by 7 standard questions that tested the ability to (1) understand the medical problem; (2) understand the proposed treatment; (3) understand treatment alternatives; (4) understand the option of refusing treatment; (5) appreciate the reasonably foreseeable consequences; (6) appreciate foreseeable consequences of refusing the proposed treatment; and (7) make a decision that is not based on delirium or psychiatric disturbances (ie, severe depression).
A board certified psychiatrist with extensive experience in performing hospital-based capacity evaluations assessed the patient’s capability of consenting to a hypothetical scenario. This scenario concerned patient preference for new versus standard blood thinner treatments. Specific aspects of consent evaluated by the psychiatrist included (1) the ability to communicate and sustain a choice; (2) understand and describe risks, benefits, and alternatives of the proposed medication; and (3) ability to communicate his/her rationale for the choice made.
The neuropsychological examination consisted of a clinical interview and brief formal subtests chosen to evaluate comprehension and judgment. Topics ranged from simple biographical and medical information to hypothetical medical decision-making. Receptive and expressive language, attention, memory, anosognosia, and problem solving ability were assessed. This was followed by administration of the Neuropsychological Assessment Battery Judgment subtest and Complex Ideational Material and Syntactic Processing subtests from the Boston Diagnostic Aphasia Examination.11,12 These were sequentially chosen based on the patient’s ability to complete the preceding tests (ie, presence of visual or hearing impairments, etc).
Results of Pilot Testing
Thrity-one patients met study inclusion criteria and 30 agreed to participate. Sixty percent of patients were men with mild to moderate ischemic stroke (median NIHSS=6) and a mean age of 67.8 (Table 1). ACE evaluation required ≈5 to 7 minutes. Thirty-seven percent exhibited aphasia and neglect, whereas the remaining participants lacked these deficits. All 30 completed the ACE, but 2 patients were unable to be assessed by both clinicians. The ACE, neuropsychologist, and psychiatrist determined many patients lacked MDC: 70% (21/30), 52% (15/29), and 28% (8/29), respectively (Table 2).
We evaluated the sensitivity and specificity as well as positive predictive value and negative predictive value of the ACE (Table 3). The ACE demonstrated high sensitivity: 93.8% compared with neuropsychologist and 100% compared with psychiatrist. It also had a high negative predictive value to detect intact capacity versus clinicians; misclassifying only 1 patient capable when clinicians recorded incapacity (false-negative rate of 6.2%). However, it demonstrated a low specificity: 53.8% compared with neuropsychologist and 42.9% compared with psychiatrist.
The benefits of using the ACE are that unlike other instruments, the ACE uses treatment information relevant to the patient’s circumstance and is therefore specific. It specifically assesses decision-making capacity using open-ended questions, and the conclusion of the ACE evaluation capacity is reliably determined.7,13 The ACE is the only tool that has been evaluated against a gold standard and has also performed strongly on the Journal of the American Medical Association’s Rational Clinical Examination. The ACE can be used in emergency situations and can be performed in <10 minutes.4
However, the ACE may be overly reliant on intact memory and comprehension as an aspect of the testing. Therefore, the ACE may fail to capture those patients who can make complex decisions but have initial comprehension deficits as a result of an acute stroke. The neuropsychologists considered memory, attention, and expressive and receptive language in their global assessment for MDC by listening to patient responses and using clinical expertise to interpret them. Comparatively, the psychiatrist analyzed capacity on a narrow spectrum by focusing the evaluation on a prescripted but simple specific decision at a specific moment in time.
Entering into a randomized clinical trial is a complex decision that requires patients to understand many things—the study purpose, risks, and benefits of participation, the concept of blinding, randomization, voluntarism, and withdrawal. Becauise of its high sensitivity, the ACE appears to be a useful initial screening tool to detect if a patient has complex decision-making capabilities.
There are several limitations to this study. The small sample size limits the conclusions that can be drawn and future studies should include larger sample sizes. Another limitation was the simple scenario used for the psychiatric assessment. Although every attempt was made to create a likely scenario that a patient may face poststroke, the decision-making capacity for the scenario was not complex and may not be applicable to true research studies.
In conclusion, lack of MDC is very common in mild to moderate stroke patients. The ACE can accurately identify those who can participate in stroke trials. However, failing the ACE does not, at present, adequately determine patients who cannot participate. Failing the ACE should trigger supplemental testing to adequately distinguish those who because of initial comprehension issues or stroke-specific deficits appear to lack MDC, but in fact do possess it. Additional refinement and testing of alternative, nonemergent hypothetical scenarios of the ACE for a stroke-specific population might increase its utility for determining participation capacity in randomized clinical trials.
Sources of Funding
This study was supported by National Institutes of Health (NIH) Clinical and Translational Award UL1 RR024148. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Research Resources or the NIH.
Guest Editor for this article was Bo Norrving, MD, PhD, FESO, FAHA.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.114.006395/-/DC1.
- Received June 7, 2014.
- Revision received August 28, 2014.
- Accepted September 4, 2014.
- © 2014 American Heart Association, Inc.
- Fonville AF,
- Samarasekera N,
- Hutchison A,
- Perry D,
- Roos YB,
- Al-Shahi Salman R
- Bateman BT,
- Meyers PM,
- Schumacher HC,
- Mangla S,
- Pile-Spellman J
- Goodglass H,
- Kaplan E,
- Barresi B.
- Stern RW,
- White T