Think to Move: a Neuromagnetic Brain-Computer Interface (BCI) System for Chronic Stroke
Background and Purpose— Stroke is a leading cause of long-term motor disability among adults. Present rehabilitative interventions are largely unsuccessful in improving the most severe cases of motor impairment, particularly in relation to hand function. Here we tested the hypothesis that patients experiencing hand plegia as a result of a single, unilateral subcortical, cortical or mixed stroke occurring at least 1 year previously, could be trained to operate a mechanical hand orthosis through a brain-computer interface (BCI).
Methods— Eight patients with chronic hand plegia resulting from stroke (residual finger extension function rated on the Medical Research Council scale=0/5) were recruited from the Stroke Neurorehabilitation Clinic, Human Cortical Physiology Section of the National Institute for Neurological Disorders and Stroke (NINDS) (n=5) and the Clinic of Neurology of the University of Tübingen (n=3). Diagnostic MRIs revealed single, unilateral subcortical, cortical or mixed lesions in all patients. A magnetoencephalography-based BCI system was used for this study. Patients participated in between 13 to 22 training sessions geared to volitionally modulate μ rhythm amplitude originating in sensorimotor areas of the cortex, which in turn raised or lowered a screen cursor in the direction of a target displayed on the screen through the BCI interface. Performance feedback was provided visually in real-time. Successful trials (in which the cursor made contact with the target) resulted in opening/closing of an orthosis attached to the paralyzed hand.
Results— Training resulted in successful BCI control in 6 of 8 patients. This control was associated with increased range and specificity of μ rhythm modulation as recorded from sensors overlying central ipsilesional (4 patients) or contralesional (2 patients) regions of the array. Clinical scales used to rate hand function showed no significant improvement after training.
Conclusions— These results suggest that volitional control of neuromagnetic activity features recorded over central scalp regions can be achieved with BCI training after stroke, and used to control grasping actions through a mechanical hand orthosis.
One year after stroke a third of all affected patients have poor or nonexistent residual hand motor function despite intensive treatment and rehabilitation.1 Significant functional recovery after this initial year is rare2,3 despite novel interventional approaches recently applied in the chronic stage, like bilateral arm training or constraint-induced movement therapy.4,5 These treatments are based on the ability of the patient to perform actions with the affected arm or hand, and therefore require a moderate degree of residual motor function. There are, however, many patients who do not have such residual function and therefore cannot use the plegic hand at all for training purposes. At present, there is no treatment available for this condition.
Over the past 15 years, an increasing number of brain-computer interface (BCI) systems have been developed.6 All of these systems record, decode, and ultimately translate some measurable neurophysiological signal into an effector action or behavior. Existing BCI systems have used invasive microelectrode arrays to record single-unit spiking activity and local field potentials,7,8 and subdural electrode arrays to record electrocorticography.9 Noninvasive signal recording approaches have used electroencephalography (EEG),10 magnetoencephalography (MEG),11 blood-oxygen-level dependent functional MRI,12 and near infrared spectroscopy.13 End-user applications developed for human BCI systems have included 1-D, 2-D, and 3-D (point-and-click) screen cursor control, as well as spelling software for communication. In addition, using microelectrode implants in various cortical areas, monkeys have been trained to control robotic arms for reaching and grasping during feeding.14 A few attempts have been made to apply these technologies to patient groups, with these attempts primarily focusing on patients with amyotrophic lateral sclerosis or tetraplegia.6,10
Here, we describe a BCI system that uses MEG activity evoked by the patient’s intent to move a completely paralyzed hand to control grasping motions of a mechanical orthosis attached to the affected hand.
Eight patients with chronic hand plegia resulting from stroke were recruited from the Stroke Neurorehabilitation Clinic, Human Cortical Physiology Section of the National Institute for Neurological Disorders and Stroke (NINDS) (n=5; mean age=58.2±7.0 years; mean hand plegia duration=25.2±11.6 months) and the Clinic of Neurology of the University of Tübingen (n=3; mean age=41.7±26.6 years; mean hand plegia duration=16.7±6.4 months). One patient recruited from the University of Tübingen, patient MD, experienced a pediatric stroke. Patients were included in the study if they had a history of a single stroke, with residual finger extension weakness rated as 0/5 on the Medical Research Council (MRC) scale (Table). Therefore, all patients included in this study were completely unable to induce any voluntary movements in extensors of the plegic hand. Spasticity of shoulder, elbow and finger flexors and extensors was rated as 3 or less on the Modified Ashworth Scale15 in eligible patients to ensure that their arm could maintain a comfortable posture while seated in the MEG chair, and that their fingers could be passively manipulated by the hand orthosis (Figure 1). Medical and neurological screening history and examination excluded patients with major cognitive deficits (Folstein Mini Mental Status Test16 lower than 23), major depressive disorder, or other uncontrolled illness. Anatomic MRI of the brain was used to exclude patients with cerebellar or brain stem lesions, but otherwise, lesions of the suprapontine corticospinal tract of varying sizes and extent were included as long as they resulted in a plegic hand. Patients provided written informed consent and the study was approved by the Institutional Review Board of the NINDS and the Ethical Committee of the Faculty of Medicine of the University of Tübingen.
Neuromagnetic activity recorded from a 275-channel (6 patients) or 153-channel (2 patients) MEG array (VSM Medtech) was used to control a BCI as previously described,11 at both the NIH and the University of Tübingen. Both MEG apparatuses were housed in a magnetically shielded room and used synthetic 3rd gradient balancing to reduce interference from environmental noise. Recordings from all MEG channels were antialiased with a 200 Hz cut-off, low-pass filter, and digitally sampled at 600 Hz.
During recording, patients were seated alone in the shielded MEG room with the lights slightly dimmed, and their head centrally positioned within the sensor array. A closed-circuit video system was used to constantly monitor the patients, while instructions were given during rest periods via an intercom system. Patients were instructed to refrain from extraneous movement while engaged in experimental tasks (especially with the healthy arm) to reduce artifacts. Adherence to these instructions was monitored during recording via the video system, as well as online electromyography recordings obtained from the brachioradialis muscle of both arms.
MEG was chosen to drive this initial proof of principle study of BCI in chronic stroke because of its noninvasiveness and exquisite spatial and temporal resolution. Two additional features of MEG made it desirable relative to EEG. The magnetic fields generated by brain activity are minimally distorted by brain lesions, making MEG particularly appropriate for studies in stroke.17,18 Furthermore, the collection of MEG data does not require the attachment of scalp electrodes or related cleaning procedures used to reduce electrode impedance. This latter point in particular allowed patients to start the task rapidly after arriving at the laboratory, avoiding the fatigue inherent to long periods of experimental preparation. Together, these features made MEG an ideal source of on-line recording and localization of dynamic cortical rhythm changes.
μ Rhythm-Based BCI
Amplitude modulation of the μ rhythm was used to control this BCI system,11 which was based on the BCI2000 software platform19 (www.bci2000.org). The μ rhythm is typically found over the sensorimotor cortex with a base frequency of 9 to 12 Hz. Its arc-shaped waveform includes a strong first harmonic in the β band at 20 to 24 Hz. The terms synchronization and desynchronization are commonly used to describe increases and decreases in μ rhythm amplitude relative to some baseline, respectively. μ rhythm desynchronization has been observed during the planning, execution, or even imagination of limb movements.20–23 In particular, the substantial and relatively somatotopic μ rhythm amplitude modulation observed during engagement in motor imagery tasks made it appealing for use in stroke patients, who could perform imagined movements or even attempt to move their plegic hand in the absence of any motor function.
During BCI training, μ rhythm amplitude estimates were derived from 3 to 4 MEG-sensors from the array (Table). The cluster(s) of MEG sensors chosen as BCI controllers were identified after an initial session (described in detail below) during which subjects imagined grasping movements of the plegic hand. The sensors chosen for BCI control were the ones that showed the highest correlated modulation of μ rhythm amplitude between conditions (see supplemental data for detailed description, available online at http://stroke.ahajournals.org).
During all BCI training sessions, a mechanical orthosis was attached to the plegic hand. Fingers 2 to 5 (index, middle, ring, and little fingers) were individually inserted into ring-like fasteners that grasped each digit at the first phalanx, and fixated by a screw-adjustable shoe to prevent slippage. Each fastener was connected to a plastic Bowden cable that allowed for hand grasping or hand opening motions. In order to minimize magnetic artifacts in the MEG environment, these cables were extended and retracted by opening and closing computer-controlled pneumatic valves. The orthosis had 2 possible movement motions: flexion or extension of fingers 2 to 5 in a hand grasping or hand opening fashion, respectively (Figure 1). All 4 fingers were synchronously moved in the same direction. The orthosis actions were synchronized with the BCI training task described below through parallel port communication with a custom control circuit.
Real-Time Feedback and BCI Training Task
During each session, patients performed between 150 to 250 trials of a goal-oriented, visual feedback task (supplemental Figure I, available online at http://stroke.ahajournals.org). The task was designed to help them achieve volitional control of μ rhythm amplitude, and thus control of the orthosis.
Each trial was initiated by the presentation of a target on either the upper or lower half of the right side of a visual display (Figure 1). The target was a visual representation of an acceptable range of μ rhythm amplitudes for the desired orthosis action. A square screen-cursor would then begin moving at a fixed rate from left-to-right across the display, with the cursor feedback updated every 300 ms. The vertical height of the cursor was a transformation of the recorded μ rhythm amplitude. The goal for the patient was to volitionally modulate the μ rhythm amplitude in such a way so that the cursor contacted the target once it reached the right edge of the screen. The BCI software maintained a history of the mean μ rhythm amplitude estimate from each trial and assigned this to a distribution representing observations for each target (or orthosis action) condition. The classification threshold, defined as the midpoint between the means of these 2 distributions, was adaptive to account for changes in the shapes of these distributions over the course of training.
At the conclusion of each trial in which the patient was successful at producing the appropriate modulation of μ rhythm amplitude (meaning the cursor hit the target), a simultaneous change in target color (red to yellow) and orthosis action occurred providing reinforcement (Figure 1). If the cursor failed to hit the target, no reinforcement was provided (meaning no orthosis manipulation of hand posture occurred).
μ Rhythm Amplitude/Orthosis Action Coupling
The coupling of μ rhythm synchronization/desynchronization to the resulting orthosis action was determined by patient preference. Patients generally achieved μ rhythm synchronization via passive relaxation imagery, and desynchronization via motor imagery of some hand action. In 5 of the 8 patients, opening of the hand by the orthosis was associated with μ rhythm synchronization, whereas grasping motions were associated with desynchronization. The other 3 patients chose the opposite coupling. This alternative choice was most likely related to the greater degree of spasticity present in muscles of the affected hand and arm of these patients, as their hands normally displayed a more grip-like posture in their passive state. Thus, all patients chose to relate the more passive form of imagery with their passive hand posture state.
In the initial session, which lasted for approximately 1 hour, patients were familiarized with the MEG environment as well as with the hand orthosis. They sat upright fixating a screen located 50 cm in front of their eyes. During that period of time, they were instructed to perform the following tasks in a randomized order: (1) repeated grasping motions of the intact hand at 0.5 Hz rate guided by a visual metronome stimulus on the screen, (2) motor imagery of the same hand motions without actually moving the intact hand, (3) motor imagery of comparable movements of the plegic hand, and (4) fixation of the metronome stimulus in a “resting” state. The instructions were displayed on the screen in front of the patients with trials separated by 2 second intervals. Twenty-four trials were recorded for motor movements and 48 trials were recorded for imagined movements during this initial session. As stated above, these data were used to determine parameters for the subsequent BCI-training sessions.
BCI Training Sessions
Over the course of approximately 3 to 8 weeks, patients participated in 13 to 22 training sessions (separated by at least 24 hours; Table). The training frequency was highly determined by each patient’s tolerance to fatigue, or additional time commitments, and ranged between 3 to 5 times per week (Table). During these sessions, they performed the training task described above. Each training session lasted 1 to 2 hours and was implemented on an outpatient basis at both locations.
The success rate (the proportion of trials in which patients were successful at contacting the target with the cursor, or alternatively, producing the requested μ rhythm amplitude modulation) was computed for trial presentations during a single training session, and used as a performance measure. A trial was considered a “success” when the cursor arrived at the requested target over the time of a trial (see supplemental Figure I). A significant hit-rate increase from “chance” levels of 50% indicated that volitional control of μ rhythm modulation at the desired MEG sensor locations was achieved (Figures 2 and 3⇓A).
Off-line analysis of all training sessions included computation of spectral power differences (Figure 3B) and statistical maps of μ rhythm amplitude correlations with target condition/orthosis action,24 for each MEG sensor and frequency band. Topographical maps plotted for a single frequency band use spatial information about the location of areas displaying more prominent and consistent synchronization/desynchronization patterns between task conditions (Figure 3C).
The Wilcoxon signed-rank test, the nonparametric homologue to the paired Student t test, was used to compare changes in group performance during training.25 To assess changes in individual patient performance during training, the change-point test was used.25 The change-point test assumes the null hypothesis that no time trend exists in the series of performance data. Based on this assumption, each session performance should rank on average near the median, and the cumulative sum of ranks should increase approximately linearly with session number. The maximal deviation from this expected linear increase in rank is considered as a potential “change-point” and is used to divide the time series into 2 components. These components are then compared using a Kolmogorov-Smirnov test to determine statistical significance.
All 8 patients at the 2 sites, NIH and University of Tübingen, successfully completed the study. On average, the group performance rate improved with training to 72.48±18.36% (median±interquartile range) during the final session (Figure 2), as compared to an initial median performance rate of 52.84±20.59% (paired Wilcoxon signed rank test; P<0.05).
Despite this general and encouraging improvement, there was substantial variability in the ability of different patients to improve their hit-rate (and consequently their μ rhythm desynchronization control, Figure 3). A majority of patients showed exponential performance increases during training with variable growth rates and delays of onset. After 15 training sessions, patient BA did not show any increase in performance, which remained near chance levels of 50%. In contrast, patient WF showed initial high rates of performance above 80% that then declined before becoming stabile at approximately 70%. The reason for this decrease in performance is not known.
The majority of patients displayed μ rhythm desynchronization in the 3 Hz-wide control frequency band (9 Hz central frequency for PA, 12 or 12.5 Hz for all other patients) for the “grasping” orthosis action. Furthermore, these differences were greatest in areas surrounding the control sensor locations (with the exceptions of BW who showed minimal modulation and MD, whose stroke occurred at a very early age, who showed diffuse modulation across the majority of the array). Collectively, the R2 statistical maps reveal that modulation of the trained μ-rhythm feature was more strongly related to the task in segregated regions of the array that surrounded the control sensor locations (Figure 3C). Four of the 8 participants were able to achieve voluntary μ-rhythm control over central ipsilesional regions of the array, whereas 3 participants achieved control using sensors from central contralesional areas of the array. Post-training MRC scores of finger extension strength remained 0/5 for all patients, indicating that the training had no effect on gross hand motor function.
These data demonstrate that most patients with chronic stroke and complete hand paralysis, in this small sample, can learn to modulate μ rhythm amplitude to achieve binary control of an orthosis that passively manipulates the grasping posture of the plegic hand. Furthermore, this control can be achieved using MEG signals recorded over the ipsilesional hemisphere. Patients achieved success rates that varied between 65% and 90% by the end of the 13 to 22 sessions training period, and 6 of the 8 patients showed a significant performance improvement over this period. Of the 2 patients (WF and BW) who did not show improvements, patient WF displayed a high success rate at the outset of training (86%) and his failure to improve may be attributed to a ceiling effect. This surprising degree of voluntary control of cortical rhythms, despite predominantly extensive subcortical lesions that in the case of 4 of the 8 patients expanded into cortical tissue, suggests that this strategy could be effective in patients with various lesion types.
It should be kept in mind that for this study, we only included patients that were completely unable to move the paretic hand because they represent the patient group with very few rehabilitation options available. The BCI approach was used here to induce hand grasping and opening in patients unable to elicit voluntary movements because of the potential importance of these motions for activities of daily living.26 More studies are clearly needed to evaluate the extent to which our conclusions apply to patients with different lesion locations, extension, etiology, or even chronicity. We also do not know if the behavioral gains demonstrated in this study consolidate over time, or fade in the absence of constant reinforcement, as has been reported in some motor learning paradigms in healthy subjects.27 Although these represent important areas of future research, our results clearly indicate that voluntary control of μ rhythm amplitude recorded over central cortical regions (either ipsilesional or contralesional) can be used to control biphasic grasping motions of a plegic hand through a hand orthosis.
Although present MEG technology is not practical for long-term or portable brain control of an orthosis, our results suggest that similar control may be achieved with EEG. Recording μ rhythm from 3 to 4 MEG sensor sites was successful in driving the BCI orthosis capable of a bimodal grasping-opening of a completely paralyzed hand. It is then theoretically possible that properly referenced EEG electrodes placed on these crucial locations could be similarly successful in driving the BCI. If so, there is the potential that relatively inexpensive and portable EEG-orthosis systems could be developed in the future to operate in home or chronic care settings.
Two forms of BCI systems have been described in the past in humans: invasive8 and noninvasive.28 Noninvasive approaches, comparable to ours, have been used to allow communication in locked-in or severely paralyzed amyotrophic lateral sclerosis patients29 and after tetraplegia,30 but to our knowledge not after stroke. Our findings now provide conclusive data demonstrating the potential usefulness of BCI-like approaches in patients with severe motor disability resulting from stroke.
Although the full analysis of the electrophysiological data recorded over the training period in our patients is clearly beyond the boundaries of this report, it is likely that this form of imagery training led to cortical reorganization in our patients, consistent with previous findings.31
In summary, these results demonstrate that patients with chronic stroke and complete hand paralysis can learn to control μ rhythm synchronization and desynchronization through motor imagery of the paralyzed hand. Harnessing the cortical activity generated by such imagery through a noninvasive BCI device, can then be used to elicit hand grasping/opening motions of an orthosis attached to the paralyzed hand.
Choice of MEG Sensors and Strategy for Training Feedback
For online feedback, we chose a single frequency band within the μ rhythm (9 to 15 Hz) or β (18 to 30 Hz) range that displayed the largest correlation with successful target hitting in the visual feedback task (see supplemental Figure I). As a spatial filter, sensors with the highest correlation values were linearly combined with weights of +1 or −1 depending on the relative orientation of the magnetic field lines at the respective sensor locations (into or out of the skull; for further details see Mellinger et al, 2007).11
The intramural NINDS, DIR, NIH as well as the NIMH MEG Core facility contributed to this research. We thank Drs R Coppola and T Holroyd for their help in managing the NIH MEG facility and technical advice, J Dagenais and Drs D Broetz, T Hinterberger, J Dax and C Rutten for useful discussions.
Sources of Funding
This work was supported by the intramural program of the NINDS, the Deutsche Forschungsgemeinschaft (DFG), and the Deutsche Akademische Austauschdienst (DAAD).
↵*These authors contributed equally to this work.
- Received September 24, 2007.
- Revision received November 29, 2007.
- Accepted December 19, 2007.
Lai SM, Studenski S, Duncan PW, Perera S. Persisting consequences of stroke measured by the stroke impact scale. Stroke. 2002; 33: 1840–1844.
Duncan PW, Goldstein LB, Horner RD, Landsman PB, Samsa GP, Matchar DB. Similar motor recovery of upper and lower extremities after stroke. Stroke. 1994; 25: 1181–1188.
Whitall J, McCombe Waller S, Silver KH, Macko RF. Repetitive bilateral arm training with rhythmic auditory cueing improves motor function in chronic hemiparetic stroke. Stroke. 2000; 31: 2390–2395.
Taylor DM, Tillery SI, Schwartz AB. Direct cortical control of 3D neuroprosthetic devices. Science. 2002; 296: 1829–1832.
Bohannon RW, Smith MB. Interrater reliability of a modified Ashworth scale of muscle spasticity. Phys Ther. 1987; 67: 206–207.
Kubler A, Nijboer F, Mellinger J, Vaughan TM, Pawelzik H, Schalk G, McFarland DJ, Birbaumer N, Wolpaw JR. Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology. 2005; 64: 1775–1777.
Siegel S, Castellan NJ. Nonparametric Statistics for the Behavioral Sciences. New York: McGraw-Hill; 1988.
Caithness G, Osu R, Bays P, Chase H, Klassen J, Kawato M, Wolpert DM, Flanagan JR. Failure to consolidate the consolidation theory of learning for sensorimotor adaptation tasks. J Neurosci. 2004; 24: 8662–8671.
Sharma N, Pomeroy VM, Baron JC. Motor imagery: a backdoor to the motor system after stroke? Stroke. 2006; 37: 1941–1952.