Toward Brain-Computer Interfacing
2010.10.5.
이정수
Contents
I. BCI Systems and Approaches
Introduction (p.27~30)
4.Graz-Brain-Computer Interface: State of Research (p.65~84)
6. The IDIAP Brain-Computer Interface: An Asynchronous
Multiclass Approach (p.103~110)
2
Introduction
Two paradigms of BCI study
Active paradigm (chap. 2,3,7)
Active & voluntary strategy for generating a specific
regulation of and EEG parameter
Motor-related μ-rhythm
self-regulation of slow cortical potentials(SCP)
Passive paradigm (chap. 2,4)
Participants only have to view an item for selection
Evoked responses such as P300
Steady-state evoked potentials(SSVEP)
“Let the machines learn”, Berlin group (chap. 4,5,6)
Learning is done by the computer not human
The subject will inevitably learn once feedback has started
Both aspect : subject & machine training
3
Introduction(cont.)
Albany BCI(chap.2)
A user is trained to manipulate this μ and β rhythms
To control a cursor in 1- or 2D
BCI control based on the P300
Tϋbingen BCI(chap.3)
Train subjects to adapt to the system using SCP
Mean for communication of ALS patients with the world
P300, μ-rhythm-based BCIs, auditory stimulation, ECoG
Graz BCI(chap.4)
Whole BCI field : sensors, feedback strategies, cognitive
aspects, novel signal processing methods, with excellent
results
4
Introduction(cont.)
Berlin BCI(chap.5)
Only 30minutes for training subjects rather than weeks or
months
Advanced machine learning and signal processing technology
Online feedback studies
Martigny BCI(chap.6)
Similar to the Berlin approach
Machine learning rather than subjects training
Online adaptation to realize a BCI
Vancouver BCI(chap.7)
Asynchronous BCI for patients
Machine learning techniques
5
4. Graz-Brain-Computer Interface
Abstract
BCI
Signals from the human brain -> commands of control
devices or application
Basic communication capabilities for severe neuromuscular
disorders
Graz-BCI system
Oscillations of β or μ rhythms, SSVEP, SSSEP
The use of complex band power features
The selection of important features
Phase-coupling & adaptive autoregressive parameter
estimation to improve single-trial classification
Control of neuroprosthses, spelling system, asynchronous BCI
6
Components of Graz-BCI
7
Components of Graz-BCI
Graz-BCI
Noninvasive EEG from the scalp
ECoG recorded during self-paced movements
To detect the motor action in single trials
Dynamic oscillations of β or μ rhythms, SSVEP, SSSEP
Three mental strategies
Operant conditioning
Birbaumer’s lab in Tϋbingen
Predefined mental task
Moto imagery(left or right hand, both feet, tongue)
Similar cortical areas activation & temporal characteristics
Attention to an externally paced stimulus
Feedback
Delayed(discrete) FB : correct or incorrect at the end
Continuous FB : indicates immediately
8
Graz-BCI Control with Motor Imagery
EEG signals from the sensorimotor cortex
Desynchronization of β or μ rhythms at the time of movement
onset
Reappearance when the movement is complete
Quantification of temporal-spatial ERD and ERS, motor
imagery can induce different types of activation patterns
Desynchroniztion(ERD) of sensorimotor rhythms
Synchronization (ERS) of u rhythms
Short-lasting synchronization of central B oscillation after
termination of motor imagery
Imagery related brain activity necessary to BCI, but subject
with no changes in EEG
Diversity of time-frequency pattern reported
9
Graz-BCI Control with Motor Imagery
10
Adaptive Autoregressive(AAR) Parameters
Spectral properties of EEG are useful feature for BCI
Due to use of FFT, feature extraction was block-based and
no feedback in continuous time
Other methods for spectral estimation
Autoregressive model, stationary estimators, adaptive
estimation algorithms(LMS, RLS, Kalman filtering)
Adaptively estimated AAR parameters
Obtained with a time-resolution as high as the sampling rate
Possible to provide continuous feedback in real time
Neural network -> linear discrimant analysis(LDA)
Simple and fast training procedure
Provide a continuous discrimination function
Became the standard classifier for AAR
11
Complex Band Power Features
Bandpower features : important for classification of brain
pattern
Squaring the values of the samples and then smoothing the
result in the time domain
FFT in frequency domain, yield phase information
CBP(complex bandpower) feature
To test importance of phase
CBP vs CSP(common spatial patterns)
CBP is superior to CSP
Require fewer electrode
Less training data than CSP
Phase information is an important and useful feature in BCI
12
Phase Synchronization Features
Almost all BCIs ignore the relationships btw EEG signals
measured at different electrode recording sites
Logarithmic bandpower feature or adaptive autoregressive
parameter
Quantifying the relationships among the signals of single
electrodes
PLV (phase locking value)
PVL value of 1 : two channels are highly synchronized
PVL value of 0 : no phase synchronization
13
Adaptive Classifier
To automatically adapt changes in the EEG patterns of the
subject
To deal with their long-term variation
ADIM
Estimate online the Information Matrix to compute an
adaptive version of the QDA
ALDA
Adaptive linear discriminant analysis based Kalman filtering
Experiments
AAR features and ADIM
BP estimation
Concatenation of AAR & BP combined with ALDA
More details in chapter 18
14
Importance of Feature Selection
Many feature extraction methods for BCI
Bandpower :extracts features for specific frequency ranges
Filter methods
E.g., Fisher distance,r2
Wrapper methods
Flexible & generally applicable but computationally demanding
E.g., genetic algorithms(to find suitable wavelet features in
ECoG data), heuristic search strategies
So-called embedded algorithms
E.g., linear programming,
DSLVQ(Distinction sensitive learning vector quantization)
electrode position and frequency components selection
15
Steady-State Evoked Potentials
SSVEP,SSSEP
Both sensory respond “resonance-like” frequency regions
Visual system
Near 10Hz : greatest SSVEP amplitude
1618Hz : medium amplitude
Near 40-50Hz : smallest amplitude
Somatosensory
around 27Hz in EEG β range
SSVEP
32LED bars (4X8), 6,7,8,13Hz modulation
SSSEP
Stimulation frequency 25-31Hz and 20-26Hz
16
Graz-BCI Applications
Control of Neuroprostheses
BCI for paralyzed limbs to restore their grasp
Functional electrical stimuliation(FES)
Two male with high spinal cord injury(SCI)
One patient (30 years)
Left hand grasp function restored with FES
After 4 month training, learned to induce 17Hz oscillations
A drinking glass
The other patient (42 years)
Freehand system implanted in his right hand and arm
Only Three days training, power decrease of EEG amplitude
during left hand movement imagination
17
Graz-BCI Applications(Cont.)
Control of a Spelling Application
A 60-year-old male patient with ALS
To enable the patient operate the cue-based two-class “virtual
keyboard”
Artificial ventilated, totally paralyzed
Two bipolar EEG channel from four gold electrodes placed
over the left and right sensorimotor area(C3, C4)
Cue-based motor imagery trials
18
Graz-BCI Applications(Cont.)
Uncued Navigation in a Virtual Environment
Three bipolar EEG channels
19
6. IDIAP Brain-Computer Interface
Portable BCI system based on the online analysis of
spontaneous EEG signals with scalp electrodes
Relies on an asynchronous protocol
Variation of EEG over several cortical areas related to
imagination of movements, arithmetic operations, or
language
To discover task-specific spatiofrequency pattern embedded
in the continuous EEG signal
Able to recognize three mental tasks with a statistical
Gaussian classifier
Virtual keyboard, video game, mobile robot
20
Operant Conditioning & Machine Learning
Subjects training
learn to control their brain activity
appropriate, but lengthy training
to generate fixed EEG patterns that the BCI transforms into
external actions
Machine learning approach
Based on a mutual learning process where the user and the
brain interface are coupled together and adapt to each other
accelerate the training time (few hours of training)
Rejection criteria to avoid making risky decisions
A low classification error is a critical performance criterion
Bayesian techniques for rejection purposes
21
Synchronous & Asynchronous BCI
Synchronous BCI
EEG is time-locked to externally paced cues repeated every 410s
limited by a low channel capacity (below 0.5bits/s)
Facilitate EEG analysis
the starting time of mental states are precisely known
differences with respect to background EEG activity can be
amplified
Normally recognize only two mental states
Asynchronous BCI
Subjects make self-paced decisions
Response time is below 1s, channel capacity(1 ~ 1.5 bits/s)
To steer a wheelchair, BCI delivers rapid & accurate command
“idle” states : no particular mental task
Giving “no” response without explicit training of classifier
22
Spatial Filtering
EEG signal : poor SNR & spatial resolution
Surface laplacian(SL) derivation
Normally 64-128 electrodes
Better cortical activity below electrodes immediately
Global method
Raw EEG potential interpolated using spherical splines
Second spatial derivative is taken
Local method
The average activity of neighboring electrodes(normally 4)
is subtracted from the electrode of interest
Common average reference(CAR)
Raw EEG potentials is transformed to CAR
Remove the average activity of all the electrodes
Other spatial filtering algorithms(chap 13)
23
Experimental Protocol
Users select three mental tasks
“relax”, “imagination of left and right hand movements”,
“cube rotation”, “subtraction”, “word association”
In training session
Subjects is seated & performed selected task for 10-15s
Consecutive training trials lasting about 5min and breaks
5~10m (repeated normally 4)
Time-shift btw starts performing & label for the next task
Acquired EEG data is not time-locked to the events
Feedback through three buttons on the screen
24
Experimental Protocol(Cont.)
Signal acquisition & signal processing
EEG with 8~64 electrode
Raw EEG potential is transformed using SL
Extract relevant features from a few channels(8~15) and
corresponding vector is used as input to the classifier
the Welch periodogram algorithm to estimate the power
spectrum of each selected SL-transformed channel
averaging three 0.5-s segments with 50 percent overlap(2 Hz)
Normalization of the frequency band 8–30 Hz
The resulting EEG sample is analyzed by the statistical
classifier.
No artifact rejection algorithm is applied
Statistical Gaussian Classifier
More details in chapter 16
25
Brain-Actuated Prototypes
BCIs is used for brain-actuated applications
A virtual keyboard on a computer screen
Whole keyboard(26 English letter & space key), 3 X 9
Divided in three blocks, each associated to one mental task
Same colors as the training phase
Three decision steps to write a single letter
3.5s waits to undo in case of wrong selection
Takes 22.0s for trained subject to select a letter
Brain game
Classical Pac-man game
Two mental task, turn left or turn right
Computer cursor in 2D
Trained subjects to control two independent EEG rhythms
26
Virtual Keyboard
RED : “cube rotation”
Yellow : “subtraction”
Green : “word association”
27
Brain-actuated Prototypes
Mobile robot - a motorized wheelchair
Millan et al(2004), first asynchronous analysis of EEG signal
Fast and frequent switches are required
A key
subject can issue high-level commands at any moment
shared control between two intelligent agents :the human user
and the robot
user gives high-level mental commands that the robot performs
autonomously, “turn right at the next occasion”
robot executes these commands autonomously using the readings
of its on-board sensors
Asynchronous , not requiring waiting for external cues
The robot relies on a behavior-based controller
to implement the high-level commands to guarantee obstacle
avoidance and smooth turns
on-board sensors are read constantly and determine the next
action
28
Discussion
Real-time control of robots and neuroprostheses is most
challenging application of BCI
1. More powerful adaptive shared autonomy framework for
the cooperation of the human user & the robot
2. Better electrical activity all across the brain with high
spatial accuracy using noninvasive scalp EEG
Local field potentials(LFP): electrical activity of small groups
of neurons
Estimated LFP has the potential to unravel scalp EEG
Grave de P Menedez(2005), more details in chapter 16
3. To improve the robustness of a BCI
More details in chapter 18
4. analysis of neural correlates of high-level cognitive
states
Errors, alarms, attention, frustration, confusion
More details in chapter 17
29
0
