: Deep neural networks (DNN) have great success in solving difficult recognition tasks such as speech and image processing. However, performance depends on the amount of data available for training the network. In BCI, very large datasets are still missing and EEG data are particularly complex and noisy. Nevertheless, it is of interest to investigate whether DNN may prove efficient in this context. We tested 6 different deep learning models to accomplish binary classification of single-trial ERPs and compared them with Riemannian Geometry based classifiers. Each model implements a different architecture and uses two different input formats: an image ( 2D ) or a video ( 3D ). All models were tested on two different datasets, and under three different scenarios: within-subject, cross-subject and cross-experiment classification. Finally, to get insights about the decision process of the most successful DNN, we visualized the learned features using saliency maps. This revealed informative and interpretable differences between the two empirical datasets used for evaluation.
Background and ObjectiveSome neurodegenerative conditions can severely limit patients' capability to communicate because of the loss of muscular control. Brain-computer interfaces may help in the restoration of communication with these patients, bypassing the muscular activity, so that brain signals can be directly interpreted by a computer. There are many studies regarding brain-controlled spellers; however, these systems do not usually leap out of the lab because of technical and economic requirements. As a consequence, the potential end users do not benefit from these scientific advances in their daily life. The objective of this paper is to present a novel brain-controlled speller designed to be used by patients due to its versatility and ease of use.MethodsThe brain-computer interface research group of the University of Málaga (UMA-BCI) has developed a speller application based on the well-known P300 potential which can be easily installed, configured and used. The application supports the common P300 paradigms: the Row-Column Paradigm and the Rapid Serial Visual Presentation Paradigm. The inner core of the application is implemented with a widely used and studied platform, BCI2000, which ensures its reliability and allows other researchers to apply modifications at will in order to test new features. Ten naïve volunteers carried out exercises using the application and completed usability tests for evaluation purposes.ResultsNew subjects using the application managed to set up and use the proposed speller in less than an hour. The positive results of the evaluation through the usability tests support this application's ease of use.ConclusionsA new brain-controlled spelling tool has been presented whose aim is to be used by severely paralyzed patients in their daily lives, as well as by researchers to test new spelling features.
Across- and within-recording variabilities in electroencephalographic (EEG) activity is a major limitation in EEG-based brain-computer interfaces (BCIs). Specifically, gradual changes in fatigue and vigilance levels during long EEG recording durations and BCI system usage bring along significant fluctuations in BCI performances even when these systems are calibrated daily. We address this in an experimental offline study from EEG-based BCI speller usage data acquired for one hour duration. As the main part of our methodological approach, we propose the concept of adversarial invariant feature learning for BCIs as a regularization approach on recently expanding EEG deep learning architectures, to learn nuisance-invariant discriminative features. We empirically demonstrate the feasibility of adversarial feature learning on eliminating drowsiness effects from event related EEG activity features, by using temporal recording block ordering as the source of drowsiness variability.
Mental Imagery based Brain-Computer Interfaces (MI-BCI) are a mean to control digital technologies by performing MI tasks alone. Throughout MI-BCI use, human supervision (e.g., experimenter or caregiver) plays a central role. While providing emotional and social feedback, people present BCIs to users and ensure smooth users' progress with BCI use. Though, very little is known about the influence experimenters might have on the results obtained. Such influence is to be expected as social and emotional feedback were shown to influence MI-BCI performances. Furthermore, literature from different fields showed an experimenter effect, and specifically of their gender, on experimental outcome. We assessed the impact of the interaction between experi-menter and participant gender on MI-BCI performances and progress throughout a session. Our results revealed an interaction between participants gender, experimenter gender and progress over runs. It seems to suggest that women experimenters may positively influence partici-pants' progress compared to men experimenters.
There are many technologies being developed to assist individuals with severe disability. Devices based on human machine interface have been used to restore or replace lost movement and communication. Unfortunately, these technologies have not yet been extensively explored in children with severe disability. This paper describes a case study of a 16 year old patient in a locked-in state with virtually no communication. She is unable to move her body and is non-verbal. BCI potential was assessed using the mindBEAGLE system which utilizes auditory and vibro-tactile modalities to evoke a response. Long-term monitoring on clinical EEG characterized the neurophysiology of the patient including nearly continuous generalized discharges. The best classification accuracy for auditory and vibrotactile BCI was 40%. Higher accuracy (54%) was achieved using the motor imagery modality. Clinical neurophysiology measures can inform or to some extend predict the success of BCI performance of different modalities contributing to the user-centered design in BCI development.
A major objective of Brain-Computer interfaces (BCI) is to restore communication and control in patients with severe motor impairments, like people with Locked-in syndrome. These patients are left only with limited eye and eyelid movements. However, they do not benefit from efficient BCI solutions, yet. Different signals can be used as commands for non-invasive BCI: mu and beta rhythm desynchronization, evoked potentials and slow cortical potentials. Whatever the signal, clinical studies show a dramatic loss of performance in severely impaired patients compared to healthy subjects. Interestingly, the control principle is always the same, namely the replacement of an impossible (overt) movement by a (covert) attentional command. Drawing from the premotor theory of attention, from neuroimaging findings about the functional anatomy of spatial attention, from clinical observations and from recent computational accounts of attention for both action and perception, we explore the hypothesis that these patients undergo negative plasticity that extends their impairment from overt to covert attentional processes.
Riemannian methods are currently one of the best ways of building classifiers for EEG data in a brain-computer interface (BCI). However, they are computationally complex and suffer from a lack of interpretability. Since the full covariance matrix is used for each classification, it is not immediately possible to see what underlying signals are generating the classified changes in variance. Particularly in a rehabilitation context, where it is essential to control which brain signals are used for classification, this can be a severely limiting factor. Further, the requirement to perform a matrix logarithm can become prohibitively complex for real-time computation. In this work, we explore a method for extracting spatial filters from a solution in the Riemannian tangent space and compare it against common spatial patterns. We show via comparisons on multiple open-access datasets that it is possible to generate filters that approach the performance of the full Riemannian solution while maintaining interpretability.
To interact with the brain in closed-loop applications despite low signal-to-noise ratios, braincomputer interfaces can make use of data-driven linear spatial filtering approaches to improve the single-trial classification performance. While the transfer of spatial filters between users and within multiple sessions of the same user under the same experimental paradigm is feasible to some extent, it is unclear, whether changing experimental conditions affects this transferability. To investigate this question with regards to spatial filters, we evoke event-related potentials by an auditory oddball paradigm under various stimulus onset asynchronies (SOAs). Using four similarity and distance measures, we analyze the between and within subject transferability of xDAWN filters. We found that the used measures reflect the differences of spatial filters between subjects. Within the same subject, the measures indicate similarity of the spatial filters for almost all SOA conditions. We conclude, that our proposed measures can be used to give indications under which circumstances spatial filters can be transferred.
Communication is a critical human function that can be severely compromised in patients with neurological diseases such as amyotrophic lateral sclerosis (ALS). The P300 speller is a brain-computer interface (BCI) device that restores communication in these patients by detecting evoked responses in subjects’ electroencephalography signals. One of the bottlenecks of these systems is the pause after character selections. This pause has been necessary for the P300 speller because it signals users that a character selection has been made and gives them time to transition to the next character. If this pause is too long, the system is slowed down unnecessarily. If it is too short, stimuli for the next character begin before the user is ready. We propose a system that does away with the pause entirely and continually flashes stimuli. We employ a joint model that determines the target characters as well as the transition times so that users can change between characters at their own pace. A preliminary study on eight subjects showed a selection rate of 16.35 characters/minute and an average accuracy of 94.85%, both significant improvements over performance in an equivalent system with standard flashing. These results suggest that the P300 speller could be improved by implementing a continuous flashing paradigm.
In a case study with a person with high cervical spinal cord injury, we show a first proof-of-concept on how to detect and classify different movement attempts of the same upper limb. The lesion was complete (AIS A) at level C4 and no hand function was preserved. We detected in a self-paced online setup hand open and palmar grasp with an accuracy of 68.4 % (chance level 50%).
