Brain–computer interfaces (BCIs) have the potential to enable individuals to interact with devices by detecting their intention from brain activity. A common approach to BCI is to decode movement intention from motor imagery (MI), the mental representation of an overt action. However, research-grade electroencephalogram (EEG) acquisition devices with a high number of sensors are typically necessary to achieve the spatial resolution required for reliable analysis. This entails high monetary and computational costs that make these approaches impractical for everyday use. This study investigates the trade-off between accuracy and complexity when decoding MI from fewer EEG sensors. Data were acquired from 15 healthy participants performing MI with a 64-channel research-grade EEG device. After performing a quality assessment by identifying visually evoked potentials, several decoding pipelines were trained on these data using different subsets of electrode locations. No significant differences (p = [0.18–0.91]) in the average decoding accuracy were found when using a reduced number of sensors. Therefore, decoding MI from a limited number of sensors is feasible. Hence, using commercial sensor devices for this purpose should be attainable, reducing both monetary and computational costs for BCI control.
Surgical neuromodulation through implantable devices allows for stimulation delivery to subcortical regions, crucial for symptom control in many debilitating neurological conditions. Novel closed-loop algorithms deliver therapy tailor-made to endogenous physiological activity, however rely on precise sensing of signals such as subcortical oscillations. The frequency of such intrinsic activity can vary depending on subcortical target nucleus, while factors such as regional anatomy may also contribute to variability in sensing signals. While artefact parameters have been explored in more 'standard' and commonly used targets (such as the basal ganglia, which are implanted in movement disorders), characterisation in novel candidate nuclei is still under investigation. One such important area is the brainstem, which contains nuclei crucial for arousal and autonomic regulation. The brainstem provides additional implantation targets for treatment indications in disorders of consciousness and sleep, yet poses distinct anatomical challenges compared to central subcortical targets. Here we investigate the region-specific artefacts encountered during activity and rest while streaming data from brainstem implants with a cranially-mounted device in two patients. Such artefacts result from this complex anatomical environment and its interactions with physiological parameters such as head movement and cardiac functions. The implications of the micromotion-induced artefacts, and potential mitigation, are then considered for future closed-loop stimulation methods.
Mental fatigue results in feelings of tiredness and decreases task performance and efficiency. It can be caused by cognitive underload (passive fatigue) or overload (active fatigue). Although fatigue detection models have been developed using electroencephalogram (EEG) recordings, it is unclear whether their predictions (derived from one task) are applicable to different tasks. We investigated if an EEG-based fatigue model previously trained on a passive fatigue-inducing driving task, can predict fatigue during an active fatigue-inducing task. We collected EEG while subjects performed a target-hitting task as a non-fatigued baseline, followed by mental arithmetic tasks of increasing complexity and mental load. Overall, the fatigue score from the passive fatigue model was significantly correlated with self-reported fatigue and sleepiness, which increased over the course of the experiment. The fatigue score was higher during simple mental arithmetic than the target-hitting task, demonstrating that the model can successfully differentiate non-fatigued and passive/low mental load situations. However, the fatigue score was higher during the simple (low mental load) than the complex (high mental load) mental arithmetic task, showing that the model cannot accurately detect active fatigue. Future fatigue models should be trained on a variety of both active and passive fatigue-inducing tasks to be more generalizable.
Virtual environments are often used in pre-wearing training and assessment of prosthetic control abilities. Here, we developed a virtual prosthetic hand training platform for evaluation of closed-loop control of grasp force. Biorealistic controllers emulated a pair of antagonistic muscles that actuated the thumb and index fingers of the hand. Surface electromyographic (sEMG) signals from a pair of antagonistic residual muscles drove the biorealistic controllers. Tactile forces from fingertip sensors were conveyed to amputees through evoked tactile sensations (ETS) elicited at the projected finger map (PFM) areas of the stump. A forearm amputee subject participated in force tracking or holding tasks using the virtual hand with residual muscle EMGs, or the contralateral intact hand. Root-mean-square error (RMSE) was used as outcome measure of motor performance. Results in this subject showed that the biorealistic controller enabled the virtual hand to track and maintain grasping forces. The best performance in both tasks was achieved by the contralateral intact hand with visual feedback. The roles of visual or tactile feedback in force tracking or maintaining were also assessed with the virtual hand. For force holding task, hybrid tactile and visual feedback with biorealistic control had a better performance than single visual or tactile feedback in terms of RMSE, success rate, and force variability. While in the force pursuing task, tactile feedback did not seem to add visual feedback in following the target force. The study suggests that training may be required for a novel virtual hand user to perceive and integrate multiple modalities of feedback information, so as to optimize the closed-loop control ability.
The classification of motor imagery in non-invasive brain-computer interface (BCI) is a challenge due to the high variation of brain evoked responses across users and the non-stationarity properties of the electroencephalography (EEG) signal. With different sessions from the same user, it is possible to find substantial differences that require the BCI system to be recalibrated. In clinical settings, it is therefore necessary to know when a system should be recalibrated or when the system should adapt itself to deal with the shifts in the signal, i.e., the covariate shift, and/or catch artefacts that deviate substantially from the original data distribution. In this paper, we propose to use density based one-class classifiers using distances based on the Riemannian geometry framework for assessing the distribution of the EEG signal in motor imagery BCI. We assess the performance of the algorithms with a database of 14 participants. The results show that sessions from the same person can be reliably detected using the proposed approach. We also assess how the one-class classifiers can be used to determine if it is necessary to run domain adaptation in the test phase. The results support the conclusion that the accuracy improves as the system is adapted to shifting domains in signals.
Language processing in the brain comprises complex neurophysiology involving multiple regions and multimodal functions like cognition, memory, speech, etc. Recent studies show language processing is not restricted within anatomically established cortical language regions, and there is considerable inter-person variability. Clinically intraoperative functional language mapping is critical for studying patient-specific language processing and identifying the eloquent cortex to retain language functionalities during resection surgeries. Cortical auditory evoked potential (CAEP) induced by different auditory stimuli can provide a profound understanding of language processing. In this study, we present a low-cost, low-latency microcontroller-based audio system to induce and analyze CAEP. The system is established using off the shelf components; a 32bit Teensy (3.2) microcontroller, Teensy (3) audio shield etc. which makes it easily replicable. The Teensy can generate digital trigger locked to auditory stimulus onset to interface the system with biosignal amplifiers to record and study neural data simultaneously. The presented hardware tools enable the analysis of neural signals aligned with the audio stimuli with sub-millisecond temporal precision. Implementation of the system can provide a flexible platform to observe and analyze real-time cortical CAEP in the clinical settings. The system also provides great customization opportunities to study not only CAEP but also complex language processing tasks.
Microelectrodes are desired to deliver more charges to neural tissues while under electrochemical safety limits. Applying anodic bias potential during neurostimulation is a known technique for charge enhancement. Here, we investigated the levels of charge enhancement with anodic bias potential in vitro and in vivo using a custom-designed portable neurostimulator. We immersed our custom microelectrode probe in saline and measured voltage transients in response to constant current stimulation with and without a 500 mV anodic bias potential. We then inserted the same microelectrode probe into the primary motor cortex of the rat brain and measured voltage transients with the same electronics. Results showed that the charge injection capacity of the activated iridium oxide microelectrode site (with 2000 µm 2 geometric surface areas (GSAs)) increased by the use of the anodic bias potentials in both in vitro and in vivo: from 10 nC/phase to 32 nC/phase for 200 µs pulse widths, and from 2 nC/phase to 8 nC/phase, respectively. Thus, the order of charge injection capacities of the four cases tested in this study is as follows (from the lowest to the highest): in vivo without anodic bias, in vivo with anodic bias, in vitro without anodic bias, and in vitro with anodic bias. This work also validated in vivo use of our new portable neurostimulator which received stimulation waveforms wirelessly.
Intracortical brain computer interfaces (iBCIs) decode neural activity from the cortex and enable motor and communication prostheses, such as cursor control, handwriting and speech, for people with paralysis. This paper introduces a new iBCI communication prosthesis using a 3D keyboard interface for typing using continuous, closed loop movement of multiple fingers. A participant-specific BCI keyboard prototype was developed for a BrainGate2 clinical trial participant (T5) using neural recordings from the hand-knob area of the left premotor cortex. We assessed the relative decoding accuracy of flexion/extension movements of individual single fingers (5 degrees of freedom (DOF)) vs. three groups of fingers (thumb, index-middle, and ring-small fingers, 3 DOF). Neural decoding using 3 independent DOF was more accurate (95%) than that using 5 DOF (76%). A virtual keyboard was then developed where each finger group moved along a flexion-extension arc to acquire targets that corresponded to English letters and symbols. The locations of these letter/symbols were optimized using natural language statistics, resulting in an approximately a $2\times$ reduction in distance traveled by fingers on average compared to a random keyboard layout. This keyboard was tested using a simple real-time closed loop decoder enabling T5 to type with 31 symbols at 90% accuracy and approximately 2.3 sec/symbol (excluding a 2 second hold time) on average.
The brain-computer interface based on steady-state visual evoked potential (SSVEP) has received increasing attention due to its high information transfer rate and low subject variation. A major challenge of current SSVEP-BCI is the uncomfortableness and fatigue induced by the strong visual flicker. Thus, it is of vital importance to optimize SSVEP stimuli for a better user experience. Reducing the pixel density of stimuli is a promising method to improve SSVEP. However, it remains unknown how the neural responses would be when faced with low-pixel density stimuli, and it is also unclear whether the corresponding decoding accuracy can be improved or not. Hereto, this study investigated neural responses induced by the stimuli with distinct pixel densities (1%, 10%, 20%, 60%, 100%) under both low (8Hz, 15Hz) and high frequencies (33Hz, 40Hz, 60Hz), responses from parietal-occipital area were recorded by functional near-infrared spectroscopy (fNIRS) and electroencephalo-gram (EEG) concurrently, aiming to have a better understanding of low-pixel-density-related responses. As a result, the behavioral performance showed that the comfort index inclined as the pixel density became lower. EEG and fNIRS signal analysis indicated that 20%-pixel induced larger EEG and fNIRS response than most densities in the low-frequency band. As to classification, comparing to the 100%, classification accuracy of 20%-pixel density classifies significantly better in low-frequency and high-frequency bands, whether in EEG, fNIRS, or hybrid. The maximum classification accuracy of 20%-density can reach 97.66% in hybrid binary classification, with 3.77% more than 100% density. This research provides a theoretical and technical basis for developing user-friendly SSVEP-BCI.
The role of high-frequency oscillations (HFO) has been established in a multitude of the brain functions such as retrieval and consolidation of memory. Moreover, HFOs have been identified as a biomarker for pathological brain conditions, including epileptogenicity. Therefore, there has been a continuous effort to reliably detect and characterize HFOs. Here, we present an unsupervised HFO detector using characteristics of signals in the time-frequency domain obtained by continuous wavelet transform. By using L1 normalization for continuous wavelet transform, we improved the detection of HFOs without the need to normalize time-frequency maps. The elimination of normalizing the time-frequency maps reduces the computational cost of the analysis. We used two different benchmark datasets available in the literature to validate our proposed automatic HFO detector. The results demonstrate that our detector outperforms other commonly available HFO detectors including those that use timefrequency maps. Our HFO detector shows superior performance especially when signal-to-noise ratio (SNR) is low. Moreover, our detector can simultaneously detect artifacts, physiological spikes, and provide useful information about the HFOs such as their dominant frequency of oscillation, their average amplitude and their duration. This information can later be utilized to stratify HFOs for further analysis. Changes in HFO characteristics may be utilized as biomarkers in pathological conditions such as posttraumatic epilepsy.
Chronic functionality of neural interfaces (NI) is hampered by the physiological response to foreign objects, in part due to the mismatch of mechanical properties between soft neural tissue and the rigid materials used in interface construction. Polymer-based NIs have emerged as a key new technology in the pursuit of chronically stable neural recording and stimulation, but most polymer NIs are bespoke devices developed as part of specific research missions; many researchers do not have access to polymer-based NIs technology and among those who do there is a severe lack of standardization in material, construction, packaging, and testing, leading to a lack of repeatability among datasets. Here we present the Polymer Implantable Electrode (PIE) Foundry, a shared-resource for fabricating and disseminating standardized polymer-based microelectrode arrays for use in NIs. The model is based on the successful shared prototyping concept developed for the field of semiconductor research. Professional staff, supported by the BRAIN Initiative funding and operating in cleanroom space provided by the University of Southern California, offer design, fabrication, packaging, and testing of polymer-based microelectrode arrays as a free service to academic and non-profit research groups. The core enabling technology is a standardized set of micromachining protocols applied to the biocompatible, thin-film polymer Parylene C. By leveraging this method, we produce microelectrode arrays of varied size, shape, channel count, and application, disseminating hundreds of arrays to 18+ research groups in our first three years of operation. By standardizing materials, fabrication, and packaging, we create repeatable and comparable devices and have built a library of shareable designs. Channel counts range from 2 to 64, electrode sizes range from 15 μm diameter to 1 mm, designs include penetrating neural probes, spinal paddle electrodes, surface arrays for electroencephalography, and peripheral nerve cuffs for recording and stimulation, animal models include songbird, mouse, rat, cat, and sheep. Here we present details of our organizational structure, fabrication and packaging methods, representative examples of ex vivo and in vivo electrode performance, and key results from the first three years of Foundry operation.
Precise timing prediction is the ability to estimate time in millisecond timescale, it can speed up behavior, optimize perception, benefit adaptive behaviors. Behavioral studies have demonstrated training can improve the performance of precise timing prediction. However, neural evidence is still lacking in describing how training changes the neural characteristics of precise timing prediction. This study designed a cue-tapping (task1)/ timing-in-mind (task2) experiment, collected behavioral and electroencephalogram (EEG) data of 24 subjects in both before and after training period. Sample entropy (SampEn) and Lempel-Ziv complexity (LZC) were calculated to measure EEG complexity, functional connectivity based on phase locking value was also involved. Consequently, behavioral results showed that error time declined and accuracy rate increased as training progressed. SampEn was much smaller after training in almost all frequency-bands in task1, and reduced in task2 as well. LZC showed a decreased tendency after training, but no statistical significance was found. Moreover, after training, much stronger functional connectivity was found in low frequency-band in both tasks. The results can shed light on the modeling of training and precise predictive timing.
Brain stimulation has become an important treatment option for a variety of neurological and psychiatric diseases. A key challenge in improving brain stimulation is selecting the optimal set of stimulation parameters for each patient, as parameter spaces are too large for brute-force search and their induced effects can exhibit complex subject-specific behavior. To achieve greatest effectiveness, stimulation parameters may additionally need to be adjusted based on an underlying neural state, which may be unknown, unmeasurable, or challenging to quantify a priori. In this study, we first develop a simulation of a state-dependent brain stimulation experiment using rodent optogenetic stimulation data. We then use this simulation to demonstrate and evaluate two implementations of an adaptive Bayesian optimization algorithm that can model a dynamically changing response to stimulation parameters without requiring knowledge of the underlying neural state. We show that, while standard Bayesian optimization converges and overfits to a single optimal set of stimulation parameters, adaptive Bayesian optimization can continue to update and explore as the neural state is changing and can provide more accurate optimal parameter estimation when the optimal stimulation parameters shift. These results suggest that learning algorithms such as adaptive Bayesian optimization can successfully find optimal state-dependent stimulation parameters, even when brain sensing and decoding technologies are insufficient to track the relevant neural state.
This paper, for the first time, compares the behaviors of nonlinear versus linear muscle networks in decoding hidden peripheral synergistic neural patterns during dynamic functional tasks. In this paper, we report a case study during which one healthy subject conducts a series of four lower limb repetitive tasks. Specifically, the paper focuses on tasks that involve the right knee joint, including walking, sit-to-stand, stepping, and drop-jump. Twelve muscles were recorded using the Delsys Trigno system. The linear muscle network was generated using coherence analysis, and the nonlinear network was generated using Spearman's correlation. The results show that the degree, clustering coefficient, and global efficiency of the muscle network have the highest value among tasks in the linear domain for the walking task, while a low linear synergistic network behavior for the sit-to-stand is observed. On the other hand, the results show that the nonlinear functional muscle network decodes high connectivity (degree) and clustering coefficient and efficiency for the sit-to-stand when compared with other tasks. We have also developed a two-dimensional functional connectivity plane composed of linear and nonlinear features and shown that it can span the lower-limb dynamic task space. The results of this paper for the first time highlight the importance of observing both linear and nonlinear connectivity patterns, especially for complex dynamic tasks. It should also be noted that through a simultaneous EEG recording (using Brain Vision System), we have shown that, indeed, cortical activity may indirectly explain highly-connected nonlinear muscle network for the sit-to-stand task, highlighting the importance of nonlinear muscle network as a neurophysiological window of observation beyond the periphery.
Although numerous treatments are available for major depressive disorder (MDD), patients can be refractory to sequential treatment regimens. Experimental studies have demonstrated promising results implementing deep brain stimulation (DBS) as a therapy for treatment resistant MDD. However, optimization of this technique requires repeated assessments of the clinical effects of treatment in each patient and the ability to reliably capture the complexity and dynamics of depression symptoms. In our initial studies evaluating the feasibility and preliminary efficacy of a novel closed-loop DBS (CL-DBS) approach, we have observed that repeated self-rated MDD metrics can be burdensome to complete and may not provide accurate measures of symptom severity fluctuations over time, making the identification of neural biomarkers of MDD a challenge. To address this, we evaluated if text analysis could identify linguistic indicators of depression, including providing insights into symptom severity. Using the Linguistic Inquiry and Word Count software, we analyzed written symptom reports from one patient in clinical trial for CL-DBS. We found significant linguistic predictors of depression symptoms that were associated with the same frequency- and region- specific spectral power correlates found when assessing symptoms captured by self-rated depression metrics. These preliminary findings suggest that the close association between language use and symptom strength could be utilized to detect neural biomarkers of depression and potentially to assess treatment outcome.
Deep brain stimulation (DBS) delivers electrical stimulation directly to brain tissue to treat neurological movement disorders such as Parkinson’s Disease (PD). Adaptive DBS (aDBS) is an advancement on DBS that uses symptom-related biomarkers to adjust therapeutic stimulation parameters in real time to improve clinical outcomes and reduce side-effects. A significant challenge for the field of aDBS is developing automated methods to optimize stimulation parameters using remote assessments of symptom severity. To address this challenge, we designed a prototype at-home data collection platform that can remotely update aDBS algorithms and explore objective assessments of motor symptom severity. Our platform collects neural, inertial, and video data, and supports clinician validation of automated symptom assessments. We deployed the system to the home of an individual with PD and collected pilot data across six days. We evaluated motor symptom severity by recording data with stimulation amplitudes set to varying levels during self-guided clinical tasks and free behavior. We assessed movement features including frequency, speed, and peak angular velocity from video-derived pose estimates and inertial data during three clinical tasks. All features showed a reduction during periods of under-stimulation and were significantly correlated with video-based clinical scores of symptom severity (Spearman rank test, p < 0.006). These results demonstrate that our prototype is capable of remote multimodal data collection and that these data can enhance aDBS research outside the clinic.
Speech imagery brain computer interfaces (BCIs) empower people with motor disorders to communicate their thoughts and intentions to their environment in a natural, user-friendly way. Due to its portability, non-invasiveness, safety, and low cost, functional near-infrared spectroscopy (fNIRS) is advanced as a brain imaging method for developing BCIs. To improve operators' speech imagery BCI performance, a new paradigm was studied in this work by modifying imagined lexical tone in Mandarin speech imagery BCIs motivated by the fact that speech processing with perceptual pitch variation exhibited more extensive activations in brain than that without perceptual pitch variation. For the traditional ternary yes and no Mandarin speech imagery paradigm, the subjects were asked to answer questions by covertly repeating the Mandarin syllable ('yes' in English, and with tone 4 in Mandarin) or ‘ (‘no’ in English, and with tone 3 in Mandarin) in mind and an unconstrained rest task was also contained. In the proposed paradigm, the subjects were asked to imagine with tone 1 and with tone 3. Compared with the traditional paradigm, tone difference in Mandarin speech imagery of the proposed paradigm could generate more discriminative brain activities among different tasks, and the mean classification accuracy was significantly improved from 49.06% to 53.85%. These results suggest that modifying imagined lexical tone in Mandarin speech imagery could influence brain activation and is promising for improving the decoding accuracy of Mandarin speech imagery BCIs.
Finding points in time where the distribution of neural responses changes (change points) is an important step in many neural data analysis pipelines. However, in complex and free behaviors, where we see different types of shifts occurring at different rates, it can be difficult to use existing methods for change point (CP) detection because they can't necessarily handle different types of changes that may occur in the underlying neural distribution. Additionally, response changes are often sparse in high dimensional neural recordings, which can make existing methods detect spurious changes. In this work, we introduce a new approach for finding changes in neural population states across diverse activities and arousal states occurring in free behavior. Our model follows a contrastive learning approach: we learn a metric for CP detection based on maximizing the Sinkhorn divergences of neuron firing rates across two sides of a labeled CP. We apply this method to a 12-hour neural recording of a freely behaving mouse to detect changes in sleep stages and behavior. We show that when we learn a metric, we can better detect change points and also yield insights into which neurons and sub-groups are important for detecting certain types of switches that occur in the brain.
We present an automated quad-channel patch-clamp technology platform for ex vivo brain slice electrophysiology, capable of both blind and two-photon targeted robotically automated patching. The robot scales up the patch-clamp single-cell recording technique to four simultaneous channels, with seal success rates for two-photon targeted and blind modes of 54% and 68% respectively. In 50% of targeted trials (where specific cells were required), two simultaneous recordings or more were obtained. For blind mode, most trials yielded dual or triple recordings. This robot, a milestone on the path to a true in vivo targeted robotic multi-patching technology platform, will allow numerous studies into the function and connectivity patterns of both primary and secondary cell types.
This work presents a spiking neural network for predicting kinematics from neural data towards accurate and energy-efficient brain machine interface. A brain machine interface is a technological system that interprets neural signals to allow motor impaired patients to control prosthetic devices. Spiking neural networks have the potential to improve brain machine interface technology due to their low power cost and close similarity to biological neural structures. The SNN in this study uses the leaky integrate-and-fire model to simulate the behavior of neurons, and learns using a local learning method that uses surrogate gradient to learn the parameters of the network. The network implements a novel continuous time output encoding scheme that allows for regression-based learning. The SNN is trained and tested offline on neural and kinematic data recorded from the premotor cortex of a primate and the hippocampus of a rat. The model is evaluated by finding the correlation between the predicted kinematic data and true kinematic data, and achieves peak Pearson Correlation Coefficients of 0.77 for the premotor cortex recordings and 0.80 for the hippocampus recordings. The accuracy of the model is benchmarked against a Kalman filter decoder and a LSTM network, as well as a spiking neural network trained with backpropagation to compare the effects of local learning.
