Naming brain–computer interfaces according to their intended application will assist stakeholders in the evaluation of the benefits and risks of neurotechnologies.

Our brains evolved to help us rapidly learn new things. But anyone who has put in hours of practice to perfect their tennis serve, only to reach a plateau, can attest that our brains aren’t infinitely flexible. New work shows that patterns of neural activity over time — the temporal dynamics of neural populations — cannot change rapidly, suggesting that neural activity dynamics may both reflect and constrain how the brain performs computations.

Goal-directed reaches give rise to dynamic neural activity across the brain as we move our eyes and arms, and process outcomes. High spatiotemporal resolution mapping of multiple cortical areas will improve our understanding of how these neural computations are spatially and temporally distributed across the brain. In this study, we used micro-electrocorticography (µECoG) recordings in two male monkeys performing visually guided reaches to map information related to eye movements, arm movements, and receiving rewards over primary motor cortex, premotor cortex, frontal eye field, and dorsolateral pre-frontal cortex. Time-frequency and decoding analyses revealed that eye and arm movement information shifts across brain regions during a reach, likely reflecting shifts from planning to execution. Although eye and arm movement temporally overlapped, phase clustering analyses enabled us to resolve differences in eye and arm information across brain regions. This analysis revealed that eye and arm information spatially overlapped in motor cortex, which we further confirmed by demonstrating that arm movement decoding performance from motor cortex activity was impacted by task-irrelevant eye movements. Phase clustering analyses also identified reward-related activity in the pre-frontal and premotor cortex. Our results demonstrate µECoG's strengths for functional mapping and provide further detail on the spatial distribution of eye, arm, and reward information processing distributed across frontal cortices during reaching. These insights advance our understanding of the overlapping neural computations underlying coordinated movements and reveal opportunities to leverage these signals to enhance future brain-computer interfaces.Significance statement Picking up your coffee mug requires coordinating movements of your eyes and hand and processing the outcomes of those movements. Mapping how neural activity relates to different functions helps us understand how the brain performs these computations. Many mapping techniques have limited spatial or temporal resolution, restricting our ability to dissect computations that overlap closely in space and time. We used micro-electrocorticography recordings to map neural activity across multiple cortical areas while monkeys made goal-directed reaches. These measurements revealed high spatial and temporal resolution maps of neural activity related to eye, arm, and reward information processing. These maps reveal overlapping neural computations underlying movement and open opportunities to use eye and reward information to improve therapies to restore motor function.

Intracortical Brain-Computer Interfaces (iBCI) aim to decode behavior from neural population activity, enabling individuals with motor impairments to regain motor functions and communication abilities. A key challenge in long-term iBCI is the nonstationarity of neural recordings, where the composition and tuning profiles of the recorded populations are unstable across recording sessions. Existing methods attempt to address this issue by explicit alignment techniques; however, they rely on fixed neural identities and require test-time labels or parameter updates, limiting their generalization across sessions and imposing additional computational burden during deployment. In this work, we introduce SPINT - a Spatial Permutation-Invariant Neural Transformer framework for behavioral decoding that operates directly on unordered sets of neural units. Central to our approach is a novel context-dependent positional embedding scheme that dynamically infers unit-specific identities, enabling flexible generalization across recording sessions. SPINT supports inference on variable-size populations and allows few-shot, gradient-free adaptation using a small amount of unlabeled data from the test session. To further promote model robustness to population variability, we introduce dynamic channel dropout, a regularization method for iBCI that simulates shifts in population composition during training. We evaluate SPINT on three multi-session datasets from the FALCON Benchmark, covering continuous motor decoding tasks in human and non-human primates. SPINT demonstrates robust cross-session generalization, outperforming existing zero-shot and few-shot unsupervised baselines while eliminating the need for test-time alignment and fine-tuning. Our work contributes an initial step toward a robust and scalable neural decoding framework for long-term iBCI applications.

Background Neuroscience researchers often surgically implant hardware into model organisms to measure and manipulate neural activity. Designing and optimizing these procedures in non-human primates often requires sedated or euthanized animals. Artificial tissue technologies can reduce animal use in this process, but existing simulators do not include all relevant tissues and do not facilitate iterative design processes. New method We created a comprehensive, customizable, and modular surgical simulator for neuroscience research. Our simulator incorporates artificial skull, brain, and soft tissues (skin and muscle) into one 3-dimensional model with adaptable components. Results Incorporating 3-dimensional soft tissues enabled surgical and implant design improvements, which may contribute to improving implant longevity, research outcomes, and animal wellbeing. Our modular design allowed researchers to rapidly prototype designs and exchange parts to reflect implant or anatomical changes across a study. Incorporating all relevant tissues also enabled surgical practice and improved communication with veterinarians. Our approach is low-cost (a few hundred dollars) and uses readily available tools like 3D printing. We also provide models of different non-human primate species to increase access to our approach. Comparison with existing methods Our method improves upon past surgical simulators for neuroscience research by: adapting existing skin and muscle artificial tissue technologies to more accurately represent cranial 3-dimensional geometry, incorporating models of all tissues relevant for implant design, and introducing modular designs that increase flexibility/customization. Conclusions We found that this surgery simulator was an inexpensive tool that was useful for planning and practicing surgical procedures, as well as prototyping new neuroscience experiment hardware.

We present an overview of the Conference on Transformative Opportunities for Modeling in Neurorehabilitation held in March 2023. It was supported by the Disability and Rehabilitation Engineering (DARE) program from the National Science Foundation’s Engineering Biology and Health Cluster. The conference brought together experts and trainees from around the world to discuss critical questions, challenges, and opportunities at the intersection of computational modeling and neurorehabilitation to understand, optimize, and improve clinical translation of neurorehabilitation. We organized the conference around four key, relevant, and promising Focus Areas for modeling: Adaptation Plasticity, Personalization, Human-Device Interactions, and Modeling ‘In-the-Wild’. We identified four common threads across the Focus Areas that, if addressed, can catalyze progress in the short, medium, and long terms. These were: (i) the need to capture and curate appropriate and useful data necessary to develop, validate, and deploy useful computational models (ii) the need to create multi-scale models that span the personalization spectrum from individuals to populations, and from cellular to behavioral levels (iii) the need for algorithms that extract as much information from available data, while requiring as little data as possible from each client (iv) the insistence on leveraging readily available sensors and data systems to push model-driven treatments from the lab, and into the clinic, home, workplace, and community. The conference archive can be found at (dare2023.usc.edu). These topics are also extended by three perspective papers prepared by trainees and junior faculty, clinician researchers, and federal funding agency representatives who attended the conference.

Myoelectric interfaces hold promise in consumer and health applications, but they are currently limited by variable performance across users and poor generalizability across tasks. To address these limitations, we consider interfaces that continually adapt during operation. Although current adaptive interfaces can reduce inter-subject variability, they still generalize poorly between tasks because they make use of task-specific data during training. To address this limitation, we propose a new paradigm to adapt myoelectric interfaces using natural eye gaze as training data. We recruited 11 subjects to test our proposed method on a 2D computer cursor control task using high-density surface EMG signals measured from forearm muscles. We find comparable task performance between our gaze-trained paradigm and the current task-dependent method. This result demonstrates the feasibility of using eye gaze to replace task-specific training data in adaptive myoelectric interfaces, holding promise for generalization across diverse computer tasks.CCS Concepts • Human-centered computing → Interaction devices; Empirical studies in HCI. ### Competing Interest StatementThe authors have declared no competing interest.

Neural interfaces can restore or augment human sensorimotor capabilities by converting high-bandwidth biological signals into control signals for an external device via a decoder algorithm. Leveraging user and decoder adaptation to create co-adaptive interfaces presents opportunities to improve usability and personalize devices. However, we lack principled methods to model and optimize the complex two-learner dynamics that arise in co-adaptive interfaces. Here, we present new computational methods based on control theory and game theory to analyze and generate predictions for user-decoder co-adaptive outcomes in continuous interactions. We tested these computational methods using an experimental platform where human participants (N=14) learn to control a cursor using an adaptive myoelectric interface to track a target on a computer display. Our framework predicted the outcome of co-adaptive interface interactions and revealed how interface properties can shape user behavior. These findings contribute new tools to design personalized, closed-loop, co-adaptive neural interfaces.### Competing Interest StatementA. L. Orsborn is a scientific advisor for Meta Reality Labs.

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using Brain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. The dimensionality of the population of neurons controlling the BCI remained constant or increased with learning, counter to expected trends from motor learning. Yet, over time, task information was contained in a smaller subset of neurons or population modes. Moreover, task information was ultimately stored in neural modes that occupied a small fraction of the population variance. An artificial neural network model suggests the adaptive decoders contribute to forming these compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations, like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

The article “In vitro neurons learn and exhibit sentience when embodied in a simulated game-world” by Kagan et al. 1 Kagan B.J. Kitchen A.C. Tran N.T. Habibollahi F. Khajehnejad M. Parker B.J. Bhat A. Rollo B. Razi A. Friston K.J. In vitro neurons learn and exhibit sentience when embodied in a simulated game-world. Neuron. 2022; 110: 3952-3969.e8https://doi.org/10.1016/j.neuron.2022.09.001 Abstract Full Text Full Text PDF PubMed Scopus (17) Google Scholar triggered a wave of positive mainstream and scientific media coverage as well as a widespread negative reaction from the scientific community. Here, we discuss why this negative reaction is legitimate and must be taken seriously. We raise concerns about the key claim of the article: that it demonstrates that “a single layer of in vitro cortical neurons can self-organize activity to display intelligent and sentient behavior when embodied in a simulated game-world.” Scientific communication and the semantics of sentienceKagan et al.NeuronMarch 01, 2023In BriefThe use of language to describe specific phenomena has always been, and will likely remain, a contentious aspect of scientific discourse. Effective scientific communication must be considered in the context of the field in which the given signifiers are used. While the response by Balci et al. may be described as polemic and contains reasoning subject to equivocation and other fallacies, we do appreciate the core concerns. Here, we address these concerns and suggest some constructive pathways that might help improve scientific communication in the future. Full-Text PDF In vitro neurons learn and exhibit sentience when embodied in a simulated game-worldKagan et al.NeuronOctober 12, 2022In BriefThe DishBrain system is the first real-time synthetic biological intelligence platform that demonstrates that biological neurons can adjust firing activity in a way that suggests the ability to learn to perform goal-oriented tasks when provided with simple electrophysiological sensory input and feedback while embodied in a game-world. Full-Text PDF Open AccessConceptual conundrums for neuroscienceRommelfanger et al.NeuronMarch 01, 2023In BriefIn their recently published work, Kagan and colleagues argue that cultures of human and mouse neurons can exhibit goal-directed activity adaptations and are thus “sentient.” In their commentary to this piece, Balci and colleagues1 offer a critical review of the work. In addition to discussing the value of the research and the soundness of the methodology, they question the appropriateness of using language such as “sentience” in the context of the study. Research in isolated neural networks provides crucial knowledge, Balci and colleagues note, but the terms that Kagan et al. Full-Text PDF

Brain-machine interfaces (BMIs) aim to treat sensorimotor neurological disorders by creating artificial motor and/or sensory pathways. Introducing artificial pathways creates new relationships between sensory input and motor output, which the brain must learn to gain dexterous control. This review highlights the role of learning in BMIs to restore movement and sensation, and discusses how BMI design may influence neural plasticity and performance. The close integration of plasticity in sensory and motor function influences the design of both artificial pathways and will be an essential consideration for bidirectional devices that restore both sensory and motor function.

Latent Variable Models (LVMs) propose to model the dynamics of neural populations by capturing low-dimensional structures that represent features involved in neural activity. Recent LVMs are based on deep learning methodology where a deep neural network is trained to reconstruct the same neural activity given as input and as a result to build the latent representation. Without taking past or future activity into account such a task is non-causal. In contrast, the task of forecasting neural activity based on given input extends the reconstruction task. LVMs that are trained on such a task could potentially capture temporal causality constraints within its latent representation. Forecasting has received less attention than reconstruction due to recording challenges such as limited neural measurements and trials. In this work, we address modeling neural population dynamics via the forecasting task and improve forecasting performance by including a prior, which consists of pairwise neural unit interaction as a multivariate dynamic system. Our proposed model---Additive, Multiplicative, and Adaptive Graph Neural Network (AMAG)---leverages additive and multiplicative message-passing operations analogous to the interactions in neuronal systems and adaptively learns the interaction among neural units to forecast their future activity. We demonstrate the advantage of AMAG compared to non-GNN based methods on synthetic data and multiple modalities of neural recordings (field potentials from penetrating electrodes or surface-level micro-electrocorticography) from four rhesus macaques. Our results show the ability of AMAG to recover ground truth spatial interactions and yield estimation for future dynamics of the neural population.

Summary It has been proposed that the nervous system has the capacity to generate a wide variety of movements because it re-uses some invariant code. Previous work has identified that dynamics of neural population activity are similar during different movements, where dynamics refer to how the instantaneous spatial pattern of population activity changes in time. Here we test whether invariant dynamics of neural populations are actually used to issue the commands that direct movement. Using a brain-machine interface that transformed rhesus macaques’ motor cortex activity into commands for a neuroprosthetic cursor, we discovered that the same command is issued with different neural activity patterns in different movements. However, these different patterns were predictable, as we found that the transitions between activity patterns are governed by the same dynamics across movements. These invariant dynamics are low-dimensional, and critically, they align with the brain-machine interface, so that they predict the specific component of neural activity that actually issues the next command. We introduce a model of optimal feedback control that shows that invariant dynamics can help transform movement feedback into commands, reducing the input that the neural population needs to control movement. Altogether our results demonstrate that invariant dynamics drive commands to control a variety of movements, and show how feedback can be integrated with invariant dynamics to issue generalizable commands.

User-machine interfaces map biological signals measured from the user to control commands for external devices. The mapping from biosignals to device inputs is performed by a decoding algorithm. Adaptation of both the user and decoder-co-adaptation-provides opportunities to improve the inclusivity and usability of interfaces for diverse users and applications. User learning leads to robust interface control that can generalize across environments and contexts. Decoder adaptation can personalize interfaces, account for day-to-day signal variability, and improve overall performance. Co-adaptation therefore creates opportunities to shape the user and decoder system to achieve robust and generalizable personalized interfaces. However, co-adaptation creates a two-learner system with dynamic interactions between the user and decoder. Engineering co-adaptive interfaces requires new tools and frameworks to analyze and design user-decoder interactions. In this article, we review adaptive decoding, user learning, and co-adaptation in user-machine interfaces, primarily brain-computer, myoelectric, and kinematic interfaces, for motor control. We then discuss performance criteria for coadaptive interfaces and propose a game-theoretic approach to designing user-decoder co-adaptation.

A bstract Neural activity tends to reside on manifolds whose dimension is lower than the dimension of the whole neural state space. Experiments using brain-computer interfaces (BCIs) with microelectrode arrays implanted in the motor cortex of nonhuman primates have provided ways to test whether neural manifolds influence learning-related neural computations. Starting from a learned BCI-controlled motor task, these experiments explored the effect of changing the BCI decoder to implement perturbations that were either “aligned” or not with the pre-existing neural manifold. In a series of studies, researchers found that within-manifold perturbations (WMPs) evoked fast reassociations of existing neural patterns for rapid adaptation, while outside-manifold perturbations (OMPs) triggered a slower adaptation process that led to the emergence of new neural patterns. Together, these findings have been interpreted as suggesting that these different rates of adaptation might be associated with distinct learning mechanisms. Here, we investigated whether gradient-descent learning could alone explain these differences. Using an idealized model that captures the fixed-point dynamics of recurrent neural networks, we uncovered gradient-based learning dynamics consistent with experimental findings. Crucially, this experimental match arose only when the network was initialized in a lazier learning regime, a concept inherited from deep learning theory. A lazy learning regime—in contrast with a rich regime—implies small changes on synaptic strengths throughout learning. For OMPs, these small changes were less effective at increasing performance and could lead to unstable adaptation with a heightened sensitivity to learning rates. For WMPs, they helped reproduce the reassociation mechanism on short adaptation time scales, especially with large input variances. Since gradient descent has many biologically plausible variants, our findings establish lazy gradient-based learning as a plausible mechanism for adaptation under network-level constraints and unify several experimental results from the literature.

Neural interfaces provide novel opportunities for augmenting human capabilities in domains like human-machine interaction, brain-computer interfaces, and rehabilitation. However, the performance of these interfaces varies significantly across users. Decoders that adapt to individual users have the potential to reduce variability and improve performance but introduce a "two-learner" problem as the user simultaneously adapts to the changing decoder. We propose and experimentally test a game-theoretic framework to optimize closed-loop performance of a myoelectric interface for continuous control (based on surface electromyography, sEMG) through co-adaptation of the user and decoder. Human subjects learned to use our interface to perform a two-dimensional trajectory-tracking task. Closed-loop performance was affected by decoder learning rate but not by initialization or decoder cost weights. Our study indicates the potential for co-adaptation in humans and machines to optimize the performance of neural interfaces.

Objective Neural dynamical models reconstruct neural data using dynamical systems. These models enable direct reconstruction and estimation of neural time-series data as well as estimation of neural latent states. Nonlinear neural dynamical models using recurrent neural networks in an encoder-decoder architecture have recently enabled accurate single-trial reconstructions of neural activity for neuronal spiking data. While these models have been applied to neural field potential data, they have only so far been applied to signal feature reconstruction (e.g. frequency band power), and have not yet produced direct reconstructions of broadband time-series data preserving signal phase and temporal resolution. Approach Here we present two encoder-decoder model architectures - the RNN autoencoder (RAE) and multi-block RAE (MRAE) for direct time-series reconstruction of broadband neural data. We trained and tested models on multi-channel micro-Electricorticography ( μ ECoG) recordings from non-human primate motor corticies during unconstrained behavior. Main Results We show that RAE reconstructs micro-electrocorticography recordings, but has reconstruction accuracy that is band-limited to model scale. The MRAE architecture overcomes these time-bandwidth restrictions, yielding broadband (0-100 Hz), accurate reconstructions of μ ECoG data. Significance RAE and MRAE reconstruct broadband μ ECoG data through multiblock dynamical modeling. The MRAE overcomes time-bandwitdh restrictions to provide improved accuracy for long time duration signals. The reconstruction capabilities provided by these models for broadband neural signals like μ ECoG may enable the development of improved tools and analysis for basic scientific research and applications like brain-computer interfaces.

Connectivity is key to understanding neural circuit computations. However, estimating in vivo connectivity using recording of activity alone is challenging. Issues include common input and bias errors in inference, and limited temporal resolution due to large data requirements. Perturbations (e.g. stimulation) can improve inference accuracy and accelerate estimation. However, optimal stimulation protocols for rapid network estimation are not yet established. Here, we use neural network simulations to identify stimulation protocols that minimize connectivity inference errors when using generalized linear model inference. We find that stimulation parameters that balance excitatory and inhibitory activity minimize inference error. We also show that pairing optimized stimulation with adaptive protocols that choose neurons to stimulate via Bayesian inference may ultimately enable rapid network inference.

Objective. Complex spatiotemporal neural activity encodes rich information related to behavior and cognition. Conventional research has focused on neural activity acquired using one of many different measurement modalities, each of which provides useful but incomplete assessment of the neural code. Multi-modal techniques can overcome tradeoffs in the spatial and temporal resolution of a single modality to reveal deeper and more comprehensive understanding of system-level neural mechanisms. Uncovering multi-scale dynamics is essential for a mechanistic understanding of brain function and for harnessing neuroscientific insights to develop more effective clinical treatment.Approach. We discuss conventional methodologies used for characterizing neural activity at different scales and review contemporary examples of how these approaches have been combined. Then we present our case for integrating activity across multiple scales to benefit from the combined strengths of each approach and elucidate a more holistic understanding of neural processes.Main results. We examine various combinations of neural activity at different scales and analytical techniques that can be used to integrate or illuminate information across scales, as well the technologies that enable such exciting studies. We conclude with challenges facing future multi-scale studies, and a discussion of the power and potential of these approaches.Significance. This roadmap will lead the readers toward a broad range of multi-scale neural decoding techniques and their benefits over single-modality analyses. This Review article highlights the importance of multi-scale analyses for systematically interrogating complex spatiotemporal mechanisms underlying cognition and behavior.

Co-adaptation in brain-machine interfaces (BMIs) can improve performance and facilitate user learning. We propose and analyze a mathematical model for co-adaptation in BMIs. We model the brain and the decoder as strategic agents who seek to minimize their individual cost functions, leading to a game-theoretic formulation of interaction. We frame our BMI model as a potential game to identify stationary points (Nash equilibria) of the brain-decoder interactions, which correspond to points at which both the brain and the decoder stop adapting. Assuming the brain and the decoder adapt using gradient-based schemes, we analytically show how convergence to these equilibria depends on agent learning rates. This theoretical framework presents a basis for modeling co-adaptation using dynamic game theory and can be extended to tasks with multiple dimensions and to different decoder models. This framework can ultimately be used to develop robust adaptive decoder designs to shape brain learning and optimize BMI performance.