Analyzing the topological patterns of brain networks is of critical importance for the diagnosis of neurodegenerative diseases such as Alzheimer’s disease and epilepsy. Brain networks constructed from fMRI data provide critical insights into functional connectivity changes underlying these conditions. While traditional methods have yielded valuable insights into abnormal brain network patterns, they fall short in capturing the state transitions of brain networks across different disease states. To effectively characterize these transitions, we propose a novel Wasserstein prototypical contrastive learning model for brain network classification, which integrates brain topological difference weighting. Specifically, we propose a brain functional convolutional layer that incorporates both intra-cluster and inter-cluster feature interactions to effectively capture the variations in brain networks. Furthermore, we employ optimal transport to represent the differences between brain networks and utilize the Wasserstein distance to incorporate these variations into the model. Experimental results show that our method outperforms existing approaches across different datasets, significantly enhancing both disease detection accuracy and the biological interpretability of the model. These improvements offer novel insights for the early detection and personalized treatment of brain diseases.
Occlusion-induced partial observability is a central challenge in multi-vehicle traffic. Physical occluders such as trucks and buildings can hide safety-critical agents and motion cues, creating systematic blind spots in local onboard observations. Such structural uncertainty makes end-to-end cooperative driving difficult, especially under distribution shifts. To address these limitations, we propose a task-grounded cooperative driving framework for occlusion-prone urban environments. Compact cooperative messages are learned from raw pixels and grounded in task consequences by jointly predicting rewards and next-step dynamics. To operationalize this grounding, we develop a bisimulation-inspired reward-dynamics coupled predictive metric, which suppresses nuisance appearance variations while preserving task-relevant interaction factors. In addition, we promote behavioral diversity by jointly optimizing policy entropy and expected return, balancing individual diversity with global cooperation. The method is evaluated in CARLA with five randomized training scenario families and six held-out out-of-distribution (OOD) tests covering density shifts, adverse visual-condition shifts, truck-induced physical occlusions, and a compositional multi-factor shift. Experimental results show consistent gains in success rate, with lower collision rate and shorter completion time over representative baselines, demonstrating robust end-to-end cooperative driving in complex urban environments.
Over the past two decades, visual–inertial navigation systems (VINS) have advanced considerably. However, many state-of-the-art methods rely on deep learning modules or semantic priors, which introduce substantial computational overhead and often suffer in environments with numerous dynamic objects or frequent occlusions. In this paper, we present HD-VINS, a method that improves robustness in highly dynamic scenes by decoupling multivariate dynamic–static residual factors, without requiring pre-trained models or scene-specific learning. Specifically, HD-VINS employs a dynamic–static kinematic model that uses IMU prior poses as short-term motion cues to filter visual features based on their consistency. In addition, a dual-factor weighting strategy enhances robustness against abnormal residual fluctuations during aggressive maneuvers. Furthermore, by explicitly modeling the motion of dynamic features, HD-VINS separates their effects into frame-wise residuals. These are then transformed into residual terms augmented with auxiliary compensation variables, allowing for controlled and stable incorporation into a sliding-window optimization framework. Comprehensive experiments on the VIODE and OpenLORIS-Scene benchmarks demonstrate that HD-VINS delivers competitive performance in highly dynamic environments with moving objects, partial occlusions, and challenging illumination conditions.
Assistive robots should be responsive to user needs and avoid undermining their sense of agency, especially in intimate tasks such as robot-assisted feeding, to avoid user intimidation and passivity. While behavioural adaptation has been used to optimise collaboration, little is known about how when adaptation occurs shapes user perception. We investigate this by comparing a Stable-Adaptive policy that prioritises predictability with a Reactive-Adaptive policy that adapts to user state as it emerges, emphasising responsiveness. We design and implement a reactive adaptation mechanism that enables low-latency, within-task behavioural updates based on continuous user state inference. Both policies are deployed on a bimanual feeding robot and evaluated in a within-participants study (N=17) combining objective algorithmic metrics with subjective self-reports of user experience. Our results show that our reactive mechanism responds near-instantaneously, enables faster corrective actions thereby reducing exposure to unsuitable behaviours, while meaningfully improving user trust, reliance, acceptance and fluency. These findings highlight reactive, well-timed adaptation as a transparent and powerful mechanism for fostering trust and collaboration in Human-Robot Interaction.
Offline Inverse Reinforcement Learning (IRL) learns a reward function from datasets to guide offline policy optimization without environmental rewards. This paradigm is crucial for tasks where online training is expensive and obtaining rewards is difficult. While existing algorithms have progressed in reward maximization, their neglect of safety constraints poses risks in cost-sensitive applications. In this work, we rethink these limitations and formulate the objective of constrained offline IRL for the first time. Accordingly, we propose a Constraint Optimization with Conditional States (COCS) algorithm. We introduce the Lagrange multiplier method for constrained optimization and utilize conditional states to mitigate extrapolation errors. Additionally, we incorporate a dynamics model into the maximum entropy IRL loop to facilitate reward learning. Finally, extensive experiments conducted in both Safety Gym and MetaDrive demonstrate that COCS remains competitive compared to baseline algorithms even when rewards are unavailable, and consistently ensures safety under various safety constraints.
Effective communication is beneficial for cooperative multi-agent reinforcement learning. However, conventional methods often convey local observations or corresponding embeddings among agents, which can be inefficient and detrimental to agent collaboration. To resolve this problem, we propose a multiagent communication method, Consensus-Based Communication for Cooperative Multi-Agent Reinforcement Learning (COCO). COCO contains consensus learning and consensus-based communication. In consensus learning, we employ contrastive learning to make each agent learn a consensus from its local observation for reflecting the global state. In consensus-based communication, we design a Mutual Information regularizer that enables the agents to learn to generate the effective and compact messages based on the consensus, and devise a self-attention network for each agent to effectively aggregate the received messages. Moreover, we construct a Communication Reduction module to prune the specific message bits whilst preserving the important information, so as to adapt COCO to the varying communication overheads. Experimental results in three benchmark multi-agent cooperation environments demonstrate the effectiveness of our proposed method: COCO not only outperforms the related advanced approaches, but also behaves robustly under certain communication constraints.
Neural Architecture Search (NAS) has achieved significant advancements in the automatic design of Convolutional Neural Networks (CNNs), but the high computational resources and time costs associated with NAS limit its practical applications. To mitigate the computational overhead, performance predictors have been introduced to rapidly estimate the performance of new architectures. Existing performance predictors are predominantly tailored for single-task scenarios, rendering them unsuitable for multi-task environments where multiple objectives need to be addressed simultaneously. In response to this limitation, this work presents a novel multi-task NAS framework named MTNP-NAS. Specifically, this framework integrates Evolutionary Multi-Task Optimization (EMTO) into NAS and employs Graph Convolutional Network (GCN)-based multi-task neural predictors to efficiently search for architectures across different tasks. Additionally, we introduce an innovative adaptive knowledge transfer strategy that dynamically adjusts the transfer of knowledge between tasks through encoding transformations and similarity measurements of feature maps, thereby alleviating the potential negative impact of knowledge transfer. In our experiments, we conducted multiple classification tasks across different datasets, including CIFAR10, CIFAR100, and ImageNet16-120, while also performing transfer learning on the complete ImageNet dataset. The experimental results demonstrate that MTNP-NAS outperforms the current state-of-the-art single-task and multi-task NAS methods in terms of accuracy on the CIFAR and ImageNet datasets. Moreover, MTNP-NAS achieves a 40% reduction in search cost compared to its multitask counterparts. These findings highlight the effectiveness of our multi-task approach in leveraging shared information across tasks to enhance both performance and efficiency in neural architecture search.
Spiking Neural Networks (SNNs) have attracted increasing attention due to their brain-inspired computing paradigm and superior energy efficiency on specialized hardware. However, the discrete and non-differentiable nature of spike signals inevitably leads to information loss during signal propagation, which limits the performance of SNNs compared with Artificial Neural Networks (ANNs). To address this issue, we propose ICE-Net, an information-complementary enhancement method for SNNs. Specifically, it employs a customized simplified ICE branch to learn complementary features and delivers them to the corresponding stages of the SNN branch, thereby mitigating information loss during propagation. Extensive experiments on multiple benchmark datasets demonstrate that ICE-Net consistently outperforms existing advanced methods. In particular, ICE-Net achieves 81.62% accuracy on CIFAR-100 with only 4 timesteps, surpassing the previous best result by 2.11%, while reducing energy consumption by approximately 25% compared with ANNs. In addition, we develop several simplified complementary modules that provide flexible trade-offs between accuracy and energy consumption.
Cortical circuits operate with tightly balanced excitatory and inhibitory interactions and attractor-like dynamics, two organising principles that are largely absent from modern deep networks. We introduce Balanced Predictive Attractor Networks (BPANs), lightweight recurrent classifier heads that attach to conventional feedforward backbones. A BPAN head is a discretetime dynamical system with separate excitatory and inhibitory populations, Dale-like sign constraints, and an explicit E/I balance regulariser. It is unrolled for a small number of steps, trained end to end with cross-entropy, and at inference supports anytime prediction by halting once the softmax confidence exceeds a threshold. We present BPAN not as a state-of-the-art architecture but as a compact, interpretable, and biologically inspired testbed in which E/I balance, attractor-like dynamics, and anytime inference can be studied jointly. Evaluations cover four image datasets (MNIST, Fashion-MNIST, CIFAR-10, and SVHN) and three sequential evidence-integration tasks (Cluttered Translated MNIST, Streaming classification, and Sequential CIFAR patches), with MLP, ACT, PonderNet, Multi-Exit, GRU, LSTM, and Transformer baselines. On MNIST, BPAN reaches 97.9% at hidden size H=256 with 152,650 parameters, while ACT, PonderNet, and Multi-Exit require four to five times as many. With a confidence threshold of 0.9, BPAN halts in about 1.4 of T=6 recurrent steps, giving a 4.2× compute reduction with no measurable accuracy loss; similar behaviour is observed on Fashion-MNIST and on SVHN with a shared convolutional backbone. A sensitivity analysis over recurrent depth T, the balance weight λbal, and the E/I ratio shows that all reported behaviours are stable under reasonable hyperparameter variation. The paper also reports honest negative results on sequential tasks, where BPAN’s capacitylimited core trails GRU, LSTM, and Transformer baselines by 12 to 22 percentage points in exchange for a six- to twenty-fold parameter reduction. We also include a theoretical analysis that characterises contractivity, convergence rate, and the dynamical role of Lbal, and we discuss when BPAN’s particular trade-offs are worthwhile.
Emergent communication (EmCom) investigates how artificial agents collaboratively develop languages that exhibit compositional generalization and message alignment between agents. Existing approaches, such as the Inter-VAE+VAE model employing recurrent neural networks (RNNs), face limitations in compositionality and message alignment. Specifically, the sequential structure of RNNs leads to gradual information loss, making it difficult to consistently encode multiple attributes of referents across message sequences and maintain alignment with agents’ internal conceptual representations. To overcome these limitations, we propose Inter-TLM+VAE, a Transformer-based model integrated into the Metropolis-Hastings Naming Game (MHNG), designed to enhance compositional generalization and message alignment between two agents. Each agent consists of two modules: a Variational Autoencoder (VAE), which encodes visual objects into internal conceptual representations, and a Transformer-based Language Model (TLM), which generates messages conditioned directly on these representations. We evaluate our model on the dSprites and 3DShapes datasets, comparing it against baselines including referential games, reconstruction tasks, and the original Inter-VAE+VAE. Results demonstrate that Inter-TLM+VAE improves both compositionality and interagent alignment on a more complex dataset. An ablation study further reveals that self-attention alone substantially outperforms recurrent architectures across all metrics, while integrating both self- and cross-attention achieves the best overall performance. These findings underscore the role of attention mechanisms in advancing robust EmCom systems within the MHNG framework.
In the fields of computer vision and semantic segmentation, enabling computers to recognize target features while filtering out irrelevant information remains an open challenge in semantic segmentation. Drawing inspiration from biological vision research, we find that during the initial conversion and processing of visual information, there exist a retinal centersurround antagonism mechanism and a visual sparse coding mechanism. These two mechanisms help the visual system capture target salient features during the early stage of visual processing. Inspired by this, we construct a novel deep learning model named bio-inspired visual sparse coding network (BVSCDNet). This network integrates the retinal center-surround antagonism mechanism and visual sparse coding mechanism to prioritize target features: the former enhances target-background contrast via mutual inhibition of receptive fields, while the latter selectively activates neurons sensitive to key characteristics to suppress redundant background signals, thereby allowing the network to focus on target features and effectively exclude irrelevant information. To evaluate this segmentation model, we build a unique fiber masterbatch agglomeration database as a benchmark for micro-defect detection, and simultaneously conduct comprehensive experimental validations on multiple design schemes. The model is evaluated on one benchmark dataset and one self-constructed fiber masterbatch agglomeration dataset. Experimental results show that the model exhibits significant effectiveness in visual image segmentation tasks.
Spiking neural networks (SNNs) provide a lowpower, energy-efficient solution by utilizing the spike-based and sparse nature of biological systems. Since the advent of Transformers, SNNs have struggled to compete with artificial networks on long sequential tasks, until the recent emergence of state space models (SSMs), which offer superior computational efficiency and modeling capability. However, despite the initial effectiveness of applying highly capable SSMs to SNNs for long sequence learning, existing SNN-based SSMs ignore the complex dynamics of biological spiking neurons, which are crucial for the functionality but challenging to simulate and exploit effectively in large networks. In addition, the membrane potential is determined by the past spiking history of the neuron, leading to reduced efficiency for sequence modeling in parallel computing scenarios. Therefore, we propose Dyn-SSM, which aims to explore and leverage biologically motivated and more interpretable dynamics in SNN-based SSMs for long sequence learning. Specifically, we propose to incorporate the reset-refractory mechanism in SNN-based SSMs to leverage the inherent temporal dimension for long sequence modeling. Then, to enable parallel processing, a novel parallel max-min boundary compression strategy (PMBC) that can be widely applied is proposed to accelerate the inference process. Finally, we hierarchically integrate the proposed neuron model to the original SSM block, and enhance the dynamics of Dyn-SSM by incorporating trainable thresholds and refractory magnitudes to balance accuracy and sparsity. Extensive experiments illustrate the effectiveness and robustness of Dyn-SSM, showing the potential of dynamic spiking neurons for balancing sparsity and precision in efficient long sequence learning. The code and Supp. Mat. are publicly available in https://github.com/zhyan567/Dyn-SSM.
Intracortical brain-computer interfaces (iBCIs) allow paralyzed individuals to regain motor functions by translating neural activity into control commands for assistive devices. However, neural variability caused by biological factors and electrode connectivity issues disrupts the mapping between neural signals and motor outputs, reducing decoding performance. While supervised recalibration of the neural decoder on subsequent days can mitigate this variability, the process often requires specialized personnel for operating recording systems and acquiring labeled data, which poses practical challenges. To address this issue, we propose Masked U-net-based Cycle-consistent Adversarial Networks (MUCAN), a framework designed to maintain the neuralto- kinematic mapping by aligning daily neural activity distributions with a reference day, thereby eliminating the need for recalibration. MUCAN employs a masking strategy to extract features that remain consistent across days, effectively mitigating daily variability in neural recording conditions. Additionally, it leverages the U-net architecture to extract hierarchical features and preserve fine-grained details of the neural activity, ensuring accurate distribution alignment. Experimental results demonstrate that MUCAN successfully maintains the neural-to-kinematic mapping and outperforms state-of-the-art iBCIs, achieving an average performance improvement of 15.69% across eight publicly available nonhuman primate datasets. MUCAN exhibits robustness against neural signal loss, even with 50 unexpectedly disabled recording channels. By projecting firing rates onto the first two principal components for visualizing latent trajectories, MUCAN effectively preserves the underlying structure of neural population activity over time. This study provides a robust approach to sustaining motor decoding performance without the necessity for continuous recalibration, enhancing the practicality of iBCIs for real-world applications.
Identifying neural correlates of long-term memory retrieval has been an area of considerable research. Previous studies used custom-built, static stimuli to elicit and study memory processes, limiting the insights into brain activity in a natural environment. Instead, we used movie clips to identify neural correlates of memory retrieval. We extracted features from the time-frequency domain from electroencephalography (EEG) signals to classify whether a 10-second-long video clip had been remembered, obtaining greater-than-chance memory decoding (median area under the receiver operating characteristic curve = 0.64) in a naturalistic movie-watching task with greater real-world validity than word-list or image recall paradigms. Feature analysis revealed that remembered clips, compared to not recognised clips, showed increased event-related desynchronisation (ERD) in the theta and low alpha bands early after stimulus onset, followed by stronger ERD in the alpha bands. While beta-band differences were observed at the group level, high within-class variability precluded their use for reliable single-trial classification, with theta and alpha activity emerging as the primary discriminative features. Our ERD/S analysis generalises observations previously demonstrated in the literature using simpler stimuli and more controlled experimental designs. Finally, we observed a significant correlation between classifier performance and retrospective memory ratings of participants (ρ = 0.49) demonstrating that individual differences in memory ability are reflected in neural activity patterns. This work demonstrates the potential of naturalistic stimuli to advance memory research using EEG, revealing neural dynamics that may not be detectable with simpler stimuli, with possible implications for the development of clinical tools in memory assessment.
A critical limitation of best-performing trajectory prediction models is their reliance on modeling spatiotemporal dependencies exclusively within the time domain. This approach renders them vulnerable to performance degradation over long prediction horizons due to error accumulation. To address this bottleneck, we propose Spectral Spatial Graph Convolutional Network (Sp-GCN), a novel framework that pioneers by modeling the dynamics of interacting agents within the graph spectral domain. The core innovation of Sp-GCN lies in its utilization of the Graph Fourier Transform to project complex spatiotemporal interactions from the time domain onto the more stable and holistic spectral representations. Within this spectral framework, we introduce a module that integrates multi-band spectral graph convolution operators (ParaGCN) with temporal gated convolutions to extract deep historical feature representations. A Social Cross-Graph Attention Network (SC-GAT) is also employed to refine these spectral features by attending to each node’s neighbors. This improvement significantly boosts the model’s capacity to capture nuanced social interactions among pedestrians. Extensive evaluations on two prominent benchmarks, ETH/UCY and the Stanford Drone Dataset (SDD), confirms that Sp-GCN delivers competitive performance with improved long-horizon stability. Most notably, the model directly addresses the challenge of long-term error accumulation: on the ETH/UCY dataset, Sp-GCN consistently exhibits a substantially reduced gap between the Final Displacement Error (FDE) and Average Displacement Error (ADE), indicating improved long-horizon stability compared to prior methods. These findings highlight the potential of spectral-temporal modeling for mitigating error amplification in long-horizon trajectory prediction.
Current methods for analyzing dynamic brain connectivity assume global synchrony, overlooking the inherent asynchrony and hierarchical nature of neural processes, thereby leading to incomplete or biased interpretations of brain dynamics. To address these limitations, we propose Spider-Brain, a spike-inspired dynamic graph structural learning (GSL) framework that models the brain as an evolving system with complex temporal organization. Spider-Brain incorporates a spike-inspired selective activation mechanism in which node updates occur through region-specific, event-triggered propagation, effectively capturing heterogeneous temporal responses across brain regions. By combining a multiscale temporal sampling strategy with a selective state-space model (SSM), the framework encodes both short-term neural fluctuations and long-range temporal dependencies, enabling explicit modeling of hierarchical temporal structure. Experiments on two real-world psychiatric datasets show that Spider-Brain consistently outperforms state-of-the-art baselines in disorder classification and provides interpretable activation patterns. The framework also uncovers disorder-specific connectivity and regional activation signatures that align with established clinical and neuroscientific findings. This demonstrates its ability to identify interpretable patterns of brain connectivity and temporal activity. Our code is available at https://github.com/mjliu99/Spider-Brain.
Deep neural networks encounter significant challenges in continuously learning new knowledge over time due to catastrophic forgetting. Although supervised continual learning has made progress, it is usually difficult and costly to provide sufficient labeled datasets for incremental training in open environments. To alleviate this problem, we propose a neuroplasticity-inspired method for semisupervised continual learning (NI-SSCL), which enables deep neural networks to learn from sparsely labeled datasets while preserving prior knowledge. Inspired by the neurogenesis and synaptic metaplasticity mechanism in biological brains, NI-SSCL consists of two core components for keeping memory stability and learning plasticity simultaneously: a neuronal expansion module and a dynamic memory constraint module. The neuronal expansion module dynamically expands the network to acquire new knowledge-specific feature representations, while maintaining feature discrimination. The dynamic memory constraint module optimizes shared network components based on adaptively rating synaptic metaplasticity, i.e., the sensitivity of previously learned knowledge to neuron weight changes, and it enables efficient reuse of shared structures for absorbing new knowledge and alleviating forgetting at the same time. Experimental results on multiple SSCL benchmarks validate the effectiveness of NI-SSCL, which achieves state-of-the-art performance. Additionally, NI-SSCL demonstrates superior feature retention and adaptability, effectively mitigating catastrophic forgetting while leveraging unlabeled data for improved generalization.
Cooperative multiagent reinforcement learning (MARL) is a promising approach for complex collaborative tasks. However, practical deployment remains challenging due to ambiguous credit assignment, inefficient exploration, and the cold-start problem, particularly in systems where the number of agents grows dynamically. Inspired by human cognitive mechanisms for cognitive task decomposition and experience-based knowledge transfer, we propose multiagent reward decomposition and knowledge transfer (MARDKT), a unified method that jointly addresses the challenges of credit assignment, exploration inefficiency, and cold-start in dynamic multiagent settings. We introduce a four-channel reward decomposition mechanism that independently separates reward signals along local/global and extrinsic/intrinsic dimensions: local rewards drive individual exploration, global rewards foster cooperation, and intrinsic curiosity at both individual and team levels promotes discovery of novel states. To enable rapid integration of new agents, we further design a teacher-student framework where students inherit knowledge from trained teachers via policy imitation and value function distillation. We prove that MARDKT ensures monotonic policy improvement from a local perspective and demonstrate its effectiveness in multi-vehicle on-ramp merging and cooperative box-pushing tasks. Furthermore, MARDKT achieves rapid convergence even as the number of agents increases dynamically, showcasing strong scalability in dynamic environments.
