Accurate localization in GPS-denied environments remains a critical challenge for autonomous robot navigation. Animals exhibit remarkable navigational abilities in complex, dynamic environments by relying on mental cognitive maps. Inspired by neural representations such as head direction cells and grid cells, numerous robotic cognitive mapping systems can efficiently cover large areas; however, they often lack the precise metric information required for accurate localization. To address this challenge, we propose a neurodynamically driven monocular visual topometric localization approach based on road network constraints. We introduce the Roadnetwork-Constraint Hidden Markov Model (RC-HMM) to enhance the semi-metric map by incorporating road network constraints, forming a coherent topometric map that maintains vertex relationships and improves localization accuracy. Experimental results in the CARLA Town07 environment demonstrate the remarkable efficiency of our topometric cognitive map. Compared to the semi-metric map, our approach achieves a 95% reduction in Absolute Pose Error (APE) and an 81% reduction in Relative Pose Error (RPE). Compared to binocular ORB-SLAM3, our monocular approach reduces CPU usage by 96.7% and map storage by 77.7%, with an APE of 3.6 m and RPE of 1.4 m — closely matching ORB-SLAM3's 3.86 m APE and 0.96 m RPE. Furthermore, by leveraging neurodynamics of grid cells and head direction cells, our monocular topometric localization robustly delivers the localization accuracy of 3.86 meters, comparable to binocular ORB-SLAM3. This approach integrates road network metrics into topological maps, enhancing brain-inspired navigation with topometric maps in complex environments.
Autonomous navigation plays a crucial role in cutting-edge scientific and technological domains, such as autonomous driving and space exploration. Current models often rely on knowledge of the discharge patterns of navigation cells in living organisms (e.g. place/grid cells) to encode spatial information, which works well in ideal environments. However, real-world autonomous navigation presents greater challenges due to complex and dynamic geospatial information, leading to issues such as low robustness, poor interpretability, and high energy consumption for existing models. To address these challenges, it is essential to explore the roles and functional connectivity of distinct brain regions involved in processing real-world geospatial information and integrate these insights into autonomous navigation algorithms. This paper reviews empirical studies using neuroscientific techniques to investigate how the human brain processes geographical information during navigation. In particular, we discuss opportunities and challenges associated with three critical aspects: (1) expanding the understanding of cognitive mechanisms from isolated regional functions to integrated functional connectivity and large-scale brain networks, (2) refining neurocognitive experiments to provide ecologically valid evidence in complex and dynamic contexts and (3) developing efficient approaches to computationally mimic and implement spatial cognition mechanisms of human brain in navigation algorithms. Addressing these difficulties would not only enable machines to navigate autonomously and effectively in complex real-world and extreme environments (e.g. space and the deep sea) but also pave the way for the development of future intelligent systems (e.g. GeoAI) with human-like cognitive capabilities.
For autonomous navigation systems, effective path planning in complex environments is critical. The widely used artificial potential field (APF) method, though simple and intuitive, has limitations due to its reliance on an artificially set scaling factor that requires manual tuning for different environments, introducing additional challenges in parameter adjustment. To address these limitations, we propose a novel approach inspired by neuroscience that redefines the attractive and repulsive forces in APF through distributed representations, accompanied by an adaptive mechanism to fine-tune their impact. This method, called the Neuro-Receptive Field Planner (NRF), derives its name from the distributional nature of these forces, which resemble neural receptive fields. Through theoretical analysis and numerical simulations, we validate NRF’s ability to decouple parameters and enhance interpretability, thereby demonstrating its flexibility and effectiveness. In tests conducted across three static and one dynamic environments, NRF exhibited good path smoothness, effective obstacle avoidance, and consistent performance across different scales, achieving the lowest average coefficient of variation (CV = 0.007 0.033) across all metrics compared to baseline methods. This study provides new insights into autonomous navigation and highlights the potential of neuroscience-inspired frameworks to enhance the robustness and adaptability of intelligent systems.
Subsea Christmas trees are often deployed in turbid coastal waters or seabed environments. During manipulator operations on Christmas tree panels, conventional optical servoing is severely limited by rapid electromagnetic attenuation and strong scattering from suspended particles, resulting in reduced visibility. Forward-looking sonar (FLS) provides stable imaging, but its unique imaging geometry and low resolution make direct 6D pose estimation challenging. To address this issue, this paper proposes a 6D object pose estimation method for FLS images, in which conventional optical control-point-based pose estimation is restructured to resolve the mismatch between optical-centric network assumptions and acoustic imaging characteristics, and is further integrated with acoustic projection-based pose inversion. First, to address the limited diversity of target appearances and the scarcity of training data, we construct an FLS imaging model based on primary truncation for image simulation, providing data for model pretraining. Second, a multi-task acoustic control-point detection network, Acoustic-Yolo6D, is designed to mitigate localization degradation caused by heavy speckle noise, low boundary contrast, and resolution variations associated with polar-coordinate imaging, through heatmap regression, auxiliary object segmentation, and explicit range-bearing positional encoding. An Acoustic-n-Point (AnP) model is then used to recover the target 6D pose. Finally, simulation and water-tank experiments on the socket target verify the feasibility and robustness of the proposed method under limited-data conditions. The method achieves a 3.1 cm mean translation error, a 10.88° mean orientation error, and 52 FPS in real underwater acoustic environments.
Audio–visual navigation requires embodied agents to localize and approach sounding targets by jointly exploiting visual observations and auditory cues. Existing methods often suffer from premature cross-modal interference, insufficient object-level alignment, and unstable policy learning in high-dimensional multimodal state spaces. We propose DSS, a dual-stream slot-based state learning framework for audio–visual navigation. DSS first applies modality-specific encoders to preserve the spatial structure of visual observations and the temporal–spectral characteristics of binaural audio. It then employs slot attention to construct compact object-centric multimodal states for target-relevant cross-modal alignment and distractor suppression. On top of this structured state, a discrete Soft Actor-Critic policy is used to improve exploration and training stability. Experiments on Replica and MP3D demonstrate that DSS achieves the best success rate among compared methods, reaching 0.965 and 0.912, respectively, while maintaining competitive path-efficiency performance. Extensive ablation and internal baseline studies further confirm the effectiveness of the proposed framework.
Spatial attention is often partitioned into endogenous, exogenous, and social forms, yet it remains unclear whether a single neural circuit can support all three and how their population codes are organized. Here we trained recurrent artificial neural networks (ANNs) with convolutional sensory front-ends on three classic cueing paradigms (central, peripheral, and gaze cues) to reproduce human reaction time (RT) profiles across cue–target onset asynchronies. Despite differences in sensory architecture and visual experience, all ANNs captured the full facilitation–inhibition time course for all three attention types. Model-based targeted dimensionality reduction (mTDR) revealed that cue- and choice-related activity in the advanced cognitive module evolved as rotations within a shared low-dimensional manifold, with angular deflections that mirrored the distinct temporal dynamics of endogenous, exogenous, and social attention. Attentional signals were encoded by highly sparse, distributed population activity: a small subset of recurrent units explained most task-related variance, was sufficient to recover human-like RT patterns after virtual lesioning, and became progressively sparser as training improved performance. At the same time, single unit responses displayed pervasive mixed selectivity, dominated by nonlinear conjunctions of cue type, cue direction, and validity, whose strength and heterogeneity robustly predicted model performance. Together, these results identified low-dimensional geometric rotations, sparse coding, and nonlinear mixed selectivity as core computational principles through which a single recurrent circuit could generate human-like temporal dynamics across endogenous, exogenous, and social orienting, and provided testable predictions for population-level mechanisms of spatial attention in the brain.
Monocular visual odometry (MVO) provides a low-cost solution for self-localization in GNSS-denied environments, yet most learning-based MVO methods regress rotation and translation as independent Euclidean variables, ignoring their intrinsic coupling on the $SE(3)$ manifold. To address this, we propose UDQN-VO, a UniFormer-based Dual Quaternion Network that formulates pose estimation on the unit dual-quaternion manifold, where rotation and translation are jointly encoded into a shared symmetric matrix derived from a UniFormerV2 backbone. A differentiable Quadratically Constrained Quadratic Program (QCQP) solver analytically recovers the globally optimal rotation in closed form, and a Rotation-Guided Transformer (RGT) decodes the translation conditioned on the recovered orientation, which together preserve geometric consistency throughout the pose decoding pipeline. Extensive experiments on the KITTI show that UDQN-VO attains state-of-the-art accuracy among recent learning-based methods, while evaluations on TUM, ZJH-VO, 4Seasons, and EuRoC further demonstrate its broader applicability across indoor, outdoor, aerial, and cross-season scenarios with real-time inference.
Learning effective representations suitable for decision making in high-level cognitive space is crucial for visual abstract reasoning tasks. The visual system of the mammalian brain is organized into parallel networks that can be roughly classified in dichotomy as the dorsal and ventral streams. How do parallel networks learn efficient representations for cognitive tasks is still an elusive question. We propose the Information Competition Learning Network (ICNet) within a mutual information-constrained framework to learn diversified representations for visual abstract reasoning tasks. ICNet comprises a representation learning module and a rule extractor module. The representation learning module learns two complementary sets of representation under different constraints. These two sets compete to prevent from learning what the other has learned, thereby minimizing mutual predictability. Subsequently, these sets are combined synergistically and relayed to the rule extractor module, where discrete abstract rules are formed to predict the correct option. Empirical experiments consistently show that ICNet achieves superior results across several visual abstract reasoning datasets. Additionally, in Out-of-Distribution relationship reasoning benchmarks, ICNet demonstrates robust generalization ability.
Swarm intelligence algorithms are a class of bionic probabilistic heuristic search methods that are inspired by the collective behaviors of biological agents. In this paper, a multigroup cooperative evolutionary optimization algorithm is proposed by referring to the interaction behaviors of species diversity and stability in the ecosystem. First, the group updating mechanism of the traditional seeking and tracking mode with a dynamic population update mechanism is adopted. The multi-population interactive update group and the quantum entanglement update group are introduced to guide the algorithm to gradually approach the global optimal solution. Second, the proposed bionic algorithm is extended for cross-field applications. The algorithm is applied to solve the function optimization problems, as well as problems in four distinct application fields, including robot routing optimization of grid maps, vehicle scheduling optimization of dairy enterprises, location optimization of logistics centers, and plasma trajectory planning optimization. The proposed multigroup cooperative evolutionary optimization algorithm achieves competitive results in these application fields, thus demonstrating its versatility and robustness.
Social attention, guided by cues like gaze direction, is crucial for effective social interactions. However, how dynamic environmental context modulates this process remains unclear. Integrating a hierarchical Bayesian model with fMRI, this study investigated how individuals adjusted attention based on the predictions about cue validity (CV). Thirty-three participants performed a modified Posner location-cueing task with varying CV. Behaviorally, individuals' allocation of social attention was finely tuned to the precision (inverse variance) of CV predictions, with the predictions updated by precision-weighted prediction errors (PEs) about the occurrence of target locations. Neuroimaging results revealed that the interaction between allocation of social attention and CV influenced activity in regions involved in spatial attention and/or social perception. Precision-weighted PEs about target locations specifically modulated activity in the temporoparietal junction (TPJ), superior temporal sulcus (STS), and primary visual cortex (V1), underscoring their roles in refining attentional predictions. Dynamic causal modeling (DCM) further demonstrated that enhanced absolute precision-weighted PEs about target locations strengthened the effective connectivity from V1 and STS to TPJ, emphasizing their roles in conveying residual error signals upwards to high-level critical attention areas. These findings emphasized the pivotal role of precision in attentional modulation, enhancing our understanding of context-dependent social attention.
Visual Odometry (VO) empowers robots with the ability to perform self-localization within unknown environments using visual cues, yet it is faced with challenges in dynamic environments. In this study, we propose a novel monocular visual odometry network called Spatiotemporal Dual-stream Network (STDN-VO) with two parallel streams, i.e. spatial stream and temporal stream, to model spatiotemporal correlation in the image sequences. Technically, the spatial stream is responsible for extracting global context information from an image, while the temporal stream is designed to effectively extract robust temporal context information from consecutive frames. The outputs of the spatial stream and the temporal stream are merged and then fed to a pose head for predicting the relative pose. Experimental results on the KITTI dataset demonstrate competitive pose estimation performance exceeding published deep learning-based methods. These results underscore the effectiveness of the proposed framework for visual odometry.
Hippocampus in the mammalian brain supports navigation by building a cognitive map of the environment. However, only a few studies have investigated cognitive maps in large-scale arenas. To reveal the computational mechanisms underlying the formation of cognitive maps in large-scale environments, we propose a neural network model of the entorhinal-hippocampal neural circuit that integrates both spatial and non-spatial information. Spatial information is relayed from the grid units in medial entorhinal cortex (MEC) by integrating multimodal sensory-motor signals. Non-spatial, such as object, information is imparted from the visual units in lateral entorhinal cortex (LEC) by encoding visual scenes through a deep neural network. The synaptic weights from the grid units and the visual units to the place units in the hippocampus are learned by a competitive learning rule. We simulated the model in a large box maze. The place units in the model form irregularly-spaced multiple fields across the environment. When the strength of visual inputs is dominant, the responses of place units become conjunctive and egocentric. These results point to the key role of the hippocampus in balancing spatial and non-spatial information relayed via LEC and MEC.
People employ both discrete and continuous quantities to quantify aspects of their environment. However, the temporal dynamics and interactions underlying the processing of these quantitative information remain insufficiently understood. Our study aimed to address this gap by employing a one-back task in conjunction with magnetoencephalography (MEG) to investigate neural responses to dot stimuli representing both discrete (e.g., number of dots) and continuous (e.g., distribution of dots in space) quantities. Our primary finding, derived from representational similarity analysis (RSA), was that processing of field area and numerosity information preceded that of individual information (e.g., individual area and shape), suggesting different timing in the processing of these visual dimensions. Furthermore, within-dimensional temporal generalization analysis revealed distinct temporal patterns of these two different information: numerosity and field area exhibited a combination of chain-like (sequential, non-overlapping processes) and reactivated (initially active, then silent, then reactivated) patterns. Notably, an intermediate 'silent' phase emerged between the initial generalizable representation and subsequent trials, indicating the retrieval of early information to meet subsequent task demands (e.g., the one-back response). In contrast, individual area and shape predominantly followed a chain-like pattern. Furthermore, cross-dimensional temporal generalization analysis showed that numerosity and individual area representations could generalize to each other point-to-point in time, and that early numerosity representations and late individual area representations also generalized to each other, implying both parallel and sequential shared representation of these quantities. Field area showed limited generalization to numerosity and individual area, suggesting that they are processed independently. In summary, our results suggest that quantity processing involves temporally distinct operations with different processing timings and a shared encoding pattern of numerosity and individual area that links these temporally distinct processes.
Brain activities often follow an exponential family of distributions. The exponential distribution is the maximum entropy distribution of continuous random variables in the presence of a mean. The memoryless and peakless properties of an exponential distribution impose difficulties for data analysis methods. To estimate the rate parameter of multivariate exponential distribution from a time series of sensory inputs (i.e., observations), we constructed a hierarchical Bayesian inference model based on a variant of general hierarchical Brownian filter (GHBF). To account for the complex interactions among multivariate exponential random variables, the model estimates the second-order interaction of the rate intensity parameter in logarithmic space. Using variational Bayesian scheme, a family of closed-form and analytical update equations are introduced. These update equations also constitute a complete predictive coding framework. The simulation study shows that our model has the ability to evaluate the time-varying rate parameters and the underlying correlation structure of volatile multivariate exponentially distributed signals. The proposed hierarchical Bayesian inference model is of practical utility in analyzing high-dimensional neural activities.
For focal epilepsy, modeling the virtual brain through large-scale network dynamics to customize treatments is currently a highly promising approach. However, after obtaining the epileptic brain connectome of subjects, most research has focused on exploring ways to help clinicians better perform brain resections. From the perspective of complex networks, we explore the possibility of utilizing the strength of network coupling to treat seizures non-destructively. We use the Epileptor model to construct heterogeneous dynamic networks with epileptogenic zones and design global indices appropriate for this model to describe systemic seizures. Based on these, we explored the effects of epileptogenic proportion and global coupling strength on different artificial networks, and finally verified on a real Allen mouse connectome that the enhancement of coupling strength can effectively control epilepsy. Our simulations found that as the epileptogenic proportion increased, seizure propagation steadily increased for the small-world and the scale-free networks, while both the random network jumped from a sustaining state of global suppression to a state of global bursting. As for the increase in global coupling strength, the small-world network maintained a steady spread, while both the random and scale-free networks had their seizures significantly controlled. Subsequently, we validated the suppression of focal seizures in the Allen mouse brain by boosting the coupling strength a little in its hippocampal formation. Our study shows that the structural nature of networks significantly affects seizure propagation and synchronization. The topology of the random network is significantly anti-epileptic, while others are more prone to maintaining seizures. Coupling strength is an effective way to control epilepsy in both random and scale-free networks. Thus, we propose the idea of using the structural nature of networks to control seizures non-destructively, which may also serve as the theoretical basis for other cognitive training therapies, such as emotional or exercise based interventions aimed at controlling epilepsy by training projection strengths from different brain regions.
AIMS:The interplay between the heart and brain has been a subject of interest for centuries, as dysfunction in this interaction is implicated in various cardiovascular diseases and neurological disorders. Despite this advancement, there is currently a limited understanding of the mechanisms that the human brain communicates with heart rhythms. Here, we aim to characterize the human brain processing of heart rhythms and map human brain topography to heart rhythms. METHODS AND RESULTS:We investigated how the human brain processes heart rhythms in a cohort of 54 drug-resistant epilepsy patients who simultaneously recorded electrocardiography and stereoelectroencephalography (SEEG) during pre-surgical evaluation. Intracranial heartbeat-evoked potentials (HEPs) derived from averaging brain responses time-locked to R peaks of heartbeats in consecutive resting-state SEEG epochs, were characterized in terms of their morphology and spatiotemporal distribution across the brain. The analysis revealed a complex brain topography to heart rhythms that includes the anticipated bilateral thalamus, insula, amygdala, and anterior cingulate cortex, while also extending to the dorsolateral pre-frontal cortex, supramarginal gyrus, and superior temporal gyrus. Employing an Eigen microstates approach, we disentangled two prominent components of the HEPs network in the time window from 100 to 400 ms post-R-peak, reflecting early (100-250 ms) and delayed (250-400 ms) processing pathways. Furthermore, we mapped human brain neurotransmitter receptor signatures onto the HEPs topography, providing the first evidence that serotonin receptor 5HT2a serves as a dominant signature of this organization at the cortical level. Additionally, brain regions exhibiting stronger HEPs showed more pronounced heart rate changes following direct electrical stimulation via SEEG. CONCLUSION:We generated a spatiotemporal dynamic map of HEPs across cortical and subcortical regions. Our characterization of HEPs revealed various dominant components and established a direct association between its topographic organization and distribution of neurotransmitter receptors. This study provides a foundational framework for understanding the brain processing of heart signals and paves the way for novel therapeutic interventions and cardiovascular diseases.
Neural population activity organizes into low-dimensional manifolds embedded within high-dimensional state spaces, yet the principles governing the topology and geometry of these manifolds remain elusive. Here, we propose the Principle of Isomorphism (PIso), which posits that the topology of a neural manifold is constrained by the mathematical structure of the computational task it supports. We apply this framework to the mammalian grid cell system through two distinct theoretical lenses: an intrinsic neural metric, which requires a locally flat Riemannian structure, and path integration, which requires a compact connected Abelian Lie group structure. We show that these two routes are both sufficient conditions that converge on the same toroidal latent topology, and that they naturally unify within Euclidean space. Using a minimal feedforward network that constrains population activity to a torus with tunable geometry, we find that hexagonal grid fields emerge only in an intermediate geometric regime, becoming diffuse or square-like otherwise. Our work clarifies the separation between three notions: latent topology, extrinsic embedding geometry, and decoded physical geometry, and identifies the topology of the population code as the more invariant consequence of the task structure, while leaving the precise mechanism that selects hexagonal single-cell firing patterns as an open problem.
Animals in nature exhibit exceptional navigational abilities, primarily due to the hippocampus’s capacity to form and utilize spatial and non-spatial memories. However, existing models often fail to accurately capture the dynamic interplay between different hippocampal regions. This study presents a unified navigation model inspired by the functional interactions between the hippocampus and surrounding neural circuits, with a focus on the transition mechanisms between vector-based navigation, controlled by grid cells, and hierarchical memory-based navigation, coordinated by the ventral–dorsal hippocampal axis. Simulations show that the model effectively replicates complex path-planning behaviors, such as robust direction selection and efficient shortcut finding, similar to those observed in advanced animals. Furthermore, simulations of hippocampal lesions indicate that ventral lesions increase cognitive load without disrupting planned paths, while dorsal lesions cause additional trajectory oscillations due to impaired spatial memory recall. These findings provide new insights into hippocampal navigation strategies and suggest potential applications for studying memory, learning, and cognitive function across various contexts.
Misha Tsodyks合作论文数Department of Neurobiology
Weizmann Institute of Science5