LiDAR place recognition is a critical component of LiDAR-based localization pipelines, tasked with identifying previously visited places across diverse environments and temporal conditions. A growing body of deep learning–based approaches has recently tackled this problem. However, their performance often degrades when the models are deployed in unseen environments. Although offline fine-tuning can partly recover performance, it is prone to catastrophic forgetting of previously acquired knowledge and cannot respond quickly enough to rapidly changing data distributions. In this paper, we introduce OCLPlace, an online continual learning framework that learns directly from highly temporally correlated LiDAR streams and strikes a trade-off between rapid domain adaptation and resistance to catastrophic forgetting. To the best of our knowledge, OCLPlace is the first LiDAR place-recognition approach enhanced by online continual learning that can automatically adapt to new environments while mitigating catastrophic forgetting. Experimental results on six large-scale datasets, which cover both ground-view and aerial-view scenarios, demonstrate the effectiveness and robustness of our method. The source code will be publicly available at: https://github.com/npu-ius-lab/OCLPlace.

LiDAR-based global localization provides accurate robot pose estimates against a prior map. Existing deep-learning methods, however, demand heavy computation and long training or inference times and degrade sharply when faced with domain shifts. This letter presents LighterBEV, a lightweight, fast, and generalizable localization method. An Informative Compression Module achieves a fourfold reduction in local-feature dimensionality while improving accuracy. We further integrate online learning to enable rapid post-deployment adaptation, mitigating degradation under distribution shift. Extensive experiments on four large-scale datasets show that LighterBEV achieves state-of-the-art performance with limited training data, maintains high accuracy under domain shift, and runs in real time on resource-constrained hardware-supporting both inference and online updates. To our knowledge, LighterBEV is the first LiDAR global localization approach to incorporate online learning for automatic adaptation to new environments, thereby narrowing the domain gap.

This paper presents NavWareSet , a novel dataset crafted to advance socially compliant robot navigation research. NavWareSet provides multi-modal recordings of both socially compliant and non-compliant robot trajectories in controlled indoor environments. Drawing upon seven carefully selected scenarios, it captures complex human-robot interactions and a range of navigation challenges that mirror realistic social contexts. NavWareSet establishes a rich dataset for evaluating and training navigation algorithms by incorporating two distinct robot platforms—Toyota Human Support Robot (HSR) and Clearpath’s Jackal—and systematically varying their navigation behaviors. With data modalities spanning lidar, RGB-D camera, odometry, and human position annotations, NavWareSet enables fine-grained analysis of the robot’s decision-making process and its impact on human comfort and safety. Ultimately, this dataset provides a versatile resource for developing robust, ethically guided navigation policies and for measuring their performance across a range of social situations. More information can be seen at: https://anr-navware.github.io/navwareset/ .

Robot social navigation needs to adapt to different human factors and environmental contexts. However, since these factors and contexts are difficult to predict and cannot be exhaustively enumerated, traditional learning-based methods have difficulty in ensuring the social attributes of robots in long-term and cross-environment deployments. This letter introduces an online context learning method that aims to empower robots to adapt to new social environments online. The proposed method adopts a two-layer structure. The bottom layer is built using a deep reinforcement learning-based method to ensure the output of basic robot navigation commands. The upper layer is implemented using an online robot learning-based method to socialize the control commands suggested by the bottom layer. Experiments using a community-wide simulator show that our method outperforms the state-of-the-art ones. Experimental results in the most challenging scenarios show that our method improves the performance of the state-of-the-art by 8%.

Deep learning has become increasingly crucial in simultaneous localization and mapping (SLAM). Supervised deep learning SLAM methods need ground truth data for training but achieve highly accurate predictions. Such training techniques are used by some visual or LiDAR SLAM methods. However, these methods mostly rely on single-sensor data, missing the combined advantages of LiDAR and visual information. Multisensor fusion SLAM merges diverse data types, enriching features and boosting performance. This paper presents a new deep visual-LiDAR SLAM method that integrates both visual and LiDAR data. The SLAM system includes a deep visual-LiDAR odometry module, a deep learning-based loop closure detection module, and a 3D mapping module. The deep visual-LiDAR odometry module uses a hierarchical feature encoding module to capture features from two frames of color point clouds at various levels. An attention feature decoding module employs an attention mechanism to merge these features, determining their relative pose. The iterative pose optimization module continuously refines the pose accuracy, improving robustness to outliers. The loop closure detection module uses a deep learning-based global descriptor for precise positional matching. The 3D mapping module constructs the environmental map. Experimental results on the KITTI odometry dataset show that our method outperforms other supervised and traditional geometric methods, with lower rotation and translation errors. By integrating visual and LiDAR data, our SLAM system achieves higher accuracy and robustness in pose estimation compared to single-modal SLAM systems.

LiDAR Place Recognition (LPR) aims to identify previously visited places across different environments and times. Thanks to the recent advances in Deep Neural Networks (DNNs), LPR has experienced rapid development. However, DNN-based LPR methods may suffer from Catastrophic Forgetting (CF), where they tend to forget previously learned domains and focus more on adapting to a new domain. In this paper, we propose Mutual Information-guided Continual Learning (MICL) to tackle this problem in LPR. We design a domain-sharing loss function Mutual Information Loss (MIL) to encourage existing DNN-based LPR methods to learn and preserve knowledge that may not be useful for the current domain but potentially beneficial for other domains. MIL overcomes CF from an information-theoretic perspective including two aspects:1) maximizing the preservation of information from input data in descriptors, and 2) maximizing the preservation of information in descriptors when training across different domains. Additionally, we design a simple yet effective memory sampling strategy to further alleviate CF in LPR. Furthermore, we adopt adaptive loss weighting, which reduces the need for hyperparameters and enables models to make optimal trade-offs automatically. We conducted experiments on three large-scale LiDAR datasets including Oxford, MulRan, and PNV. The experimental results demonstrate that our MICL outperforms state-of-the-art continual learning approaches. The code of MICL is publicly available at: https://github.com/npu-ius-lab/MICL .

For autonomous driving, the performance of 3D object detection is limited by offline training, and these methods usually lack the adaption ability for long-term autonomy, which leads to significant performance degeneration across different scenarios, i.e. domain shift. This paper proposes a few-shot online learning method to transfer knowledge from 2D images to 3D point clouds. In particular, the point cloud clusters are automatically labeled by the 3D-2D projection and 3D object tracking, and the learning strategy allows the classifier to learn multiple classes with limited samples in a short period of time. The final 3D detection results are obtained from the fusion of the online learning 3D detector and an end-to-end 3D detector. Experimental results on the KITTI dataset demonstrate the effectiveness of our system compared to the baseline methods.

Perception of human beings is one of the basic capabilities of service robots and is the prerequisite for interaction between robots and humans. Although enabling hardware and software technologies have made great strides, there are not many open-source solutions that organically integrate the two. To address this shortfall, this paper introduces an open-source scheme of hardware and software integration for robotic embodied human perception. The embodied entity includes a robot chassis, a computing unit based on ARM architecture, a 3D lidar, a 2D lidar and four RGB-D cameras for robot exterior perception, a display panel for human-robot interaction, a set of LED lights to show the robot’s status and a sonar strip for low-level obstacle avoidance. The perception software is fully based on the Robot Operating System (ROS) which allows high modularity, fully deployed to the embodied entity and running at a rate of 30 Hz. The entire integration solution is very portable and publicly available at https://github.com/ Nedzhaken/human_aware_navigation.

Autonomous vehicles require online learning capabilities to enable long-term, unattended operation. However, long-term online learning is accompanied by the problem of forgetting previously learned knowledge. This paper introduces an online learning framework that includes a catastrophic forgetting prevention mechanism, named Long-Short-Term Online Learning (LSTOL). The framework consists of a set of shortterm learners and a long-term controller, where the former is based on the concept of ensemble learning and aims to achieve rapid learning iterations, while the latter contains a simple yet efficient probabilistic decision-making mechanism combined with four control primitives to achieve effective knowledge maintenance. A novel feature of the proposed LSTOL is that it avoids forgetting while learning autonomously. In addition, LSTOL makes no assumptions about the model type of short-term learners and the continuity of the data. The effectiveness of the proposed framework is demonstrated through experiments across well-known datasets in autonomous driving, including KITTI and Waymo. The source code for the method implementation is publicly available at https://github.com/epan-utbm/lstol.

In the past ten years, the use of 3D Time-of-Flight (ToF) LiDARs in mobile robotics has grown rapidly. Based on our accumulation of relevant research, this article systematically reviews and analyzes the use 3D ToF LiDARs in research and industrial applications. The former includes object detection, robot localization, long-term autonomy, LiDAR data processing under adverse weather conditions, and sensor fusion. The latter encompasses service robots, assisted and autonomous driving, and recent applications performed in response to public health crises. We hope that our efforts can effectively provide readers with relevant references and promote the deployment of existing mature technologies in real-world systems.

Light detection and ranging (LiDAR) limitations in adverse weather (e.g., rain, fog, and snow) prevent adopting high-level autonomous vehicles in all weather conditions. Furthermore, collecting and annotating these sparse point clouds in adverse weather is often cumbersome, inefficient, and expensive. In this article, we propose a data-driven approach to statistically model the performance of a popular near-infrared (NIR) time-of-flight (ToF) LiDAR in fog, with noisy point clouds collected in a well-controlled artificial fog chamber. Given manually defined visibility describing the levels of fog, our models can directly forecast a probability distribution of a laser's noisy range measurement. Moreover, the real road data collected in clear weather is utilized to produce auto-labeled noisy point clouds using a LiDAR performance simulator, which is then used to train a semantic segmentation network to recognize point cloud noise in the real world in adverse weather. Qualitative and quantitative experimental results verify the applicability of our LiDAR performance models in fog and show how our Sim2Real strategy of the denoising algorithm can be applied to noisy point clouds under various weather conditions. The developed robot operating system (ROS) packages are publicly available at: https://github.com/cavayangtao/lanoise_pp.

The automotive industry plays a critical role in the global economy, and particularly important is the expanding Chinese automobile market due to its immense scale and influence. However, existing automotive sector datasets are limited in their coverage, failing to adequately consider the growing demand for more and diverse variables. This paper aims to bridge this data gap by introducing a comprehensive dataset spanning the years from 2016 to 2022, encompassing sales data, online reviews, and a wealth of information related to the Chinese automotive industry. This dataset serves as a valuable resource, significantly expanding the available data. Its impact extends to various dimensions, including improving forecasting accuracy, expanding the scope of business applications, informing policy development and regulation, and advancing academic research within the automotive sector. To illustrate the dataset’s potential applications in both business and academic contexts, we present two application examples. Our developed dataset enhances our understanding of the Chinese automotive market and offers a valuable tool for researchers, policymakers, and industry stakeholders worldwide.

The spatial information provided by 3-D object detection is critical for autonomous driving. Although visual-based object detection has made significant progress in 2-D/3-D in recent years, nonvisual sensors such as 3-D light detection and ranging (LiDAR) still have inherent advantages in the accuracy (ACC) of object localization. However, the challenge still lies in the interpretability of the sparse point clouds it generates and the difficulty of being manually annotated. In this article, we propose an online transfer learning framework based on multimodal sensor systems for 3-D object detection of urban road participants including pedestrians, cyclists, and cars. The framework aims to automatically and efficiently transfer object detection capabilities from 2-D monocular camera to 3-D LiDAR through a multitarget tracker-based pipeline, enabling knowledge transfer between sensors of different modalities. Furthermore, online random forest (ORF), an inherently fast and multiclass learning method, is innovatively integrated into our system. Experiments on two very different datasets, including KITTI and Waymo, demonstrate that the proposed framework not only rapidly builds 3-D detection capabilities about road participants in a single dataset but also maintains this capability across datasets. This reveals that the proposed framework is particularly suitable for in situ deployment of unmanned vehicles and can also solve the problem of insufficient generalization ability of current offline-trained detectors.

Social compatibility is one of the most important parameters for service robots. It characterizes the quality of interaction between a robot and a human. In this paper, a human-centered benchmarking framework is proposed for socially-compliant robot navigation. In an end-to-end manner, four open-source robot navigation methods are benchmarked, two of which are socially-compliant. All aspects of the benchmarking are clarified to ensure the reproducibility and replicability of the experiments. The social compatibility of robot navigation methods with the Robotic Social Attributes Scale (RoSAS) is measured. After that, the correspondence between RoSAS and the robot-centered metrics is validated. Based on experiments, the extra robot time ratio and the extra distance ratio are the most suitable to judge social compatibility.

Exterior facelift has become an effective method for automakers to boost the consumers’ interest in an existing car model before it is redesigned. To support the automotive facelift design process, this study develops a novel computational framework – Generator, Evaluator, Optimiser (GEO) , which comprises three components: a StyleGAN2-based design generator that creates different facelift designs; a convolutional neural network (CNN) -based evaluator that assesses designs from the aesthetics perspective; and a recurrent neural network (RNN) -based decision optimiser that selects designs to maximise the predicted profit of the targeted car model over time. We validate the GEO framework in experiments with real-world datasets and describe some resulting managerial implications for automotive facelift. Our study makes both methodological and application contributions. First, the generator’s mapping network and projection methods are carefully tailored to facelift where only minor changes are performed without affecting the family signature of the automobile brands. Second, two evaluation metrics are proposed to assess the generated designs. Third, profit maximisation is taken into account in the design selection. From a high-level perspective, our study contributes to the recent use of machine learning and data mining in marketing and design studies. To the best of our knowledge, this is the first study that uses deep generative models for automotive regional design upgrading and that provides an end-to-end decision-support solution for automakers and designers.

The performance of deep learning-based detection methods has made them an attractive option for robotic perception. However, their training typically requires large volumes of data containing all the various situations the robots may potentially encounter during their routine operation. Thus, the workforce required for data collection and annotation is a significant bottleneck when deploying robots in the real world. This applies especially to outdoor deployments, where robots have to face various adverse weather conditions. We present a method that allows an independent car tansporter to train its neural networks for vehicle detection without human supervision or annotation. We provide the robot with a hand-coded algorithm for detecting cars in LiDAR scans in favourable weather conditions and complement this algorithm with a tracking method and a weather simulator. As the robot traverses its environment, it can collect data samples, which can be subsequently processed into training samples for the neural networks. As the tracking method is applied offline, it can exploit the detections made both before the currently processed scan and any subsequent future detections of the current scene, meaning the quality of annotations is in excess of those of the raw detections. Along with the acquisition of the labels, the weather simulator is able to alter the raw sensory data, which are then fed into the neural network together with the labels. We show how this pipeline, being run in an offline fashion, can exploit off-the-shelf weather simulation for the auto-labelling training scheme in a simulator-in-the-loop manner. We show how such a framework produces an effective detector and how the weather simulator-in-the-loop is beneficial for the robustness of the detector. Thus, our automatic data annotation pipeline significantly reduces not only the data annotation but also the data collection effort. This allows the integration of deep learning algorithms into existing robotic systems without the need for tedious data annotation and collection in all possible situations. Moreover, the method provides annotated datasets that can be used to develop other methods. To promote the reproducibility of our research, we provide our datasets, codes and models online.

As a critical sensor for high-level autonomous vehicles, LiDAR’s limitations in adverse weather (e.g. rain, fog, snow, etc.) impede the deployment of self-driving cars in all weather conditions. However, studies in literature on LiDAR’s performance in harsh conditions are insufficient. In this paper, based on a dataset collected with a popular Near-InfraRed (NIR) ToF LiDAR in a well-controlled artificial fog chamber, we statistically model the LiDAR ranging process in fog conditions through a data-driven approach. Specifically, giving an object at a known distance, our model is able to predict LiDAR measures (range and intensity) under various fog conditions. For a transmitted laser under fog, we first model and predict the minimum visibility required to detect its true range or not. Then, the noisy range and intensity measures are sampled from the probabilistic measurement distributions inferred from the dataset. The performance of the proposed method has been quantitatively and qualitatively evaluated. Experimental results show that our approach can provide a promising performance prediction of the utilized NIR ToF LiDAR under fog, which opens a new gate to the quantitative assessment of adverse weather and contributes to the specification of relevant Operational Domain Designs (ODDs). The developed ROS package is available at: https://github.com/cavayangtao/lanoising .