In frame-based vision, object detection faces substantial performance degradation under challenging conditions due to the limited sensing capability of conventional cameras. Event cameras output sparse and asynchronous events, providing a potential solution to solve these problems. However, effectively fusing two heterogeneous modalities remains an open issue. In this work, we propose a novel hierarchical feature refinement network for event-frame fusion. The core concept is the design of the coarse-to-fine fusion module, denoted as the cross-modality adaptive feature refinement (CAFR) module. In the initial phase, the bidirectional cross-modality interaction (BCI) part facilitates information bridging from two distinct sources. Subsequently, the features are further refined by aligning the channel-level mean and variance in the two-fold adaptive feature refinement (TAFR) part. We conducted extensive experiments on two benchmarks: the low-resolution PKU-DDD17-Car dataset and the high-resolution DSEC dataset. Experimental results show that our method surpasses the state-of-the-art by an impressive margin of 8.0% on the DSEC dataset. Besides, our method exhibits significantly better robustness (69.5% versus 38.7%) when introducing 15 different corruption types to the frame images. The code can be found at the link (https://github.com/HuCaoFighting/FRN).

Estimating the rigid transformation between two LiDAR scans through putative 3D correspondences is a typical point cloud registration paradigm. Current 3D feature matching approaches commonly lead to numerous outlier correspondences, making outlier-robust registration techniques indispensable. Many recent studies have adopted the branch and bound (BnB) optimization framework to solve the correspondence-based point cloud registration problem globally and deterministically. Nonetheless, BnB-based methods are time-consuming to search the entire 6-dimensional parameter space, since their computational complexity is exponential to the solution domain dimension in the worst-case. To enhance algorithm efficiency, existing works attempt to decouple the 6 degrees of freedom (DOF) original problem into two 3-DOF sub-problems, thereby reducing the search space. In contrast, our approach introduces a novel pose decoupling strategy based on residual projections, decomposing the raw registration problem into three sub-problems. Subsequently, we embed interval stabbing into BnB to solve these sub-problems within a lower two-dimensional domain, resulting in efficient and deterministic registration. Moreover, our method can be adapted to address the challenging problem of simultaneous pose and registration. Through comprehensive experiments conducted on challenging synthetic and real-world datasets, we demonstrate that the proposed method outperforms state-of-the-art methods in terms of efficiency while maintaining comparable robustness.

Point cloud registration is challenging in the presence of heavy outlier correspondences. This paper focuses on addressing the robust correspondence-based registration problem with gravity prior that often arises in practice. The gravity directions are typically obtained by inertial measurement units (IMUs) and can reduce the degree of freedom (DOF) of rotation from 3 to 1. We propose a novel transformation decoupling strategy by leveraging the screw theory. This strategy decomposes the original 4-DOF problem into three sub-problems with 1-DOF, 2-DOF, and 1-DOF, respectively, enhancing computation efficiency. Specifically, the first 1-DOF represents the translation along the rotation axis, and we propose an interval stabbing-based method to solve it. The second 2-DOF represents the pole which is an auxiliary variable in screw theory, and we utilize a branch-and-bound method to solve it. The last 1-DOF represents the rotation angle, and we propose a global voting method for its estimation. The proposed method solves three consensus maximization sub-problems sequentially, leading to efficient and deterministic registration. In particular, it can even handle the correspondence-free registration problem due to its significant robustness. Extensive experiments on both synthetic and real-world datasets demonstrate that our method is more efficient and robust than state-of-the-art methods, even when dealing with outlier rates exceeding 99%.

Radar is among the most popular sensors in modern Intelligent Transportation Systems (ITSs), enabling weather-robust perception. The orientation and position of the traffic radar relative to the ITS coordinate system are necessary for the perception fusion in ITSs. However, due to the unknown target association, sparseness and noisiness of traffic radar measurements, the robust and accurate extrinsic calibration of traffic radar is challenging. In this paper, we propose a targetless traffic radar calibration method based on GPS to overcome the inconvenience during ITS operation, because the installation of a dedicated calibration target on the highway is impractical and dangerous. On the other hand, the high-precision GPS device installed on the moving vehicle can provide traffic radar with accurate positioning information of the detection target. Furthermore, during the optimization process of extrinsic calibration, we propose a globally optimal registration method, which is robust to noise and outliers in radar measurements, and is called Gaussian Mixture Robust Branch and Bound (GMRBnB). Specifically, we first construct the robust objective function by utilizing the Gaussian Mixture Model (GMM). Then, we derive novel relaxation bounds and present the GMRBnB algorithm that overcomes the susceptibility to local minima and the dependence on initialization of traditional optimization methods. Compared with existing methods, extensive experiments in synthetic and real-world data demonstrate that our method is not only globally optimal, but also more accurate and robust.

Point set registration is the technology used to estimate the spatial transformation between two LiDAR scans, which is challenging in the presence of outlier correspondences and noise. Our focus is on 4 degrees of freedom (DOF) point set registration, in which 1DOF rotation and 3DOF translation need to be estimated. It is commonly found in practical scenarios, such as arbitrarily mobile robots equipped with an inertial measurement unit (IMU), terrestrial LiDAR scanners, or planar moving vehicles in urban environments. Recently, many solutions have leveraged branch and bound (BnB) in global and deterministic approaches to solve the registration problem with performance guarantees. However, BnB-based methods are usually time-consuming since their convergence speed is exponential to the dimensionality of the solution domain, and existing methods estimate these 4DOF simultaneously. Our key idea is to speed up BnB-based methods by decoupling the joint pose into separate translation and rotation with the aid of known gravity directions. This effectively reduces the search domain to 3DOF+1DOF, thereby enhancing algorithm efficiency. Specifically, we propose a novel BnB-based consensus maximization method for a fast 3DOF translation search and derive the specific lower and upper bound functions. We then propose an efficient global voting method for estimating the rotation with 1DOF. To demonstrate the superiority of our proposed method, we conduct extensive experiments on both synthetic and real-world datasets. The experimental results show that (1) our proposed method is more robust against outliers and noise than several existing methods and far faster than the existing BnB-based 4DOF method by almost an order of magnitude, (2) our proposed method is robust against the biases in gravity directions, such that the general error of the IMU is acceptable, and (3) thanks to its significant robustness, our proposed method can solve the challenging problem of simultaneous pose and correspondence registration (SPCR). Moreover, the proposed approach is also more robust and accurate than several SPCR benchmark methods. Code is available at https://github.com/Xinyi-tum/Fast-and-Deterministic-Registration.

Vision Transformers (ViTs) have recently achieved promising results in various computer vision tasks. However, ViTs have high computation costs and a large number of parameters due to the stacked multi-head self-attention (MHSA) and expanded feed-forward network (FFN) modules. Since the complexity of Transformer-based models is quadratic with the length of the input tokens, most current efforts focus on reducing the number of tokens in ViTs to improve the model efficiency. Unlike previous studies, we argue that diverse redundant features help ViTs understand the data comprehensively. In this paper, we propose GhostViT, which achieves both computation and storage efficiency. The key concept of GhostViT is to generate more diverse features using cheap operations in the MHSA and FFN modules. We experimentally demonstrate that our GhostViT can significantly reduce both the parameters and FLOPs of ViTs while achieving the similar or better accuracy. For example, about 14% of parameters and 17% of FLOPs of the DeiT-tiny model are reduced without any accuracy loss on the ImageNet-1 K dataset. The codes and trained models can be found at https://github.com/HuCaoFighting/GhostViT .

Localization in the environment is an essential navigational capability for animals and indoor robotic vehicles. In indoor environments, it is still challenging to perfectly solve the global localization problem using probabilistic methods. However, animals are able to instinctively localize themselves with much less effort. Therefore, an intriguing and promising approach is to seek biological inspiration from animals. In this paper, we present a biologically-inspired global localization system using a LiDAR sensor that utilizes a hippocampal model and a landmark-based relocalization approach. The experiment results show that the proposed method is competitive with Monte Carlo Localization, and the results demonstrate the high accuracy, applicability, and reliability of the proposed biologically-inspired localization system in various localization scenarios.