The area of Video Camouflaged Object Detection (VCOD) presents unique challenges in the field of computer vision due to texture similarities between target objects and their surroundings, as well as irregular motion patterns caused by both objects and camera movement. In this paper, we introduce TokenMotion (TMNet), which employs a transformer-based model to enhance VCOD by extracting motion-guided features using a learnable token selection. Evaluated on the challenging MoCA-Mask dataset, TMNet achieves state-of-the-art performance in VCOD. It outperforms the existing state-of-the-art method by a 12.8% improvement in weighted F-measure, an 8.4% enhancement in S-measure, and a 10.7% boost in mean IoU. The results demonstrate the benefits of utilizing motion-guided features via learnable token selection within a transformer-based framework to tackle the intricate task of VCOD. The code of our work will be available when the paper is accepted.

Common image-based LiDAR point cloud semantic segmentation (LiDAR PCSS) approaches have bottlenecks resulting from the boundary-blurring problem of convolution neural networks (CNNs) and quantitation loss of spherical projection. In this work, we propose a transformer-based plug-and-play uncertain point refiner, i.e., TransUPR, to refine selected uncertain points in a learnable manner, which leads to an improved segmentation performance. Uncertain points are sampled from coarse semantic segmentation results of 2D image segmentation where uncertain points are located close to the object boundaries in the 2D range image representation and 3D spherical projection background points. Following that, the geometry and coarse semantic features of uncertain points are aggregated by neighbor points in 3D space without adding expensive computation and memory footprint. Finally, the transformer-based refiner, which contains four stacked self-attention layers, along with an MLP module, is utilized for uncertain point classification on the concatenated features of self-attention layers. As the proposed refiner is independent of 2D CNNs, our TransUPR can be easily integrated into any existing image-based LiDAR PCSS approaches, e.g., CENet. Our TransUPR with the CENet achieves state-of-the-art performance, i.e., 68.2% mean Intersection over Union (mIoU) on the Semantic KITTI benchmark, which provides a performance improvement of 0.6% on the mIoU compared to the original CENet.

Motivated by the increasing application of low-resolution LiDAR, we target the problem of low-resolution LiDAR-camera calibration in this work. The main challenges are two-fold: sparsity and noise in point clouds. To address the problem, we propose to apply depth interpolation to increase the point density and supervised contrastive learning to learn noise-resistant features. The experiments on RELLIS-3D demonstrate that our approach achieves an average mean absolute rotation/translation errors of 0.15cm/0.33° on 32-channel LiDAR point cloud data, which significantly outperforms all reference methods.

Many studies show that using synthetic data or mixed synthetic and real data might improve machine learning (ML) performance but it is difficult to draw generalizable conclusions. A contribution to this problem is the fact that the synthetic data from most vendors are improperly filtered and contain aliased (wrapped-around) high frequency information which they should not possess. Most vendors use spatial-domain, low-pass FIR filters to generate synthetic images at various ranges. Unfortunately, these FIR filters aim for interpolation with a desired frequency domain cutoff and spatial spacing (general non-integer scale factor and/or decimation). Hence, instead of a sharp cutoff at the desired low-band, they produce aliased data. This erroneous information in synthetic imagery could actually mislead an ML algorithm. In addition, most synthetic images do not account for a camera’s MTF (Modulation Transfer Function). A Fourier-based filtering can easily incorporate any MTF in the frequency domain based on Rayleigh resolution theory, properties of a camera lens and digital image properties. The spectral properties of images which are acquired with real sensors are studied and compared them with synthetic images from several vendors. We have also developed a metric that exhibits that the camera system’s MTF shows the same spectral property for real images at different ranges. The metric can help us to determine if a synthetic image generation engine violates this property and, hence, produces erroneous information.

Visual odometry has gained increasing attention due to the proliferation of unmanned aerial vehicles, self-driving cars, and other autonomous robotics systems. Landmark detection and matching are critical for visual localization. While current methods rely upon point-based image features or descriptor mappings we consider landmarks at the object level. In this paper, we propose LMNet a deep learning based landmark matching pipeline for city-scale, aerial images of urban scenes. LMNet consists of a Siamese network, extended with a multi-patch based matching scheme, to handle offcenter landmarks, varying landmark scales, and occlusions of surrounding structures. While there exist a number of landmark recognition benchmark datasets for ground-based and nadir aerial or satellite imagery, there is a lack of datasets and results for oblique aerial imagery. We use a unique unsupervised multi-view landmark image generation pipeline for training and testing the proposed matching pipeline using over 0.5 million real landmark patches. Results for aerial landmark matching across four cities show promising results.

In this paper we evaluate the quality of feature detection and 3D reconstruction on a Wide Area Motion Imagery (WAMI) sequence with increasing JPEG compression ratio. Feature detection is critical for computer vision tasks such as 3D reconstruction. For some 3D reconstruction approaches, the quality of a 3D model relies upon consistent detection of the same feature points over consecutive frames in an image sequence. Since the performance of feature detectors is highly sensitive to compression artifacts, we evaluate the influence of image quality on feature detection accuracy. Many datasets (e.g. WAMI) use JPEG compression to decrease the data storage and network bandwidth utilization while attempting to preserve image quality by adaptively adjusting the compression ratio. Consequently, it is important to understand the impact of JPEG compression on the quality of feature detection in 2D space and the subsequent 3D reconstruction results. We design and perform two evaluation procedures on the WAMI sequence. We use structure tensor to detect feature points on an image sequence with increasing JPEG compression ratio (10:1, 15:1, 20:1, 30:1, 40:1, 100:1, and 150:1). Compression ratio of 10:1 is used as the baseline (groundtruth). First we compare the feature points from images of different qualities with the groundtruth features and evaluate them on pixel level in 2D space. After that, a 3D model in the form of point cloud is generated from each set of feature points and compared with the groundtruth point cloud. We provide quantitative and visualized results for the evaluation.