The problem of localization on a large-scale satellite image given a frame of query ground view point clouds remains challenging. Existing LiDAR-to-image cross-view localization methods struggle in large-scale scenarios due to limited semantic alignment and the modality gap between point clouds and satellite images. This paper introduces the large-scale LiDAR-to-image geo-localization pipeline called GeoISF. GeoISF introduces an instance semantic forest constructed using WordNet, which enhances temporal semantic representation and discriminative power by integrating semantic trees from multiple frames. By leveraging environmental semantic representation as a shared medium, GeoISF effectively bridges the modality gap and improves semantic matching accuracy. Extensive experiments demonstrate the superior performance of GeoISF in large-scale cross-view localization, which achieves a 13.22-fold improvement compared to the parallel LiDAR-to-image method in the R@10 metric on KITTI dataset. The proposed method addresses the existing gap in large-scale LiDAR-to-image cross-view localization, offering a robust solution to the computational and accuracy challenges inherent in such scenarios. We will release the code as an open-source resource available online for the broader research community.

Open-Vocabulary Aerial Detection (OVAD) and Remote Sensing Visual Grounding (RSVG) have emerged as two key paradigms for aerial scene understanding. However, each paradigm suffers from inherent limitations when operating in isolation: OVAD is restricted to coarse category-level semantics, while RSVG is structurally limited to single-target localization. These limitations prevent existing methods from simultaneously supporting rich semantic understanding and multi-target detection. To address this, we propose OTA-Det, the first unified framework that bridges both paradigms into a cohesive architecture. Specifically, we introduce a task reformulation strategy that unifies task objectives and supervision mechanisms, enabling joint training across datasets from both paradigms with dense supervision signals. Furthermore, we propose a dense semantic alignment strategy that establishes explicit correspondence at multiple granularities, from holistic expressions to individual attributes, enabling fine-grained semantic understanding. To ensure real-time efficiency, OTA-Det builds upon the RT-DETR architecture, extending it from closed-set detection to open-text detection by introducing several high efficient modules, achieving state-of-the-art performance on six benchmarks spanning both OVAD and RSVG tasks while maintaining real-time inference at 34 FPS.

Highlights What are the main findings? A novel geometry-constrained controllable generation model is proposed to synthesize remote sensing images from ground-level images and corresponding point clouds. A dual remote sensing feature fusion module that leverages the complementary characteristics of image and point cloud data is designed to guide the diffusion model for generating realistic remote sensing imagery. What are the implications of the main finding? This approach significantly enhances the fidelity and realism of synthesized remote sensing images while effectively reducing spatial structural randomness. It establishes a robust and efficient solution for cross-modal and cross-view image generation, offering significant value for observation in inaccessible areas like UAV no-fly zones and underground regions.Highlights What are the main findings? A novel geometry-constrained controllable generation model is proposed to synthesize remote sensing images from ground-level images and corresponding point clouds. A dual remote sensing feature fusion module that leverages the complementary characteristics of image and point cloud data is designed to guide the diffusion model for generating realistic remote sensing imagery. What are the implications of the main finding? This approach significantly enhances the fidelity and realism of synthesized remote sensing images while effectively reducing spatial structural randomness. It establishes a robust and efficient solution for cross-modal and cross-view image generation, offering significant value for observation in inaccessible areas like UAV no-fly zones and underground regions.Abstract Remote sensing image analysis is crucial for many research fields, yet acquiring frequent high-quality remote sensing imagery is not always feasible due to prohibitive costs and logistical efforts. As a solution, ground-to-satellite cross-view image generation has emerged as a promising approach for synthesizing remote sensing images from readily available ground sensor data. However, existing methods face two critical limitations that bottleneck their performance, including instability in object structural attributes in ground views and reduced image fidelity and consistency due to environmental occlusions. To address these challenges, this paper proposes a geometrically constrained controllable generation model specifically tailored for remote sensing image generation, called GCCG-RSI. To overcome the limitation of structural instability, GCCG-RSI introduces LiDAR ranging accuracy to constrain the geometric shapes of the generated image. To mitigate occlusion-induced fidelity issues, GCCG-RSI employs an attention mechanism to derive a unified fused representation that integrates texture and spatial structure information. The representation is utilized as a conditional control signal to guide the diffusion model in accurately synthesizing remote sensing imagery. Experimental results demonstrate that, compared with state-of-the-art methods, GCCG-RSI infers remote sensing images with superior realism and fidelity using ground-view images and point clouds with limited perspective. Overall, the proposed method provides an effective image preprocessing approach that contributes to significantly narrowing the domain discrepancy between ground and satellite images, thereby facilitating the execution of downstream tasks.

Mamba has made its debut in several visual tasks, but a more effective scanning strategy is still needed to unfold visual data into logical one-dimensional token sequences, ensuring spatial continuity and semantic structures. In this paper, an objectness scanning strategy with a dual-attention mechanism is proposed for efficient vision Mamba. Visual data is unfolded according to its objectness. The produced one-dimensional token sequences maintain visual spatial continuity while exhibiting a clear semantic structure, akin to the syntactic structures of natural language, owing to three built-in linguistic characteristics—the dependency distance minimization, the primacy effect, and high accessibility. A multi-scale foreground proposer is also proposed as an essential component to evaluate the objectness of tokens. It mitigates interference from non-object information and prevents the attenuation of semantic information by a parallel enhancement modulation mechanism. By incorporating the objectness scanning strategy, the proposed vision Mamba model, ObjM, demonstrates significant superiority in accurately and efficiently identifying foreground objects. On both camouflaged object detection and salient object detection, it delivers improved performance while achieving an average reduction of 31.04 % in computational cost and 56.90 % in parameter count by merely replacing the backbones of several state-of-the-art camouflaged object detection models. On the MSCOCO multi-object detection and instance segmentation task, it also consumes fewer computational resources. Codes are available at https://github.com/kaiopen/objm.

The joint classification of hyperspectral imaging (HSI) and Light Detection and Ranging (LiDAR) data holds significant importance for various practical uses, including urban mapping, mineral prospecting, and ecological observation. Achieving robust and transferable feature representations is essential to fully leverage the complementary properties of HSI and LiDAR modalities. However, existing methods are often constrained to scene-specific training and lack generalizability across datasets, limiting their discriminative power. To tackle this challenge, we introduce a new dual-phase approach for the combined classification of HSI and LiDAR data. Initially, a transformer-driven network is trained on various HSI-only datasets to extract universal spatial–spectral features. In the second stage, LiDAR data is incorporated as a task-specific prompt to adapt the model to HSI-LiDAR scenes and enable effective multimodal fusion. Through extensive testing on three benchmark datasets, our framework proves highly effective, outperforming all competing approaches.

In recent years, language-guided open-set aerial object detection has gained significant attention due to its better alignment with real-world application needs. However, due to limited datasets, most existing language-guided methods primarily focus on vocabulary-level descriptions, which fail to meet the demands of fine-grained open-world detection. To address this limitation, we propose constructing a large-scale language-guided open-set aerial detection dataset, encompassing three levels of language guidance: from words to phrases, and ultimately to sentences. Centered around an open-source large vision-language model and integrating image-operation-based preprocessing with BERT-based postprocessing, we present the OS-W2S Label Engine, an automatic annotation pipeline capable of handling diverse scene annotations for aerial images. Using this label engine, we expand existing aerial detection datasets with rich textual annotations and construct a novel benchmark dataset, called MI-OAD, addressing the limitations of current remote sensing grounding data and enabling effective language-guided open-set aerial detection. Specifically, MI-OAD contains 163,023 images and 2 million image-caption pairs, approximately 40 times larger than comparable datasets. To demonstrate the effectiveness and quality of MI-OAD, we evaluate three representative tasks. On language-guided open-set aerial detection, training on MI-OAD lifts Grounding DINO by +31.1 AP_50 and +34.7 Recall@10 with sentence-level inputs under zero-shot transfer. Moreover, using MI-OAD for pre-training yields state-of-the-art performance on multiple existing open-vocabulary aerial detection and remote sensing visual grounding benchmarks, validating both the effectiveness of the dataset and the high quality of its OS-W2S annotations. More details are available at https://github.com/GT-Wei/MI-OAD.

Camouflaged object detection (COD) is a challenging task aimed at identifying and segmenting camouflaged targets that are difficult to distinguish from complex backgrounds. To address the issues of incomplete detection and missing edges in camouflaged targets, this paper proposes a Bilateral Enhanced Complementary Network (BECNet) for COD. The network adopts a two-branch detection method, which is used for object recognition, edge recognition, and texture supervision respectively, to alleviate the feature ambiguity of features extracted from a single branch. Additionally, we introduce a Semantic Amplification Module (SAM) to further extract multi-scale semantic features. To effectively aggregate the discriminative features generated by both branches, we designed a Semantic-Texture Interaction Module (SIM). Finally, we incorporate an Edge Complementary Dual Attention Module (ECDA) during the decoding process to refine the model using edge information. Extensive experiments demonstrate the effectiveness and robustness of BECNet.

Travelable area boundaries not only constrain the movement of field robots but also indicate alternative guiding routes for dynamic objects. Publicly available road boundary datasets have outlined boundaries by binary segmentation labels. However, hard post-processes have to be done to extract from detected boundaries further semantics including the shapes of the boundaries and guiding routes, which poses challenges to a real-time visual navigation system without detailed prior maps. In addition, boundary detectors suffer from insufficient data collected from complex roads with severe occlusion and of different shapes. In this paper, a travelable area boundary dataset is semi-automatically built. 82.05% of the data is collected from bends, crossroads, T-shape roads and other irregular roads. Novel guiding semantics labels, shape labels and scene complexity labels are assigned to boundaries. With the support of the new dataset, travelable area boundary detectors could be trained, evaluated and fairly compared. The dataset can also be used to train, evaluate or test detectors for the road boundary detection task.

State-of-the-art autonomous driving systems heavily depend on prior dense maps. However, creating, storing, and transmitting such dense maps beyond complex environments is challenging due to the vast spatial scales involved. This limitation significantly hinders the widespread adoption of autonomous driving technology despite its potential to bring substantial societal benefits. In this paper, we propose a layered hybrid map construction method, which aims to improve the robustness and efficiency of autonomous mapping by using intersections as reference landmark nodes. Each node is organized across metric, semantic, and topology layers, offering crucial references for autonomous navigation without needing prior maps. To capture the road structure at intersection nodes, we propose an intelligent perception model that extracts three types of road semantic information: intersection type, curb characteristics, and road width. This perception model comprehensively and effectively extracts spatial structural representations around nodes, thereby aiding in accumulating semantic layers for map construction. Additionally, we have developed an enhanced Iterative Closest Point algorithm within a hierarchical architecture to improve node re-identification during the mapping process, ensuring stable and rapid localization in large-scale hybrid maps. This method incorporates artificial intelligence and offers a new paradigm for the re-identification of topology nodes. Experiments on six datasets demonstrate that the proposed hybrid mapping method can consistently and effectively construct semantic maps, achieving up to 97.58% similarity of road network topology structure with actual maps. The proposed method holds significant engineering application value in fields such as autonomous driving, robotics, geographic information systems, and urban planning.

Visible-Infrared person re-identification is a challenging yet important task in the field of intelligent surveillance. Most existing approaches focus on designing powerful deep networks to learn modality-shared representations, while little attention has been paid to using pre-training methods, although they can improve the performance of cross-modality tasks stably. This paper proposes a cross-modality masked pre-training (CMMP) method for visible-infrared person re-identification. Specifically, we generate color-irrelevant images using random channel exchangeable augmentation to minimize the difference between modalities at first. In the pre-training process, the visible together with the generated image, and the infrared image are masked by sharing the same random mask. Considering the misalignment of visible and infrared images in the datasets, we then reconstruct the masked areas only of the visible and the generated images using a lightweight decoder, which makes the pre-training process more efficient. Extensive experiments on two visible-infrared person re-identification datasets verify the effectiveness of the proposed method. CMMP outperforms the baseline method by +1.87\% and +1.24\% mAP on SYSU-MM01 and RegDB, respectively.

This article presents a new approach to address the challenge of combining ground-based LiDAR data with satellite images for cross-view image geo-localization. The task is to figure out the position and orientation of the LiDAR within the given satellite image. While previous research has mainly focused on imagery, the integration of ground-based point clouds with satellite images has been limited due to significant differences in modalities. To release this limitation, we propose a novel method that utilizes the road structure as a consistent reference between satellite images and ground LiDAR data for accurate geo-localization. Our methodology encompasses the extraction of road structures from both point clouds and satellite images. To extract road structures from point clouds, we leverage the enhanced viewpoint beam model, which effectively captures the spatial characteristics of ground landmarks. In addition, we utilize fractional-order differential-based super-resolution technology for satellite images to improve road structure detection, ensuring reliable performance across different altitudes. Following this, our approach involves matching road structures from the ground and satellite views, simplifying the localization process to a template-matching task. Consequently, we successfully address the challenge of accurately determining the 3-DoF pose of the LiDAR within the satellite image context. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in geo-localization, outperforming comparable methods. In addition, the approach shows versatility across various altitudes.

LiDAR super-resolution can improve the quality of point cloud data, which is critical for improving many downstream tasks such as object detection, identification, and tracking. Traditional LiDAR super-resolution models often struggle with issues like block artifacts, staircase edges, and misleading edges. To address these challenges, a novel super-resolution model of LiDAR based on fractional-order total variation (FOTV) is proposed in this paper. We propose a FOTV regularization optimization problem, utilizing an end-to-end trainable iterative network to capture data attributes.This enables the precise reconstruction of fine details and complex structures in point clouds. Specifically, the half-quadratic splitting algorithm divides the problem into data fidelity and prior regularization subproblems. We then propose a deep unfolding network, which iteratively deals with the two subproblems within the FOTV-HQS framework. Numerous experiments have shown that our approach significantly reduces the number of parameters by up to 99.68

Aerial object detection plays a crucial role in numerous applications. However, most existing methods focus on detecting predefined object categories, limiting their applicability in real-world open scenarios. In this paper, we extend aerial object detection to open scenarios through image-text collaboration and propose RT-OVAD, the first real-time open-vocabulary detector for aerial scenes. Specifically, we first introduce an image-to-text alignment loss to replace the conventional category regression loss, thereby eliminating category constraints. Next, we propose a lightweight image-text collaboration strategy comprising an image-text collaboration encoder and a text-guided decoder. The encoder simultaneously enhances visual features and refines textual embeddings, while the decoder guides object queries to focus on class-relevant image features. This design further improves detection accuracy without incurring significant computational overhead. Extensive experiments demonstrate that RT-OVAD consistently outperforms existing state-of-the-art methods across open-vocabulary, zero-shot, and traditional closed-set detection tasks. For instance, on the open-vocabulary aerial detection benchmarks DIOR, DOTA-v2.0, and LAE-80C, RT-OVAD achieves 87.7 AP_50, 53.8 mAP, and 23.7 mAP, respectively, surpassing the previous state-of-the-art (LAE-DINO) by 2.2, 7.0, and 3.5 points. In addition, RT-OVAD achieves an inference speed of 34 FPS on an RTX 4090 GPU, approximately three times faster than LAE-DINO (10 FPS), meeting the real-time detection requirements of diverse applications. The code will be released at https://github.com/GT-Wei/RT-OVAD.

Abstract Deep images can provide rich spatial structure information, which can effectively exclude the interference of illumination and road texture in road scene segmentation and make better use of the prior knowledge of road area. This paper first proposes a new cross‐modal feature maintenance and encouragement network. It includes a quantization statistics module as well as a maintenance and encouragement module for effective fusion between multimodal data. Meanwhile, for the problem that if the road segmentation is performed directly using a segmentation network, there will be a lack of supervised guidance with clear physical meaningful information and poor interpretability of learning features, this paper proposes two road segmentation models based on prior knowledge of deep image: disparity information and surface normal vector information. Then, a two‐branch neural network is used to process the colour image and the processed depth image separately, to achieve the full utilization of the complementary features of the two modalities. The experimental results on the KITTI road dataset and Cityscapes dataset show that the method in this paper has good road segmentation performance and high computational efficiency.

The adaptability of cross-view ground-to-aerial localization to varying illumination is a prerequisite for robot systems to work day and night. The proposed multi-modality semantic-shared cross-view ground-to-aerial localization method based on ground-view LiDAR-camera dual-domain joint perception enhances this adaptability. A point cloud semantic extraction module based on graph attention network is proposed to extract point cloud semantic features under the guidance of object-level semantic prior references which are less influenced by illumination. As a novel multi-modality fusion paradigm, the semantic sharing module prompts two observations interact with each other instead of fusing into one feature, which allows the two to complement each other and directly function in positioning stage. Our method is validated on the KITTI dataset where it surpasses the baseline by 8.1% in median localization error. In the CARLA simulator, through an unprecedented illumination adaptability experiment, we show the adaptability of our method to different illumination, including daytime, sunset, night and the weak illumination on rainy days. Compared to the baseline, our method reduces median errors up to 28.0% and 17.9% at night and on rainy days.

Intersection detection plays a crucial role in localizing and planning the path of autonomous vehicles in urban environments. This paper presents a novel approach, PVWO, for adaptive intersection detection in autonomous vehicles equipped with 3D LiDAR. Firstly, the algorithm removes ground and obstacles effectively, and eliminates noise points. Secondly, a polyarticular viewpoints beam model is applied to detect the type and location of intersections. Thirdly, a linear feature extraction approach in a rolling window is proposed to detect road width and optimize key parameters, enhancing model robustness across different scenes. Compared to parallel intersection detection algorithms, our method exhibits excellent performance under diverse road conditions. Experiments in sparse point clouds show an average precision exceeding 89

Fully convolutional neural networks have achieved great success in salient object detection, in which the effective use of multi-layer features plays a critical role. Based on this advantage, many saliency detectors have emerged in recent years, and most of them designed a series of network structures to integrate the multi-level features generated by the backbone network. However, information in different layer play different roles in saliency object detection, how to integrate them effectively is still a great challenge. In this article, a selective feature fusion network which consists of a selective feature fusion module (SFM) and an attention-guide hierarchical feature emphasis module (AEM) is proposed. Most of the previous works mainly integrate multi-level feature by addition and concatenation, as a difference, SFM adaptively selects the important information from the input features in the fusion, which effectively avoids introducing too much redundant information. Besides, AEM combines spatial attention and channel attention to enhance features simply and effectively by hierarchical iteration, and further improve the accuracy of salient object detection. Experiments on five datasets show that the proposed selective feature fusion method achieve satisfactory results when comparing to other state-of-the-art salient object detection approaches.

Self-supervised pretraining (e.g., MAE) based on reconstructing masked image patches has become increasingly popular in recent months. It commonly pretrains models on the large-scale ImageNet dataset with a considerably wider range of image categories, and then finetunes on downstream tasks such as classification, detection, segmentation, etc. However, we discover that for finetuning domain-specific tasks usually with a few or limited semantic spaces (e.g., object re-identification (ReID)), the model pretrained on a wider variety of sources (e.g., ImageNet) fails to be a good initialization. To address the problem, we propose a second-phase self-supervised pretraining on the specific domain, based on the self-supervised ImageNet pretraining models in the first phase. We perform studies on 2 domains of object ReID, i.e., person and vehicle ReID, where person ReID domain involves four datasets (Market-1501, DukeMTMC, MSMT17, Occluded-Duke) and vehicle ReID domain contains 2 datasets (VeRi-776, VehicleID). Through extensive experiments, we observe significant improvements in every single dataset based on the second-phase self-supervised pretraining over the entire unlabeled domain data. Specifically in person ReID task, compared to the self-supervised model in the first phase, our approach achieves +8.5%/+11.5%/+13.3%/+17.2% mAP improvements in Market-1501, DukeMTMC, MSMT17 and Occluded-Duke datasets, respectively. Furthermore, it also outperforms the supervised pretraining baselines, achieving improved performance by +1.5%/+4.3%/+3.9%/+4.2% mAP. We observe similar inspiring improvements in vehicle ReID tasks. Additionally, we also perform the second-phase pretraining on the specific tasks using its own unlabeled data, namely task-specific pretraining (TSP), which also consistently shows encouraging results across object ReID tasks.

Multi-focus image fusion aims to generate an all-in-focus image from multiple images focused on different regions. The targets of multi-focus image fusion vary with different regions of the image, e.g., flat regions retain smoothness, while edges and textures should be sharpened. However, the existing deep learning-based multi-focus image fusion methods usually treat the image as a whole for fusion and train the model by optimizing homogenous pixel-wise loss (e.g., MSE). This leads to the trained model tends to generate flat regions that are easy to reconstruct, failing to infer realistic details. In this paper, we propose a component divide-and-conquer model for multi-focus image fusion, which uses discrete wavelet transform to decompose source images into low-frequency and high-frequency components, and feeds them into different branches to aggregate them separately by the proposed attention feature fusion network. Finally, the fused image is obtained by inverse discrete wavelet transform. Such a strategy not only can address the challenges of different difficulties in the fusion of low-frequency and high-frequency components, but also able to supervise different components flexibly by the intermediate supervision learning strategy to generate realistic details of the fused image. Extensive experiments show that the proposed component divide-and-conquer model achieves significant improvements in both quantitative and qualitative evaluation.

Transformer has shown its effectiveness and advantage in many computer vision tasks, for example, image classification and object re‐identification (ReID). However, existing vision transformers are stacked layer by layer, lacking direct information exchange among every layer. Inspired by DenseNet, we propose a dense transformer framework (termed Denseformer) that connects each layer to every other layer through class tokens. We demonstrate that Denseformer can consistently achieve better performance on person ReID tasks across datasets (Market‐1501, DukeMTMC, MSMT17, and Occluded‐Duke), only at a negligible increase of computation. We show that Denseformer has several compelling advantages: it pays more attention to the main parts of human bodies and obtains discriminative global features.