In recent years, vehicle detection in uncrewed aerial vehicle (UAV) videos has attracted significant attention. However, the traditional detection paradigm based on single frame image is faced with the performance bottleneck caused by rare pose, motion blur, and occlusion. To address these issues, we propose a novel network comprising three important modules designed to fully exploit the spatiotemporal contextual information across video frames. First, To strengthen the backbone network’s capacity for extracting global contextual information from single-frame images, we propose the visual state space context module (VSSCM). By incorporating the 2D-selective-scan module (SS2D), VSSCM captures global dependencies and enriches contextual information without significantly increasing computational complexity. Second, the temporal information-guided spatial attention aggregation module (TGSAM) is introduced to fuse features from critical regions in adjacent frames. Finally, the self-attention-based classification feature aggregation module (SACFAM) is designed to model the relation between object features across frames and perform feature aggregation based on the learned relation matrix, thereby effectively improving the quality of classification features in UAV videos. Extensive experiments are conducted on the challenging VisDrone2019-VID dataset, and the experimental results demonstrate the effectiveness and superiority of the proposed method.

Remote sensing instance segmentation plays a vital role in geographic information systems, and it is crucial for Internet of Things applications such as smart city infrastructure management, traffic monitoring, and autonomous driving systems. Although fully supervised methods have achieved promising accuracy, they rely heavily on large amounts of pixel-level annotations, which are extremely costly to obtain for high-resolution remote sensing images. Box-supervised instance segmentation typically leverages horizontal bounding boxes as weak supervision signals, significantly reducing the annotation burden. However, mask prediction under box supervision faces challenges due to limited utilization of spatial information, including the lack of geometric details, the disconnect between spatial localization and pixel-level prediction, and the neglect of spatial priors in traditional pairwise affinity loss. To address these issues, this article proposes SALInst, a spatial affinity learning framework for box-supervised remote sensing instance segmentation. Specifically, a spatial information enhancement module and a dual-stream residual gate fusion mechanism are designed to strengthen spatial awareness and semantic coherence of mask features. Furthermore, by leveraging the spatial constraint prior, we propose a spatial affinity loss based on Gaussian kernel and total variation loss to reinforce the spatial consistency of predicted masks. Extensive experiments on the iSAID and NWPU VHR-10 datasets demonstrate that SALInst outperforms existing box-supervised methods while narrowing the performance gap between weakly and fully supervised instance segmentation.

Object detection in Unmanned Aerial Vehicle (UAV) imagery presents significant challenges, particularly regarding small, densely packed objects characterized by low resolution and feature ambiguity. To address these issues, this paper introduces D2-DETR, a novel detection framework that leverages a dual-domain architecture to synergistically model both frequency and spatial information. Central to our model is the D2-Backbone, which integrates three key components: (1) a Frequency Attention (FA) module that dynamically amplifies high-frequency features of small objects while suppressing background noise via learnable spectral weighting; (2) a Spatially-Guided Adaptive Dilation Rate Mechanism (SG-ADRM) that modulates dilation rates based on local frequency energy to balance detail preservation with large-context modeling; and (3) a Dynamic Dual-Shape Attention (DDA) module employing shape-specific convolutional kernels to capture both compact vehicle contours and elongated traffic queues. Furthermore, we propose a Spatial-Frequency Feature Pyramid Network (SF-FPN) that integrates dual-branch processing with real-valued FFT compression. This design enhances multi-scale fusion and ensures effective fine-grained information propagation while maintaining computational efficiency. Extensive experiments on challenging public benchmarks, VisDrone and UAVDT, demonstrate the superiority of our approach. Specifically, D2-DETR achieves state-of-the-art performance with mAP@.50 scores of 41.40% on VisDrone and 38.21% on UAVDT, outperforming existing detectors while retaining real-time inference capabilities. This work validates the efficacy of joint frequency-spatial modeling for small object detection, offering a robust solution for real-world aerial surveillance applications.

Image fusion aims to synthesize comprehensive scene representations by integrating complementary characteristics from multi-source images. Inspired from the human knowledge prior, existing methods typically focus on extracting predefined features to assist the fusion process, e.g., texture structure in visible images and thermal radiation in infrared images. However, such task-dependent schemes inevitably neglect the rich diversity of latent characteristics in multi-source image, which not only limits the exploration of the rich complementary cross-source information, but also degrades the generalization capability to diverse fusion tasks. In this paper, we propose a simple and unified method for generalized image fusion. Specifically, we generalize the image fusion task into common information collection and unique information maintenance, and design a common-unique decoupling network with a tri-encoder structure, in which a shared encoder and two private encoders are developed to extract common and unique features, respectively. To achieve robust common-unique information disentanglement, we design a decoupling loss that constrains the mutual-information among the tri-encoder outputs and source features. Unlike conventional approaches requiring human knowledge priors, our method automatically learns essential characteristics directly from input images. We conduct comprehensive experiments covering four image fusion tasks on nine popular datasets. Extensive experimental results demonstrate that our method achieves superior performance on all these tasks as compared of the respective state-of-the-art fusion methods, and show impressive cross-task generalization capability.

Multimodal image fusion and cross-modal translation are fundamental yet challenging tasks in computer vision, with their performance directly impacting downstream applications. Existing approaches typically treat these tasks independently, developing specialized models that fail to exploit the intrinsic relationships between different modalities. This limitation not only restricts model generalizability but also hinders further performance improvements. In this paper, we propose a joint optimization framework for image generation and fusion. Specifically, we generalize multimodal image tasks as the fusion and transformation of cross-modal features, and design a hybrid task training strategy. At the data level, we introduce a self-supervised and mutual-supervised hybrid mechanism for content-style feature decoupling, which achieves superior feature separation through stepwise training on intra-modal and cross-modal data. At the model level, we construct a triple-branch decoupling head along with fusion and transformation modules to ensure synchronous and efficient execution of dual tasks. Our method not only breaks through the single task limitation of the model, but also innovatively introduces mixed supervision into multimodal processing. We conduct comprehensive experiments covering four modalities fusion tasks on seven popular datasets. Extensive experimental results demonstrate that our method achieves superior performance on two tasks as compared of the respective state-of-the-art methods, and show impressive cross-task generalization capability.

Semi-supervised object detection (SSOD) provides a promising solution to mitigate the annotation costs in remote sensing applications. Mainstream teacher-student based SSOD methods leverage unlabeled images through pseudo labeling, and their effectiveness is fundamentally limited by the inevitable noise in pseudo labels, particularly for remote sensing (RS) scenarios with complex backgrounds and dense, multi-scale and oriented objects. Current methods primarily focus on reducing pseudo label noise through category, scale and Intersection over Union information mining, as well as designing fine-grained confidence thresholding strategies. However, the inherent discrepancy between classification and localization reliability is neglected. In this study, with analyzing the characteristic discrepancies between the classification and localization branches, We propose artificial intelligence (AI) methodological innovation method named cross-branch information incorporation method (i.e., CBI-SSOD) to utilize these discrepancies to assist the training of the classification branch, and thus improve the performance of SSOD methods. Specifically, our method present two key AI innovations. Firstly, we propose a pretext task to extract cross-branch information, which can improve the classification ability by reinforce the consistent predictions between the classification branch and the pretext task. Besides, we propose a pseudo label reassignment approach to adjust the soft classification pseudo labels, and thus suppress pseudo label noise and improve the detection performance. Extensive experiments on Dataset for Object Detection in Aerial Images (DOTAv1.0) and DOTAv1.5 datasets validate the effectiveness and superiority of our method, and demonstrate the practical engineering impact of our method on RS applications and interpretation systems.

Remote sensing instance segmentation aims to detect and assign pixel-level labels to each instance in remote sensing images, which holds critical engineering significance for both civil and military applications. While existing domain-specific methods have made progress, they still struggle with three persistent challenges: ineffective context modeling in cluttered backgrounds, information loss during multi-scale feature fusion, and blurred boundaries for densely clustered small objects. To address these limitations, we propose a novel remote sensing instance segmentation framework with three artificial intelligence (AI) methodological innovations, which comprises: a Context Perception Module (CPM) for context modeling, a Context Guided Multi-Scale Feature Aggregation (CGFA) method for multi-scale feature fusion, and a Multi-Path Region Proposal Extractor (MPRPE) with boundary-refined segmentation. The CPM leverages the selective state space model (Mamba) to capture long-range contextual information, effectively addressing the issue of cluttered backgrounds in remote sensing images. The CGFA replaces standard feature pyramid network architecture which is limited by direct summation or concatenation, preserving fine-grained spatial details with context guidance. The MPRPE and boundary-aware segmentation head mitigate the challenges of missed detection of small objects and blurred edge predictions, which arise from the clustered distribution of small objects and semantic ambiguity. Extensive experiments on the challenging iSAID and NWPU VHR-10 datasets validate the proposed method's consistent improvements across metrics while demonstrating its practical engineering impact on remote sensing interpretation systems.

Unmanned aerial vehicles (UAV) are widely used for their small size and flexibility. However, the large number of small objects and the significant difference in object size in UAV images bring great challenges to the detection task. Therefore, we propose an object detection method for UAV images with four improvements on the strong baseline model YOLOX-S, which is robust to detect small objects and multi-scale objects. Firstly, we introduce a high-resolution feature map to retain rich detailed information about small objects. Secondly, we propose new up-sampling and down-sampling modules to reduce the feature information loss during the sampling process. Thirdly, we present the triple-scale feature fusion module (TSFFM) to fuse more abundant multi-scale features in the neck’s bottom-up feature fusion process. Finally, the parrell dilated convolution attention module (PD-CAM) is proposed to learn the multi-receptive field features. Experiment results on the VisDrone-VID2019 dataset validate the effectiveness and superiority of the proposed method.

Remote sensing (RS) instance segmentation is an important but challenging task due to multi-oriented, densely arranged objects and lack of mask annotation. Compared with redundant horizontal bounding-box (HBB) and expensive pixel-level annotation, oriented bounding box (OBB) annotations can provide compact object depicts with lower annotation costs. Therefore, we propose the first weakly supervised remote sensing instance segmentation method with OBB supervision (namely OBBInst) to reduce the annotation burden and make full use of existing abundant OBB annotations. Based on BoxInst (a high-performance instance segmentation method with box annotations), OBBInst has customized a framework for OBB annotation to unify the incompatibility between existing HBB-based and OBB-based methods. In addition, we propose an oriented projection method with a corresponding loss function to achieve more precise target depicts of OBB annotation. Moreover, we propose an edge similarity loss to incorporate Canny edge prior into deep learning framework for more precise edge identification of densely arranged objects. We have conducted extensive experiments on iSAID and HRSC datasets, and the experimental results demonstrate that OBBInst can achieve the state-of-the-art performance as compared to existing box-supervised methods. In addition, OBBInst dramatically narrows the performance gap between weakly and fully supervised instance segmentation (23.9% vs. 35.6% in iSAID dataset and 79.5% vs. 84.9% in HRSC dataset).

Semantic segmentation is an essential task in polarimetric synthetic aperture radar (PolSAR) image interpretation. To address the issue of insufficient measurement ability of single-view similarity, an unsupervised semantic segmentation method for PolSAR images based on multiview similarity is proposed to estimate the number-of-classes (NoC) and perform classification. NoC estimation is commonly neglected in semantic segmentation methods, to ensure rationality, a NoC estimation method before clustering is proposed based on multiview polarimetric rotation domain features and the visual assessment of tendency method for PolSAR images without supervision. Then, the norm distance, geodesic distance, maximum likelihood distance, and generalized likelihood ratio test distance based on the statistical characteristics of PolSAR images are comprehensively analyzed. Various advantages of different distances are integrated to combine the multiview vector information and scattering information to construct six multikernel similarity matrices. Subsequently, the consensus similarity network fusion method is utilized to further strengthen the discriminative ability of the similarity matrices. In addition, efficient superpixel segmentation is also adopted to reduce the speckle noise. Finally, based on the estimated NoC and the fused similarity matrix, spectral clustering is utilized to obtain semantic segmentation results. Extensive experiments are conducted on two AIRSAR datasets and one Gaofen-3 dataset demonstrate that the proposed method can effectively combine the spatial neighborhood similarity information and achieve higher semantic segmentation accuracy.

Pointly Supervised Object Detection (PSOD) has attracted considerable interests due to its lower labeling cost as compared to box-level supervised object detection. However, the complex scenes, densely packed and dynamic-scale objects in Remote Sensing (RS) images hinder the development of PSOD methods in RS field. In this paper, we make the first attempt to achieve RS object detection with single point supervision, and propose a PSOD method tailored for RS images. Specifically, we design a point label upgrader (PLUG) to generate pseudo box labels from single point labels, and then use the pseudo boxes to supervise the optimization of existing detectors. Moreover, to handle the challenge of the densely packed objects in RS images, we propose a sparse feature guided semantic prediction module which can generate high-quality semantic maps by fully exploiting informative cues from sparse objects. Extensive ablation studies on the DOTA dataset have validated the effectiveness of our method. Our method can achieve significantly better performance as compared to state-of-the-art image-level and point-level supervised detection methods, and reduce the performance gap between PSOD and box-level supervised object detection. Code is available at https://github.com/heshitian/PLUG.

Supervised learning based object detectors suffer from the high cost and difficulty of labeling datasets. Self-supervised learning methods require no manual annotations. However, the misalignment between the pretext task designed for image classification and the downstream task affects the detection performance. Therefore, this paper proposes a self-supervised dense contrastive learning method to improve performance of object detection in remote sensing images. Specifically, first, Swin Transformer substitutes popular CNN to extract features of augmented multiple views. Second, global and local features are extracted using parallel global and dense projector heads, respectively. Third, a predictor head is added to increase the nonlinear transformations in the network. Extensive experiments on the NWPU VHR-10 dataset show that the proposed method outperforms two representative strong baseline methods, including MoCoV2 and DenseCL.

As wastewater treatment usually involves complicated biochemical reactions, leading to strong coupling correlation and nonlinearity in water quality parameters, it is difficult to analyze and optimize the control of the wastewater treatment plant (WWTP) with traditional mathematical models. This research focuses on how deep learning techniques can be used to model the data from a specific WWTP so as to optimize the required energy consumption. In the operation of a wastewater treatment plant, various sensors are used to record the treatment process data; these data are used to train deep neural networks (DNNs). A long short-term memory with multilayer perceptron network (LMPNet) model is proposed to model the water quality parameters and site control parameters, such as COD, pH, NH3-N, et al., and the LMPNet model prediction error is then measured by criteria such as the MSE, MAE, and R-2. The experimental results show that the LMPNet model demonstrates great accuracy in the modeling of the control of WWTPs. A life-long learning strategy is also developed for the LMPNet in order to adapt to the environment that may change over time. By developing performance evaluation metrics, the purification performance can be analyzed, and the prediction reference can be provided for the subsequent control optimization and energy saving plan.

A significant challenge in detecting objects in complex remote sensing (RS) datasets is from small objects. Existing detection methods achieve much lower accuracy on small objects than medium and large ones. These methods suffer from limited feature information, susceptibility to complex background interferences, and insufficient contextual information. To address these issues, a small object detection method with the enhanced receptive field, ERF-RTMDet, is proposed to achieve a more robust detection capability on small objects in RS images. Specifically, three modules are employed to enhance the receptive field of small objects’ features. First, the Dilated Spatial Pyramid Pooling Fast Module is proposed to gather more contextual information on small objects and suppress the interference of background information. Second, the Content-Aware Reassembly of Features Module is employed for more efficient feature fusion instead of the nearest-neighbor upsampling operator. Finally, the Hybrid Dilated Attention Module is proposed to expand the receptive field of object features after the feature fusion network. Extensive experiments are conducted on the MAR20 and NWPU VHR-10 datasets. The experimental results show that our ERF-RTMDet attains higher detection precision on small objects while maintaining or slightly enhancing the detection precision on mid-scale and large-scale objects.

Superpixel generation of polarimetric synthetic aperture radar (PolSAR) images is widely used for intelligent interpretation due to its feasibility and efficiency. However, the initial superpixel size setting is commonly neglected, and empirical values are utilized. When prior information is missing, a smaller value will increase the computational burden, while a higher value may result in inferior boundary adherence. Additionally, existing similarity metrics are time-consuming and cannot achieve better segmentation results. To address these issues, a novel strategy is proposed in this article for the first time to construct the function relationship between the initial superpixel size (number of pixels contained in the initial superpixel) and the structural complexity of PolSAR images; additionally, the determinant ratio test (DRT) distance, which is exactly a second form of Wilks' lambda distribution, is adopted for local clustering to achieve a lower computational burden and competitive accuracy for superpixel generation. Moreover, a hexagonal distribution is exploited to initialize the PolSAR image based on the estimated initial superpixel size, which can further reduce the complexity of locating pixels for relabeling. Extensive experiments conducted on five real-world data sets demonstrate the reliability and generalization of adaptive size estimation, and the proposed superpixel generation method exhibits higher computational efficiency and better-preserved details in heterogeneous regions compared to six other state-of-the-art approaches.

Synthetic aperture radar (SAR) images have been extensively used in earthquake monitoring, resource survey, agricultural forecasting, etc. However, it is a challenge to interpret SAR images with severe speckle noise and geometric deformation due to the nature of radar imaging. The translation of SAR-to-optical images provides new support for the interpretation of SAR images. Most of the existing translation networks, which are based on generative adversarial networks (GANs), are vulnerable to part information loss during the feature reasoning stage, making the outline of the translated images blurred and semantic information missing. Aiming to solve these problems, cross-fusion reasoning and wavelet decomposition GAN (CFRWD-GAN) is proposed to preserve structural details and enhance high-frequency band information. Specifically, the cross-fusion reasoning (CFR) structure is proposed to preserve high-resolution, detailed features and low-resolution semantic features in the whole process of feature reasoning. Moreover, the discrete wavelet decomposition (WD) method is adopted to handle the speckle noise in SAR images and achieve the translation of high-frequency components. Finally, the WD branch is integrated with the CFR branch through an adaptive parameter learning method to translate SAR images to optical ones. Extensive experiments conducted on two publicly available datasets, QXS-SAROPT and SEN1-2, demonstrate a better translation performance of the proposed CFRWD-GAN compared to five other state-of-the-art models.

Due to the large sizes of synthetic aperture radar (SAR) images, traditional deep learning-based ship detection methods usually utilize the sliding window preprocessing strategy to obtain the small-sized subimages. However, there are amounts of background clutter areas without ships in SAR images. Thus, traditional sliding window-based methods may generate numerous subimages without ships, which can bring high computation redundancy and numerous false alarms. To deal with the above problems, in this article, a novel detection method called global and local context-aware ship detector for high-resolution SAR images is proposed. The proposed method mainly contains three parts: the global context-aware-based subimages selection (GCSS) module, the deep learning module, and the local context-aware-based false alarms suppression (LCFS) module. The GCSS module employs the global context information to eliminate the subimages without ships, which can enhance detection efficiency and avoid generating numerous false alarms. The deep learning module is used to further obtain the preliminary detection boxes. The LCFS module is a postprocessing step, which employs the local context information around the detection boxes to further eliminate the false alarms. The experimental results on the measured data of AIR-SARShip-1.0 and 2.0 high-resolution SAR images demonstrate that the proposed method has higher precision and efficiency than the original deep learning methods.