As the use of autonomous aerial vehicles (AAVs) for waterfront monitoring increases, combining multiple AAV multispectral (MS) images into a single, seamless panoramic image has become crucial. This process ensures the accuracy and effectiveness of waterfront monitoring. However, the varying reflective properties of different wavelengths bring challenges for existing single-band MS image stitching frameworks, especially in complex waterfront areas. To address this challenge, we developed the individual band enhanced waterfront multispectral stitching (IBEWMS) framework. Central to this framework is the individual band spectral feature enhancement (IBSFE) module, which enhances each spectral band based on varying reflectance of different land covers, yielding clearer and more reliable features. Using IBSFE, we designed a detector-free framework to effectively extract and match feature points in waterfront MS images. In addition, we implemented an image fusion technique to address issues, such as ghosting and global reflectance inconsistency, in panoramic images. To support this work, we provided the Wuhan AAV Waterfront Environment MS Dataset, comprising 12 315 high-resolution 5-band MS images. Experiments show that IBEWMS outperforms both deep learning and traditional stitching frameworks, offering valuable support for downstream applications.

Crop segmentation enables agricultural producers to comprehensively understand the state of their farmland, make more informed management decisions and thereby ensure food security. Unmanned aerial vehicle (UAV) remote sensing technology offers cost-effective high-resolution imaging for crop segmentation. Deep learning-based methods have continued to improve the accuracy of crop segmentation over time, but accurate edge segmentation remains a challenge. In this letter, we introduce a convolutional neural network (CNN) termed edge enhancement network (EENet) to improve the representation of edge information for crop segmentation in UAV RGB images. Our approach centers on an innovative edge enhancement (EE) Strategy that augments the learning of edge information during the training phase and refines the representation of edge details during the generation phase. To facilitate the evaluation of our method, we have produced a publicly available dataset for experimentation. Our results, as demonstrated on this self-constructed dataset, illustrate that our proposed approach surpasses competing methods in critical metrics such as mean intersection over union (mIoU), F1_score, and model complexity.

Change detection (CD) in remote sensing imagery is a crucial task with applications in environmental monitoring, urban development, and disaster management. CD involves utilizing bitemporal images to identify changes over time. The bitemporal spatial relationships between features at the same location at different times play a key role in this process. However, existing change detection networks often do not fully leverage these spatial relationships during bitemporal feature extraction and fusion. In this work, we propose SRC-Net: a bitemporal spatial relationship concerned network for CD. The proposed SRC-Net includes a perception and interaction module that incorporates spatial relationships and establishes a cross-branch perception mechanism to enhance the precision and robustness of feature extraction. In addition, a patch-mode joint feature fusion module is introduced to address information loss in current methods. It considers different change modes and concerns about spatial relationships, resulting in more expressive fusion features. Furthermore, we construct a novel network using these two relationship concerned modules and conducted experiments on the LEVIR-CD and WHU Building datasets. The experimental results demonstrate that our network outperforms state-of-the-art methods while maintaining a modest parameter count. We believe that our approach sets a new paradigm for CD and will inspire further advancements in the field.

Amid the brisk evolution of remote sensing (RS) technology, the domain of RS cross-modal text-image retrieval (RSCTIR) has captivated scholarly interest for its superior adaptability and symbiotic interaction with human operators. However, due to the heterogeneity between image and text data modalities, feature alignment poses a significant challenge. The existing methodologies overlook the sufficient incorporation of structural guidance during the cross-modal feature interaction alignment process to foster alignment between text and image features. In light of this, we propose an innovative approach for RS image-text retrieval task called visual global-salient-guided network (VGSGN), which comprises two branches: the image branch and the text branch. In the image branch, visual global-salient information sensing module (VGSM) is devised to extract visual global and salient features, aiming to enhance the perception capability for complex backgrounds and scenes in RS images. In the text branch, the textual graph enhancement module (TGEM) is crafted to filter out redundant information in the text features and capture the interactions between words within the text. The design of the multiple visual-guided dynamic fusion (MVGF) module aims to leverage the global and salient features of image to guide the text feature, facilitating cross-modal alignment of text and image features. The experimental results on the widely recognized RSICD and RSITMD datasets corroborate the effectiveness and advancement of our proposed VGSGN in tackling the RSCTIR task.

Accurate remote sensing estimation of inland water quality parameters (WQPs) plays a crucial role in guiding water resource management. To achieve this, researchers have explored various data-driven approaches utilizing machine learning (ML) techniques. However, there are two major challenges in WQPs estimation for inland waters. Firstly, current data-driven approaches focus on building a unified estimation model for an entire study area, which underestimates the complex dynamics of water constituents and optical properties. Secondly, ML models, particularly neural networks, require extensive hyperparameter tuning and are not user-friendly for researchers lacking relevant background and experience. In this paper, we propose an innovative method called adaptive and automated data-driven water quality parameter estimation (A2DWQPE) to address both challenges. Our method operates under the assumption that water bodies with similar spectral characteristics should share the same WQP estimation model. A2DWQPE is composed of three phases. Firstly, water types are automatedly classified by unsupervised hierarchical clustering according to spectral similarity. Then, optimal Deep Neural Network (DNN) models for estimating WQPs from multi-spectral satellite images are customized for each water type utilizing Bayesian optimization (BO). Finally, the target WQP is estimated based on the type-specific estimates and degree of membership of each water type. To evaluate the effectiveness of A2DWQPE, we applied it to estimate Secchi disk depth (SDD) in Lake Erie with in situ measurements and Moderate Resolution Imaging Spectroradiometer (MODIS) images. The results demonstrate that A2DWQPE outperforms the traditional approaches of developing a unified model for the entire study area. A2DWQPE achieved high accuracy with coefficient of determination (R2) over 0.72 and root mean square error (RMSE) below 1.4 m. Our method also outperforms the methods that applied Genetic Algorithm (GA) and Particle Swarm Optimization (PSO) instead of BO, and several traditional ML algorithms. We firmly believe that A2DWQPE holds great potential for accurate inland water quality estimation and will contribute significantly to various applications in water quality monitoring and pollution prevention.

The depth estimation of nighttime images is a challenging problem due to the lack of accurate ground-truth depth labels. Although various self-supervised methods leveraging texture information have been proposed to solve the problem, the performance is still not satisfactory due to the imaging limitations of visible cameras. To this end, we propose a self-supervised Reflectance-Aware Depth Estimation approach based on reflectance for nighttime images. Two major factors strengthen the proposed approach: a Reflectance Extraction Network and a feature consistency loss. We introduce the Reflectance Extraction Network to extract texture information based on the finding that the texture is beneficial for depth estimation. Then, we utilize the feature consistency loss to help the baseline network to learn the intrinsic feature rather than the images’ light. Experiment results on the challenging Oxford RobotCar dataset confirm the robustness and effectiveness of our approach.

The expensive acquisition of labeled data limits the practical use of supervised learning on polarimetric synthetic aperture radar (PolSAR) image analysis. Semi-supervised learning has attracted considerable attention as it can utilize few labeled data and very many unlabeled data. The scattering response of PolSAR data is strongly spatial distribution dependent, which provides rich information about land-cover properties. In this paper, we propose a semi-supervised learning method named multi-domain fusion graph network (MDFGN) to explore the multi-domain fused features including spatial domain and feature domain. Three major factors strengthen the proposed method for PolSAR image analysis. Firstly, we propose a novel sample selection criterion to select reliable unlabeled data for training set expansion. Multi-domain fusion graph is proposed to improve the feature diversity by extending the sample selection from the feature domain to the spatial-feature fusion domain. In this way, the selecting accuracy is improved. By few labeled data, very many accurate unlabeled data are obtained. Secondly, multi-model triplet encoder is proposed to achieve superior feature extraction. Equipped with triplet loss, limited training samples are fully utilized. For expanding training samples with different patch sizes, multiple models are obtained for the fused classification result acquisition. Thirdly, multi-level fusion strategy is proposed to apply different image patch sizes for different expanded training data and obtain the fused classification result. The experiments are conducted on Radarsat-2 and AIRSAR images. With few labeled samples (about 0.003–0.007%), the overall accuracy of the proposed method ranges between 94.78% and 99.24%, which demonstrates the proposed method’s robustness and excellence.

Optical remote sensing image (RSI) is easily affected by weather conditions. When the ground target is sheltered by clouds, extracting scene information from the RSI becomes quite challenging. In this work, we propose a distraction-attention-driven adversarial training network (DA2Net) to learn a robust RSI scene classification model. The distraction module employs a gradient-based class activation mapping (GradCAM++) method to produce partially occluded samples. Through feature map visualization, GradCAM++ can quantify the contribution of each region to the network prediction. Regions in the input image are erased and filled with white pixels if the corresponding contribution is higher than a given threshold. In this way, the distraction module enriches the training sample diversity and benefits the network’s robustness and generalization performance. Training with the partially erased samples, the model can extract sufficient information from other regions even though the target with prominent features is occluded. The attention module highlights important features and information. It encourages the network to mine critical features from the uncovered regions. Competition between the two modules drives the network to improve its robustness and overall performance. Extensive experiments show that the DA2Net provides a promising approach for data augmentation and network training. Analysis of cloud-covered scene classification demonstrates the DA2Net’s robust performance.

Change detection (CD) is an essential earth observation technique. It captures the dynamic information of land objects. With the rise of deep learning, convolutional neural networks (CNNs) have shown great potential in CD. However, current CNN models introduce backbone architectures that lose detailed information during learning. Moreover, current CNN models are heavy in parameters, which prevents their deployment on edge devices such as unmanned aerial vehicles (UAVs). In this work, we tackle this issue by proposing RDP-Net: a region detail preserving network for CD. We propose an efficient training strategy that constructs the training tasks during the warmup period of CNN training and lets the CNN learn from easy to hard. The training strategy enables CNN to learn more powerful features with fewer floating point operations (FLOPs) and achieve better performance. Next, we propose an effective edge loss that increases the penalty for errors on details and improves the network's attention to details such as boundary regions and small areas. Furthermore, we provide a CNN model with a brand new backbone that achieves the state-of-the-art (SOTA) empirical performance in CD with only 1.70 M parameters. We hope our RDP-Net would benefit the practical CD applications on compact devices and could inspire more people to bring CD to a new level with the efficient training strategy. The code and models are publicly available at https://github.com/Chnja/RDPNet.

Pavement texture evaluation is important for driving both skid resistance and pavement maintenance. Limited by the requirements of automation, efficiency and data coverage requirements, most pavement methods focus on damaged areas and static measurement environment. However, maintenance work is practically performed on the entire pavement rather than only the damaged areas, thus a gap between theory and practice is observed. In this study, we designed an efficient texture decomposition method based on the proposed step signal extraction algorithm, which can overcome road fluctuations and accurately extract pavement texture. The Pavement Transformer is introduced for fine texture evaluation, and can better serve pavement maintenance in practice. We conducted experiments on 22,800 pieces of 3D laser scanning data. The results demonstrate that our decomposition method has improved accuracy and stability. Moreover, the classification accuracy of texture level evaluation is 95.2%, which is better than that of the Vision Transformer.

The scattering response of polarimetric synthetic aperture radar (PolSAR) data is strongly target orientation-dependent. Formulating the polarimetric matrix as sequential data by rotating the polarimetric matrix along the radar line of sight would provide rich information about land-cover properties. In this work, we propose a composite sequential network (CSN) with polarization orientation angle (POA) attention to model the polarimetric coherency matrix sequence and explore target scattering orientation diversity features. Three major factors strengthen the proposed method for PolSAR image analysis. First, CSN improves the feature comprehensiveness by extending the interpretation mode of PolSAR data from spatial polarization to spatial polarization orientation. In this way, CSN could describe polarimetric response dynamics at different orientations. Second, a two-stream composite network with both real- and complex-valued convolutional long short-term memory (ConvLSTM) network is proposed to process the diagonal and off-diagonal elements of the coherency matrix sequence, respectively. Compared to existing real-/complex-valued networks, the CSN explores the significant phase information of the off-diagonal elements by operations in the complex domain. Meanwhile, CSN prevents padding 0 meaninglessly in the imaginary part of the real-valued diagonal elements. Third, during the sequential modeling of the polarimetric matrix, a POA attention mechanism is proposed. Equipped with POA-sensitive decomposition loss, the CSN attends to substantial POA range derived by targets' physical scattering mechanism and learns features closely related to the scattering mechanism. Extensive experiments and analysis on land-cover classification demonstrate the proposed method's robustness and excellence.

Synthetic aperture radar (SAR) imagery change detection (CD) is still a crucial and challenging task. Recently, with the boom of deep learning technologies, many deep learning methods have been presented for SAR CD, and they achieve superior performance to traditional methods. However, most of the available convolutional neural networks (CNN) approaches use diminutive and single convolution kernel, which has a small receptive field and cannot make full use of the context information and some useful detail information of SAR images. In order to address the above drawback, pyramidal convolutional block attention network (PCBA-Net) is proposed for SAR image CD in this study. The proposed PCBA-Net consists of pyramidal convolution (PyConv) and convolutional block attention module (CBAM). PyConv can not only extend the receptive field of input to capture enough context information, but also handles input with incremental kernel sizes in parallel to obtain multi-scale detailed information. Additionally, CBAM is introduced in the PCBA-Net to emphasize crucial information. To verify the performance of our proposed method, six actual SAR datasets are used in the experiments. The results of six real SAR datasets reveal that the performance of our approach outperforms several state-of-the-art methods.

In this paper, we present a challenging stereo-inertial dataset collected onboard a sports utility vehicle (SUV) for the tasks of visual-inertial odometry (VIO), simultaneous localization and mapping (SLAM), autonomous driving, object detection, and other computer vision techniques. We recorded a large set of time-synchronized stereo image sequences (2 × 1280 × 720 @ 30 fps RGB) and corresponding inertial measurement unit (IMU) readings (400 Hz) from a Stereolabs ZED2 camera, along with centimeter-level-accurate six-degree-of-freedom ground truth (100 Hz) from a u-blox GNSS-IMU navigation device with real-time kinematic correction signals. The dataset comprises 34 sequences recorded during November 2020 in Wuhan, the largest city of Central China. Further, the dataset contains abundant unique urban scenes and features of a complex modern metropolis, which have rarely appeared in previously released benchmarks. Results from milestone VIO/SLAM algorithms reveal that methods exhibiting excellent performance on established datasets such as KITTI and EuRoC perform unsatisfactorily when moved outside the laboratory to the real world. We expect our dataset to reduce this limitation by providing more challenging and diverse scenarios to the research community. The full dataset with raw and calibrated data is publicly available along with a lightweight MATLAB/Python toolbox for preprocessing and evaluation. The dataset can be downloaded in its entirety from the uniform resource locator (URL) we provide in the main text.

Limited labels and increasing multisource data promote domain adaptation (DA) problem as a challenging study for polarimetric synthetic aperture radar (PolSAR) interpretation. Existing DAs for optical images cannot generalize over PolSAR imagery due to its special side-imaging characteristics and complex distribution shifts. In this letter, a general feature paradigm (GFP) is proposed for unsupervised cross-domain PolSAR image classification. The GFP is based on a key observation that interclass aggregation is optimized after four-step feature transformations. This key observation leads to GFP that not only reduces the domain shifts but also compatible with typical DA methods. The GFPs are conducted on both source and target domain by unsupervised manner, including polarimetric basis extraction, the Wishart clustering, histogram statistics, and dimensionality reduction. After these transformations, the unlabeled target PolSAR image can be classified based on obtained GFP, DA, and limited labeled samples only from the source domain. Extensive unsupervised cross-domain experiments on 27 scenarios verified that GFP leads to at most 93.76% accuracy for full- and dual-polarized synthetic aperture radar (SAR) images’ classification. Moreover, the GFP shed light on extensive cross-domain PolSAR applications about built-up areas, vegetation, and bare land analysis.

Increasing amounts of polarimetric synthetic aperture radar (PolSAR) images from different sensors covering different scenes are available, but limited labeled samples and trained models can hardly work well in these cross-domain data interpretations. Fortunately, domain adaptation (DA) can transfer knowledge in existing images to new yet related images. DA shows attractive potential for PolSAR classification, and it is still challenging due to more complex domain shifts caused by different sensors, imaging conditions, and distributions. Inspired by the widely applicable polarimetric scattering mechanisms and DA ability of subspace alignment (SA), this article is devoted to constructing a robust unsupervised cross-domain PolSAR classification framework, by exploring scattering and statistical characteristics mapping between the source and target domains. First, classical scattering components of both source and target data were extracted, and Wishart clustering was adopted to derive the statistical information of scattering components at patch level. Second, the intrinsic polarimetric scattering components were estimated and extracted, which were called statistical scattering components (SSCs). Third, by applying SA, the source SSC was aligned with target SSC, and domain shift was further reduced. Finally, the target PolSAR image was classified based on labeled samples from source domain, and unsupervised cross-domain classification was achieved by SSC-based SA (SSC-SA). The unsupervised cross-domain experiments are conducted on 49 units among 11 data sets, including Radarsat-2, Gaofen-3, AIRSAR, and Pi-SAR images. With randomly selected labeled samples (about 2%–10%) from source domain, the accuracies of the proposed cross-domain classifications range between 80.20% and 95.64%. Also, the proposed SSC feature pattern is proved extensible for other polarimetric basis and decompositions.

Benefiting from the advantages of flexibility and timeliness, Unmanned Aerial Vehicles (UAVs) play an important role in crop growth monitoring, precision agriculture and intelligent agriculture. This paper focuses on the farmland boundary detection in UAV images Traditional farmland boundary detection methods have problems such as over-segmentation and discontinuous boundary. To address these problems, we propose a Deep Boundaries Combination (DBC) algorithm for the detection of farmland plots boundaries in UAV remote sensing images. DBC uses deep convolutional networks to obtain edge probability map of farmland images, and then applies Oriented Watershed Transform (OWT) and Ultrametric Contour Map (UCM) to convert edge probability map into closed boundary hierarchy tree, which layers the boundaries by edge probability. We perform experiments on two farmland images acquired by UAV. Experimental results show that our method can extract more accurate farmland boundaries than other methods.

Supervised deep learning models usually need large amounts of labeled data due to the data-driven training strategies, and its applicability to the newly emerging categories that lack annotated images is severely limited. In contrast, few-shot learning aims to recognize novel targets from very few labeled examples, so it will be a promising method for synthetic aperture radar (SAR) image interpretation, where numerous labeled data may not exist. In this paper, we introduced a few-shot learning method based on relation network and graph neural network (GNN). Relation network extracts the feature similarity between query samples and support samples through a convolutional neural network, and it has achieved good performance in few-shot learning problems. GNNs have received increasing attention in recent years, and they have shown superior performance in relation extraction. In this work, we replaced the relation module in the relation network with attention GNN, aiming to model the relationship between the samples more effectively and learn a better metric for feature similarity. Experiments on the MSTAR dataset demonstrate that the proposed method can better extract the relationship between query samples and support samples, thereby improving the performance for few-shot image classification tasks.

Polarimetric synthetic aperture radar (PolSAR) provides important support for the built-up areas (BA) information analysis, due to the ability of weather-independent imaging and sensitivity to targets scattering and geometric characteristics. However, PolSAR BA with large orientation angles is usually misdetected as vegetation, and labeled BA samples with special orientations are difficult to obtain. Furthermore, the labeled BA samples and trained models can hardly work well in the cross-domain PolSAR imagery BA analysis. This article presents a PolSAR BA extraction method based on eigenvalue statistical components (ESC) and PU-Learning (PUL), and it helps to realize cross-domain BA extraction by combining subspace alignment (SA). First, the roll invariance of coherency-matrix eigenvalues and building orientation effects are analyzed. Then, by adopting the eigenvalue-Wishart unsupervised classification, regional statistical information and rotation-invariant property are comprehensively utilized in ESC. Finally, the BA can be extracted by combining a PUL classifier with only positive samples at the same distinguishable orientation. Combined with SA, the novel ESC-PUL-SA domain adaptation facilitates a robust unsupervised cross-domain PolSAR BA analysis, reducing the differences caused by sensors and imaging scenes. The ESC-PUL BA extraction on seven PolSAR imageries showed that the accuracies reach 92%–98% with only a few positive samples (less than 0.65%). The ESC-PUL-SA performance was further validated by 14 unsupervised cross-domain BA analysis units among 10 datasets, including Radarsat-2, Gaofen-3, AirSAR, and UAVSAR images. With randomly selected positive samples from the source domain, the proposed ESC-PUL-SA achieved accuracies of all cross-domain BA extraction range from 89.64% to 95.53%.

Deep learning can archive state-of-the-art performance in polarimetric synthetic aperture radar (PolSAR) image classification with plenty of labeled data. However, obtaining large number of accurately labeled samples of PolSAR data is very hard, which limits the practical use of deep learning. Therefore, unsupervised PolSAR image classification is worthy of further investigation that is based on deep learning. Inspired by the superior performance of deep mutual information in natural image feature learning and clustering, an end-to-end Convolutional Long Short Term Memory (ConvLSTM) network is used in order to learn the deep mutual information of polarimetric coherent matrices in the rotation domain with different polarimetric orientation angles (POAs) for unsupervised PolSAR image classification. First, for each pixel, paired “POA-spatio” samples are generated from the polarimetric coherent matrices with different POAs. Second, a special designed ConvLSTM network, along with deep mutual information losses, is used in order to learn the discriminative deep mutual information feature representation of the paired data. Finally, the classification results can be output directly from the trained network model. The proposed method is trained in an end-to-end manner and does not have cumbersome pipelines. Experiments on four real PolSAR datasets show that the performance of proposed method surpasses some state-of-the-art deep learning unsupervised classification methods.

The total phosphorus (TP) concentration is a key water quality parameter for water monitoring and a major indicator of the state of eutrophication in inland lakes. Using remote-sensing to estimate TP concentration is useful, as it provides a synoptic view of the entire water region; however, the weak optical characteristics of TP lead to difficulty in accurately estimating TP concentration. The differences in water characteristics and components between lakes mean that most TP estimation methods are not applicable to all lakes. An artificial neural network (ANN) model was created to represent the correlation between TP concentration and the spectral bands of Moderate Resolution Imaging Spectroradiometer (MODIS) images in different research areas. We investigated the causal inference under the potential outcome framework to analyze the sensitivity of each band with regard to the TP concentration of different lakes for the research of water characteristics. Our results show that the accuracy of the ANN-based TP concentration estimation, with R2 > 0.73, root mean squared error (RMSE) < 0.037 mg/L in Lake Okeechobee and R2 > 0.73, RMSE < 4.1 μg/L in Lake Erie, respectively, is much higher than traditional empirical methods, e.g., linear regression. We found that the sensitive bands of TP concentration in Lake Erie are blue bands, whereas the sensitive bands in Lake Okeechobee are green bands. Various TP concentration maps were drawn to indicate the distribution of TP concentration and its tendency to change. The maps show that the distribution of TP concentration closely corresponds to the shore land-use, and a high TP concentration corresponds to the latest algal blooms breakout. Our proposed approach shows good potential for the remote-sensing estimation of TP concentration for inland lakes. Identifying the sensitive bands not only help characterize the lakes, but will also help the researchers to further observe the TP concentration of specific lakes in an efficient way.