Pixel-level annotation of hyperspectral images (HSIs) is expensive and time-consuming, making few-shot hyperspectral image classification (HSIC) a challenging task. Existing few-shot HSIC methods mainly exploit spatial–spectral representations, while frequency domain cues remain insufficiently explored. Moreover, conventional frequency domain methods usually rely on fixed transformations, limiting their flexibility in modeling complex intra-class variations under limited supervision. To address these limitations, this paper proposes S2FNet, a spatial–spectral–frequency learning framework that introduces learnable frequency prompts to adaptively modulate frequency representations for few-shot HSIC. First, a frequency domain prompt pool mechanism is designed to adaptively select relevant frequency sub-prompts to learn discriminative frequency representations. Second, a multi-scale frequency domain reconstruction module (MFDRM) is constructed to effectively integrate high-level semantic information and low-level spatial textures across frequency sub-bands of different scales. Finally, a dual-stream spectral-frequency enhancement module (DSFEM) is designed to model frequency responses along the spectral dimension and enhance informative spectral bands. Experimental results on various public datasets demonstrate that S2FNet achieves competitive classification accuracy and computational efficiency compared with existing advanced methods. The code will be available online at https://github.com/clay235/S2FNet.
Target detection tasks are now widely applied in the field of remote sensing. However, remote sensing target detection tasks are confronted with problems such as cluttered backgrounds and large scale variations. To address these issues, this paper proposes a high-precision aggregation head-auxiliary target detector (PHAS-YOLO). PHAS-YOLO includes two innovative plug-and-play modules: the spatial awareness attention module (SAAM) and the convolutional re-calibration multiscale feature fusion module (CRMSFF), as well as the context aggregation bidirectional connection structure (CABi-FPN) and the adaptive auxiliary head structure (AAHS). The proposed modules enable the model to have good spatial feature aggregation capabilities to retain key feature information, incorporate an adaptive weighting mechanism to reduce information loss caused by the fusion of different scales, and refine the features of the images to be detected. A series of experiments were conducted on three public remote sensing target detection datasets, namely DIOR, DOTAv1.0, and HRRSD, to verify the effectiveness and superiority of the proposed method in remote sensing target detection tasks.
In recent years, Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) have achieved significant progress in Hyperspectral Image (HSI) classification. However, in practical applications, the high cost of sample annotation and the limited availability of training samples lead to overfitting in CNNs and ViTs under few-shot learning scenarios. Siamese networks, as an effective metric learning method, show promising performance in few-shot learning due to their low dependency on sample information. However, traditional siamese networks rely on static parameter-sharing mechanisms, lack feature interaction between the two subnetworks, and struggle to effectively capture the spatial-spectral heterogeneity in hyperspectral data. Additionally, they are prone to noise interference, resulting in insufficient discriminative power of key features. To address these challenges, this paper proposes a Contextual Interaction Siamese Network for Few-Shot Hyperspectral Image Classification (CISNet). First, an Interactive Feature Fusion Module (IFFM) is introduced to capture the similarities and differences between features from the two subnetworks, thereby enhancing the discriminative power of key features. Second, an Enhanced Token Generation Module (ETGM) is designed to generate correlated class tokens for the two subnetworks. Finally, this paper innovatively proposes a Context Interaction Transformer Block (CITB) and a Guided Attention (GA) mechanism to strengthen global context interaction between the two subnetworks. Extensive experiments demonstrate that CISNet achieves superior performance under few-shot conditions and outperforms other state-of-the-art methods in classification accuracy.
Remote sensing image super-resolution (SR) aims to enhance image spatial resolution while preserving the accuracy of texture information. Recently, denoising diffusion probabilistic models have overcome issues such as excessive smoothing and modal collapse in generative models, demonstrating excellent performance in image super-resolution tasks. However, due to the diverse and complex nature of land cover types in remote sensing images, existing methods often exhibit performance fluctuations across different scenes, leading to generated details that do not align with ground truth. To address this issue, we propose a novel land-cover prior diffusion probabilistic model (LPDiffSR) specifically for remote sensing image super-resolution. We integrate the land cover modulation branch with the feature recovery branch into the diffusion land-cover aware block (DLCAB), enabling LPDiffSR to more effectively reconstruct the detailed features of various land covers. Then, diffusion transformer block (DTB) is incorporated in the encoder to produce more accurate and rich feature representation. In addition, we design a scalable attention-enhanced depthwise network (SAED-Net) that leverages multi-scale and attention mechanisms to explore potential priors from low-resolution inputs, thereby improving reconstruction performance. Rigorous quantitative and qualitative evaluations on the LoveDA and DLRSD datasets demonstrate that LPDiffSR significantly outperforms existing CNN, GAN, and Diffusion-based SR methods in terms of image generation quality and visual fidelity. Meanwhile, the experimental results in downstream vision tasks further validate the practicality and advantages of the SR images generated by LPDiffSR. The code will be available at GitHub https://github.com/whuzzzz/LPDiffSR.
Molecular property prediction is essential in diversified applications, as it helps identify molecules with the desired characteristics. However, the task often suffers from limited data, making the few-shot learning challenging. We introduce a Context-informed Few-shot Molecular Property Prediction via a Heterogeneous Meta-Learning approach, which employs graph neural networks combined with self-attention encoders to effectively extract and integrate both property-specific and property-shared molecular features, respectively. Based on the property-shared molecular features, we further infer molecular relations by using an adaptive relational learning module. The final molecular embedding is improved by aligning with the property label in the property-specific classifier. Furthermore, we employ a heterogeneous meta-learning strategy that updates parameters of the property-specific features within individual tasks in the inner loop and jointly updates all parameters in the outer loop. This enhances the model's ability to effectively capture both general and contextual information, leading to a substantial improvement in predictive accuracy. The model's performance was rigorously evaluated across various real molecular datasets, showcasing its superiority over current methods, especially in challenging few-shot learning scenarios.
In recent years, due to the unique spatial-spectral characteristics of hyperspectral images, they have played a crucial role in many fields. The effective extraction of features using deep neural networks, followed by the design of efficient and high-precision network algorithm structures, has gradually become a research hotspot. Hyperspectral images are difficult to obtain and have limited samples. Although hyperspectral image classification methods based on convolutional neural networks (CNN) have noticeably improved performance, there are still certain shortcomings in the extraction of detailed and local features. Therefore, how to fully utilize spatial and spectral information in situations with limited samples has become a challenging problem. To address this issue, inspired by the PID controller, this paper proposes a Progressive Multi-Scale Multi-Attention Fusion (PMMF) network structure that simultaneously extracts features from the Proportional (P), Integral (I), and Derivative (D) branches. The complementary responsibilities of the three branches address the issue of feature loss in details and improve the network’s learning efficiency across feature maps of different scales. By cleverly extracting features from different branches multiple times, the fusion of multi-scale features is achieved, avoiding the limitations of single-scale feature representation. The proposed multi-attention fusion module applies the most suitable attention mechanism according to the representation form of each branch, fully extracting features from each branch, enriching the information contained in the feature maps, and greatly enhancing the classification accuracy of hyperspectral images.
In the research of hyperspectral image classification, deep learning-based methods have achieved promising results. However, due to the high cost and difficulty in obtaining labeled hyperspectral image samples, traditional deep learning methods experience a significant performance decline under few-shot conditions. This paper proposes a deep few-shot learning method, referred to as Wavelet-Based Multi-stream Spatial-Frequency Fusion Network, to address the issue of small sample size in hyperspectral image classification. Through a multi-stream structure, features at various scales are obtained, effectively alleviating the issue of insufficient feature representation caused by limited training samples. These features are then fed into a multi-level wavelet transformation module integrated with a frequency-domain attention module. By decomposing the input into high-frequency and low-frequency components via wavelet transformation, and subsequently inputting these components into the frequency-domain attention module, a richer representation of features in the frequency domain is achieved. Finally, a Spatial-Frequency Fusion module is employed to not only extract abundant feature information but also fuse features from different scales. This process results in feature maps that encompass more comprehensive global feature information, thereby significantly enhancing classification accuracy under small sample conditions. The proposed method has been compared with traditional machine learning approaches and recent deep learning methods on the Houston, Dioni, and University of Pavia datasets. Under training conditions using only 1 to 5, 10, 15, and 20 samples per class, the experimental results demonstrate the superiority of the method proposed in this paper.
The core of hyperspectral image (HSI) classification lies in the effective fusion of spatial-spectral features. However, traditional methods are limited by the capacity of handcrafted feature representation, while deep learning methods face challenges such as overfitting with small sample sizes and high computational complexity. This article proposes a Mamba-driven multiscale spatial-spectral fusion network ((MSF)-S-2-F-2-Net). This network extracts spatial-spectral features at different granularities through the spatial-spectral multigranularity feature extraction module, adaptively enhances the spatial-spectral correlation through the spatial-spectral fusion attention module, optimizes feature fusion by combining local and global streams with the feature fusion enhanced vision transformer, and establishes long-sequence dependencies using the dual-path feature fusion mamba. The (MSF)-S-2-F-2-Net employs a multistage feature fusion strategy of "coarse fusion-fine optimization-strong screening" to achieve efficient classification with few samples. The network was validated on three publicly available HSI datasets to demonstrate its superiority in few-shot scenarios, with significant improvements in classification accuracy. It also exhibited remarkable classification performance across different numbers of training samples.
Hyperspectral images (HSI) have been extensively applied in a multitude of domains, due to their combined spatial and spectral characteristics along with a wealth of spectral bands. The ingenious combination of spatial and spectral information in HSI classification has remained a central research area for an extended period. In the classification process, it is essential to choose an expanded neighborhood window for learning. Nonetheless, employing an extensive window could lead to the problem of a lack of independence between the training dataset and the test dataset. Hence, this paper puts forward a three-branch spatial-spectral fusion network (TBSSF-Net) for HSI classification based on a smaller patch size. The network is composed of a spatial key details aggregation branch, a spatial semantic knowledge refinement branch, and a spectral band signal granularity branch. By employing the spatial branch, the network not only retains the key characteristics of details within the space but also captures the contextual relationships of global semantic information. The introduction of the spectral branch permits the combination of signal granularity at diverse levels, supplementing the performance of the spectral dimension. The TBSSF-Net has been validated for its superiority and effectiveness on four public HSI datasets. Additionally, it demonstrates significant classification performance across diverse amounts of training sets.
Forecast verification is very important in the nowcasting operation and technical development of strong convective weather. The current conventional verification method for nowcasting uses a binary classification event verification method, which exists with double punishment, leading to low scoring issues. In order to make up for the shortcomings of conventional verification methods and explore the potential value of forecasting, based on the characteristics and requirements of strong convective weather nowcasting operations, this paper proposes a neighborhood verification method that considers spatial scale, time scale, and intensity error information simultaneously, based on the spatial neighborhood fraction skill score (FSS) verification method. The paper designs an intensity neighborhood correction scheme, a time neighborhood correction scheme, and a comprehensive correction scheme that considers both intensity and time neighborhoods. By introducing spatial neighborhood probability, the strict spatial matching requirement is weakened. Based on the Gaussian membership function in fuzzy logic, the value of forecasting grid below the threshold is explored. Time neighborhood adaptive weighting strategy is adopted to solve the problem of early and delayed forecasts. The evaluation and verification of the optical flow radar extrapolation nowcasting results show that compared to the traditional "point-to-point" verification indicators, the proposed indicators can increase the tolerance of verification, provide more useful information, and are more in line with the needs of practical application.
Accurate and timely pest and disease monitoring during the cultivation process of traditional Chinese medicinal materials is crucial for ensuring optimal growth, increased yield, and enhanced content of effective components. This paper focuses on the essential requirements for pest and disease monitoring in a planting base of Cinnamomum Camphora var. Borneol (CCB) and presents a solution using unmanned aerial vehicle (UAV) images to address the limitations of real-time and on-site inspections. In contrast to existing solutions that rely on advanced sensors like multispectral or hyperspectral sensors mounted on UAVs, this paper utilizes visible light sensors directly. It introduces an ensemble learning approach for pest and disease monitoring of CCB trees based on RGB-derived vegetation indices and a combination of various machine learning algorithms. By leveraging the feature extraction capabilities of multiple algorithms such as RF, SVM, KNN, GBDT, XGBoost, GNB, and ELM, and incorporating morphological filtering post-processing and genetic algorithms to assign weights to each classifier for optimal weight combination, a novel ensemble learning strategy is proposed to significantly enhance the accuracy of pest and disease monitoring of CCB trees. Experimental results validate that the proposed method can achieve precise pest and disease monitoring with reduced training samples, exhibiting high generalization ability. It enables large-scale pest and disease monitoring at a low cost and high precision, thereby contributing to improved precision in the cultivation management of traditional Chinese medicinal materials.
Under the background of climate change and global warming, extreme drought events in China are becoming increasingly frequent. Drought is one of the primary natural causes of damage to China's agriculture, economy, and environment, making timely, accurate, and high-resolution drought monitoring particularly crucial. The global standardized precipitation - evapotranspiration index database (SPEIbase) is a widely accepted and used global-scale drought monitoring product. However, limited by its spatial resolution of 0.5 degrees, it is difficult to describe the local spatio-temporal structure of drought. How to improve its spatial resolution while maintaining spatio-temporal consistency is one of the current research hotspots. Based on the response of vegetation growth status to drought, this paper proposes a simple and feasible SPEI prediction method, which improves the resolution of SPEIbase from 0.5 degrees to 1 km. Sixteen remote sensing inversion indices, reflectance and elevation data related to drought were selected from Google Earth Engine (GEE) as features. After preprocessing such as gridding and sample balancing, a random forest regression model was constructed to achieve high spatial resolution prediction of SPEI. SPEI with time scales of 1, 3, 6, 9, 12 and 24 months in July 2020, August 2019 and August 2018 in China was selected for experiments. The accuracy of 1 km resolution SPEI was evaluated through metrics such as root mean square error (RMSE), Pearson correlation coefficient (PCC) and determination coefficient (R2). At the same time, it was compared with the existing 1 km resolution SPEI dataset and the site-scale SPEI values. The results show that the method in this paper can obtain accurate prediction results more stably. The PCC and R2 of different months and multiple time scales are all higher than 0.9 and 0.8, and the RMSE is lower than 0.4, showing a good application prospect. Despite the good consistency between the Proposed SPEI and SPEIbase with the site-scale SPEI values, there is still significant room for improvement.
Ensemble forecasting is crucial for improving weather predictions, especially for forecasts of extreme events. Constructing an ensemble prediction system (EPS) based on conventional NWP models is highly computationally expensive. ML models have emerged as valuable tools for deterministic weather forecasts, providing forecasts with significantly reduced computational requirements and even surpassing the forecast performance of traditional NWP models. However, challenges arise when applying ML models to ensemble forecasting. Recent ML models, such as GenCast and SEEDS model, rely on the ERA5 EDA or operational NWP ensemble members for forecast generation. Their spatial resolution is also considered too coarse for many applications. To overcome these limitations, we introduce FuXi-ENS, an advanced ML model designed to deliver 6-hourly global ensemble weather forecasts up to 15 days. This model runs at a significantly increased spatial resolution of 0.25°, incorporating 5 atmospheric variables at 13 pressure levels, along with 13 surface variables. By leveraging the inherent probabilistic nature of Variational AutoEncoder (VAE), FuXi-ENS optimizes a loss function that combines the CRPS and the KL divergence between the predicted and target distribution, facilitating the incorporation of flow-dependent perturbations in both initial conditions and forecast. This innovative approach makes FuXi-ENS an advancement over the traditional ones that use L1 loss combined with the KL loss in standard VAE models for ensemble weather forecasting. Results demonstrate that FuXi-ENS outperforms ensemble forecasts from the ECMWF, a world leading NWP model, in the CRPS of 98.1 360 variable and forecast lead time combinations. This achievement underscores the potential of the FuXi-ENS model to enhance ensemble weather forecasts, offering a promising direction for further development in this field.
Machine learning (ML) models have become increasingly valuable in weather forecasting, providing forecasts that not only lower computational costs but often match or exceed the accuracy of traditional numerical weather prediction (NWP) models. Despite their potential, ML models typically suffer from limitations such as coarse temporal resolution, typically 6 hours, and a limited set of meteorological variables, limiting their practical applicability. To overcome these challenges, we introduce FuXi-2.0, an advanced ML model that delivers 1-hourly global weather forecasts and includes a comprehensive set of essential meteorological variables, thereby expanding its utility across various sectors like wind and solar energy, aviation, and marine shipping. Our study conducts comparative analyses between ML-based 1-hourly forecasts and those from the high-resolution forecast (HRES) of the European Centre for Medium-Range Weather Forecasts (ECMWF) for various practical scenarios. The results demonstrate that FuXi-2.0 consistently outperforms ECMWF HRES in forecasting key meteorological variables relevant to these sectors. In particular, FuXi-2.0 shows superior performance in wind power forecasting compared to ECMWF HRES, further validating its efficacy as a reliable tool for scenarios demanding precise weather forecasts. Additionally, FuXi-2.0 also integrates both atmospheric and oceanic components, representing a significant step forward in the development of coupled atmospheric-ocean models. Further comparative analyses reveal that FuXi-2.0 provides more accurate forecasts of tropical cyclone intensity than its predecessor, FuXi-1.0, suggesting that there are benefits of an atmosphere-ocean coupled model over atmosphere-only models.
IntroductionThe Cinnamomum Camphora var. Borneol (CCB) tree is a valuable timber species with significant medicinal importance, widely cultivated in mountainous areas but susceptible to pests and diseases, making manual surveillance costly.MethodsThis paper proposes a method for detecting CCB pests and diseases using Unmanned aerial vehicle (UAV) as an advanced data collection carrier, capable of gathering large-scale data. To tackle the high cost and challenging data processing issues associated with traditional hyper-spectral/multi-spectral sensors, this method only relies on UAV visible light RGB bands. The process first involves calculating and normalizing 24 visible light vegetation indices from the UAV RGB images of the monitoring area, along with the original RGB bands. To account for the collinearity relationship between indices, the random forest variable importance and correlation coefficient iterative analysis algorithm are employed to select indices, retaining the most important or lowest collinearity multiple vegetation indices. Subsequently, the Beluga Whale Optimization (BWO) algorithm is utilized to generate a new vegetation index, which is then combined with the multi-threshold segmentation method to propose a BWO-weighted ensemble strategy for obtaining the final pests and diseases detection results.Results and discussionThe experimental results suggest that the new BWO-based vegetation index has a higher feature expression ability than single indices, and the new BWO-based ensemble strategy can yield more accurate detection results. This approach provides an effective means for low-cost pests and diseases detection of CCB trees.
设计了一种适用于大区域、小重叠卫星影像有理多项式模型(RPC)定位精度优化的技术流程,基于开源数字正射影像(DOM)、数字高程模型(DEM)数据和RPC几何模型约束,通过优化尺度不变特征变换(SIFT)特征提取与匹配算法,实现了无像幅约束的控制点和连接点的快速稳健匹配,应用DEM支持下的RPC模型二维区域网方法实现了多景影像RPC模型精度的联合精化.GF-1卫星WFV影像的实验结果表明:在DEM支持下经过区域网平差后,RPC模型的定位精度可达到像素级水平.
Dongping granite in the northwest margin of Qaidam Basin is of great value in regional geological evolution research and resource development. Therefore, the geochemical characteristics and zircon U-Pb chronology of Dongping granite are analyzed. Dongping granite can be divided into two types. The first type is Adakitic granite with high Sr and low Yb, which has the geochemical characteristics of Adakitic rock. There are garnet, rutile and amphibolite residues in the source area, and no or only a small amount of plagioclase residue. It is speculated that the rock may be formed by partial melting of rutile-bearing eclogite in the thickened lower crust, with melting depth greater than 50 km. The other type is Himalayan type granite with low Sr and low Yb, and the residual facies in the source area are garnet, amphibolite and plagioclase, which may be formed by partial melting of granulite in the thickened lower crust, and the melting depth is about 40-50 km. LA-ICP-MS zircon U-Pb dating results show that the crystallization age of the two types of granites is about 418 Ma, which is the result of the same magmatic activity, and the tectonic discrimination diagram shows that both types of granites belong to the same collision granite. Based on previous research results, this paper believes that at about 418Ma, the “Southern Altyn Ocean” has been closed, and turned to the stage of intracontinental subduction collision orogeny, resulting in the mutual overlap and thickening of the crust. As the thickened lower crust is heated by the underplating of mantle materials, partial melting occurs at different depths, forming two different types of granites in Dongping. The origin and content of helium in Dongping and its adjacent Jianbei and Niudong gas fields are closely related to the basement granite. Specifically, helium is mainly produced by the decay of U, Th and other radioactive elements in the basement granite and granite gneiss. The longer the basement granitoid body is formed, the higher the helium content in the gas field. Under the background of collision orogenic structure, the granite formed by partial melting of pure crustal materials has higher U and Th contents and stronger helium generation potential.
The unique spatial–spectral integration characteristics of hyperspectral imagery (HSI) make it widely applicable in many fields. The spatial–spectral feature fusion-based HSI classification has always been a research hotspot. Typically, classification methods based on spatial–spectral features will select larger neighborhood windows to extract more spatial features for classification. However, this approach can also lead to the problem of non-independent training and testing sets to a certain extent. This paper proposes a spatial shuffle strategy that selects a smaller neighborhood window and randomly shuffles the pixels within the window. This strategy simulates the potential patterns of the pixel distribution in the real world as much as possible. Then, the samples of a three-dimensional HSI cube is transformed into two-dimensional images. Training with a simple CNN model that is not optimized for architecture can still achieve very high classification accuracy, indicating that the proposed method of this paper has considerable performance-improvement potential. The experimental results also indicate that the smaller neighborhood windows can achieve the same, or even better, classification performance compared to larger neighborhood windows.
Hyperspectral image (HSI) classification, due to its characteristic combination of images and spectra, has important applications in various fields through pixel-level image classification. The fusion of spatial–spectral features is a topic of great interest in the context of hyperspectral image classification, which typically requires selecting a larger spatial neighborhood window, potentially leading to overlaps between training and testing samples. Vision Transformer (ViTs), with their powerful global modeling abilities, have had a significant impact in the field of computer vision through various variants. In this study, an ensemble learning framework for HSI classification is proposed by integrating multiple variants of ViTs, achieving high-precision pixel-level classification. Firstly, the spatial shuffle operation was introduced to preprocess the training samples for HSI classification. By randomly shuffling operations using smaller spatial neighborhood windows, a greater potential spatial distribution of pixels can be described. Then, the training samples were transformed from a 3D cube to a 2D image, and a learning framework was built by integrating seven ViT variants. Finally, a two-level ensemble strategy was employed to achieve pixel-level classification based on the results of multiple ViT variants. Our experimental results demonstrate that the proposed ensemble learning framework achieves stable and significantly high classification accuracy on multiple publicly available HSI datasets. The proposed method also shows notable classification performance with varying numbers of training samples. Moreover, herein, it is proven that the spatial shuffle operation plays a crucial role in improving classification accuracy. By introducing superior individual classifiers, the proposed ensemble framework is expected to achieve even better classification performance.
Hyperspectral unmixing, which decomposes mixed pixels into the endmembers and corresponding abundances, is an important image process for the further application of hyperspectral images (HSIs). Lately, the unmixing problem has been solved using deep learning techniques, particularly autoencoders (AEs). However, the majority of them are based on the simple linear mixing model (LMM), which disregards the spectral variability of endmembers in different pixels. In this article, we present a multi-attention AE network (MAAENet) based on the extended LMM to address the issue of the spectral variability problem in real scenes. Moreover, the majority of AE networks ignore the global spatial information in HSIs and operate pixel- or patch-wise. We employ attention mechanisms to design a spatial–spectral attention (SSA) module that can deal with the band redundancy in HSIs and extract global spatial features through spectral correlation. Moreover, noticing that the mixed pixels are always present in the intersection of different materials, a novel sparse constraint based on spatial homogeneity is designed to constrain the abundance and abstract local spatial features. Ablation experiments are conducted to verify the effectiveness of the proposed AE structure, SSA module, and sparse constraint. The proposed method is compared with several state-of-the-art unmixing methods and exhibits competitiveness on both synthetic and real datasets.