Hyperspectral image (HSI) classification has emerged as a critical task in remote sensing. However, the inherent high spectral dimensionality, limited labeled data, and complex spatial structures in HSIs present substantial challenges to traditional and even deep learning-based approaches. Traditional convolutional neural networks (CNNs) often struggle to capture the irregular, non-Euclidean spatial relationships among pixels, whereas standard Transformer architectures are limited in modeling localized spectral-spatial dependencies and are prone to overfitting in scenarios with limited training data. In this article, we propose a novel Dual-Scale Graph Transformer with Spectral-Spatial Convolution framework that integrates multiscale structural modeling and deep spectral feature extraction in a unified architecture. Specifically, we construct two complementary graph representations: 1) a pixel-level graph that captures fine-grained spatial relationships; and 2) a super-pixel level graph that encodes broader semantic structures. These dual-scale graphs are processed by a Graph Transformer module that incorporates node features and enables effective modeling of long-range dependencies and context-aware feature aggregation. Furthermore, we introduce a dedicated Spectral-Spatial Convolution module to enhance spectral feature discrimination by maintaining local spatial continuity. Extensive experiments on four benchmark hyperspectral datasets demonstrate that our method achieves state-of-the-art classification performance. Moreover, compared with existing Transformer-based models, our framework significantly reduces model complexity and computational cost, while maintaining superior performance.

Hyperspectral unmixing has attracted increasing attention in remote sensing applications. Unfortunately, significant unmixing residuals often arise from the coupled nonlinear mixing effects and spectral variability (SV), bringing challenges for reliably solving the underlying optimization problems in practical applications. Although deep autoencoder (AE) architectures have shown advantages in learning latent unmixing features from hyperspectral data, their performance in capturing accurate spectral-spatial information and interpreting both nonlinearity and SV remains limited. This article proposes a novel AE-based unmixing method that introduces a spectral-spatial attention mechanism to learn refined global-to-local semantic features of land covers. Leveraging the features, the model enables efficient parametric learning of nonlinearity and SV through the integration of physically interpretable second-order scatterings and scaled SV factors. Experimental results indicate that the proposed method has superior unmixing performance compared to the state-of-the-art methods.

Hawthorn (Crataegus pinnatifida) is a commonly consumed medicinal fruit. This study proposes a rapid and nondestructive technique that integrates hyperspectral imaging (HSI) with interpretable deep learning for the classification of hawthorn cultivars from different regions and the quantitative prediction of key quality indicators, including citric acid, total sugar, and vitamin C content. A total of 1227 samples were collected from 11 categories, representing different cultivars and origins. Model robustness was ensured by acquiring HSI data in three different orientations, with the stalk positioned horizontally, up, and down. Classification results showed an EfficientNet model achieved the highest accuracy (95.92 %) by fusing spectral data from all three orientations. In the regressions, the EfficientNet model outperformed both PLSR and CNN. Among the measured compounds, citric acid and total sugar yielded satisfactory results, with R2 values of 0.94 and 0.92 and RPD values of 4.08 and 3.55, respectively. Furthermore, combining gradient-weighted class activation mapping (Grad-CAM) and Shapley additive explanations (SHAP) enabled a visual and quantitative interpretation of spectral feature contributions, effectively addressing black-box issues. This is the first study to integrate HSI with interpretable deep learning for simultaneous classification and quality prediction in hawthorn, with improved model performance through multi-orientation spectral fusion.

In recent years, a series of bilinear mixture models (BMMs) have been well studied, based on which nonlinear unmixing algorithms for hyperspectral images have been proposed. However, the performance of unsupervised nonlinear spectral unmixing can be largely affected by the collinearity and the nonconvex unmixing problem's inherent complexity. To solve the problems, this paper proposes a geometrical projection improved multi-objective particle swarm optimization (GPMOPSO) method for nonlinear unmixing, which consists of two dynamically updating procedures. First, under the assumption of the BMMs, observed pixels are geometrically projected to their approximate linear mixture components to alleviate the collinearity. Second, the linear mixture components are efficiently decomposed to refined endmembers and abundances in an improved linear unmixing framework using multi-objective particle swarm optimization. Two advanced multi-objective optimization strategies are adopted to balance the influence of an endmember distance constraint and an abundance sparsity constraint on the achieved unsupervised linear unmixing process, respectively. Then, accurate unmixing variables can be fed back to the geometrical projection procedure to produce more reliable linear mixture components which facilitate the execution of the former in turn. Experimental results of both simulated data and real hyperspectral remote sensing data verify the superior nonlinear unmixing performance of the proposed method.

This letter presents an unsupervised unmixing method to interpret both nonlinear mixing effects and spectral variability (SV). First, the traditional multilinear mixing model (LMM) is augmented by introducing physically meaningful factors representing wavelength-dependent SV into the modeling. This model-driven improvement effectively provides a concise numerical explanation for the nonlinearity and SV, facilitating the consideration of their coupled effects on unmixing. Second, based on the augmented model, total variation (TV) regularizers to improve abundances' spatial smoothness and scaling factors' local similarity, and a constraint to confine the perturbations' energy of SV, are exploited to formulate the unmixing problem. A multiswarm particle swarm optimization (PSO) algorithm is employed as the solver for this problem to achieve robust unmixing results with higher accuracy. Finally, experiments on numerical model-based and physical-based simulated data and real hyperspectral remote sensing images demonstrate the proposed method's superiority over the state-of-the-art methods.

Deep autoencoder networks’ flexibility and advantages in learning hidden features have made them popular in unmixing hyperspectral images recently. Based on specific network structures, the effective learning of spatial-spectral features for unmixing is always crucial. This paper presents a novel unmixing method using spatial-spectral twin autoencoders. One autoencoder generates coarse abundances with strong local spatial similarity for pixels in hyperedges defined in homogeneous superpixels. The other autoencoder with identical modules further adopts the coarse abundances to build a spatial resemblance constraint in the loss function to produce refined abundances. Experiments with simulated and real hyperspectral remote sensing data have confirmed the validity of the proposed method.

To mitigate the impact of mixed pixels in hyperspectral images (HSIs), substantial progress has been made in both model- and deep-learning-based unmixing methods. However, the issues, such as complex computational processes and limited interpretability, hinder the improvement of their unmixing performance. Particularly, unsupervised nonlinear hyperspectral unmixing (HU) remains a great challenge. In this article, we propose an extended multilinear mixing (EMLM) model-inspired dual-stream network for unsupervised nonlinear HU. First, the alternating direction method of multipliers (ADMM) algorithm for the EMLM-based unmixing problem is unfolded to construct an encoder network. Subsequently, it is connected to a decoder network derived from the EMLM, creating an autoencoder (AE)-like network architecture. Second, the original HSIs and superpixel-averaging-based coarse HSIs are input into two network branches with identical architectures, respectively, to build a novel weight-sharing dual-stream network. Furthermore, the estimates of abundances and nonlinear parameters obtained from the two branches are utilized to formulate local spatial similarity regularizers, enhancing the network’s loss function and effectively improving unmixing accuracy. Finally, experiments conducted on the laboratory-created dataset and real-world datasets validate that the proposed method exhibits superior unmixing performance compared to the state-of-the-art methods. In addition, our code is available at: https://github.com/I3ab/EMLM-Net .

Unsupervised band selection identifies informative bands in hyperspectral images (HSIs) without prior labeling, reducing spectral redundancy. Besides spectral information, the spatial-spectral structures of HSIs can be exploited jointly to select more valuable bands and reduce the impact of noises. In this article, we present a novel spatial-spectral hypergraph-based unsupervised band selection (SSHUBS) method. First, since hypergraphs are effective in expressing complex high-order relations among pixels and bands, a spatial hypergraph is built using pixels within local spatial homogeneous regions, and a spectral hypergraph is built using bands in clusters generated by an over-clustering strategy. The two hypergraphs could embed the HSIs' spatial and spectral information into the band selection process, respectively. Second, two normalized hypergraph Laplacian matrices are generated to reformulate the optimization problem of the classical sparse self-representation (SR) band selection framework. Combining the obtained coefficient matrix with the cluster sizes to rank each spectral band, representative bands are selected. Finally, experiments conducted on hyperspectral remote sensing data verify the effectiveness of the proposed method in selecting bands to improve the classification accuracy compared to state-of-the-art methods.

Recently, the research on nonlinear unmixing for hyperspectral images (HSIs) has received more and more attention. However, unsupervised nonlinear unmixing methods that jointly estimate endmembers and abundances from HSIs are insufficiently studied. Besides, the reasonable description of the wavelength-dependent nonlinear intensity and the effective utilization of the spectral and spatial information of HSIs remain to be improved. Based on an extended multilinear mixing (EMLM) model, a coarse-to-fine scheme is proposed for unsupervised nonlinear hyperspectral unmixing to address the above issues. Coarse HSIs generated based on the superpixel segmentation are unmixed first, and then, fine unmixing on the original HSIs is achieved with the guidance of the coarse unmixing results. The endmembers extracted by the coarse unmixing are used to update the endmembers in the fine unmixing, and the coarse abundances and nonlinear parameters are integrated into regularizers. To be specific, a weighted sparse regularizer of abundances and a weighted graph regularizer of nonlinear parameters are constructed and incorporated into the objective function. In this way, some priors can be well modeled and exploited, including that the neighboring pixels share similar sparsity patterns in the abundances and show the consistency in correlations between different bands of the nonlinear parameters. Finally, the alternative optimization strategy (AOS) and the alternating direction method of multipliers (ADMM) are applied to derive the algorithm. Experimental results on the synthetic, laboratory-created, and real hyperspectral data demonstrate that the proposed method outperforms the state-of-the-art nonlinear unmixing methods.

Objective: To analyze the clinical manifestations and genetic examination results of affected members of two Chinese ATP1A2 gene variants pedigrees, and to summarize their phenotypic and genotypic features. Methods: The clinical features were recorded in detailed. The cranial magnetic resonance imaging for patients and gene sequencing of two Chinese ATP1A2 gene variant pedigrees were perform. The corre-lation between the types of variants and the clinical phenotypes of ATP1A2 gene were analyzed. Results: These two pedigrees are diagnosed as familial hemiplegic migraine type 2 (FHM2) with ATP1A2 heterozygous missense variants, c.1091C > T (p.T364M) found in pedigree 1 and c.899T > C (p.L300P) in pedigree 2. Multiple phenotypes coexist in both families, and the two probands have severe cranial magnetic resonance imaging manifesting hemiplegic contralateral cortical swelling and diffusion weighted imaging hyperintense signal, which can be fully recovered. ATP1A2 gene variants were seen in FHM2, sporadic hemiplegic migraine and atypical alternating hemiplegia of childhood (AHC) families or sporadic cases, etc. The clinical features of ATP1A2 variant c.1091C > T (p.T364M) are basically similar in Chinese patients and European patients. Conclusion: These two Chinese pedigrees had FHM2 due to ATP1A2 heterozygous missense variation. It would expand the understanding of ATP1A2.& COPY; 2023 Published by Elsevier Ltd on behalf of Tsinghua University Press. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

Hyperspectral unmixing has received increasing attention as a technique for estimating endmember spectra and fractional abundances of land covers. Encoding high-dimensional hyperspectral data into a low-dimensional latent space to generate reasonable abundances, autoencoder (AE) has shown its great potential and attractive advantages in spectral unmixing. AEs decode abundances back to spectra, which can effectively reflect the general spectral mixing process. However, most existing AE-based unmixing methods often do not fully exploit the spatial information of hyperspectral images, hindering the improvement of unmixing accuracy. This article proposes a graph attention convolutional autoencoder architecture for hyperspectral unmixing. By incorporating graph attention convolution into AE, the proposed method performs better in leveraging both long-range and short-range spatial information of hyperspectral images. Accurate abundances with global and local spatial consistency can be efficiently learned by the network. Moreover, the decoder is further improved based on the postpolynomial nonlinear mixing model to make the network have stronger physical interpretability to deal with the issue of nonlinear blind unmixing. Experimental results indicate that the proposed method has good unmixing performance. It can reduce the root mean squared error of estimated abundances for synthetic data by over 10% compared to other methods. In experiments with real hyperspectral data, the difference between its unmixing results' accuracy and the best is less than 5%.

Deep learning(DL)has shown its superior performance in dealing with various computer vision tasks in recent years.As a simple and effective DL model,autoencoder(AE)is popularly used to decompose hyperspectral images(HSIs)due to its powerful ability of feature extraction and data reconstruction.However,most existing AE-based unmixing algorithms usually ignore the spatial information of HSIs.To solve this problem,a hypergraph regularized deep autoencoder(HGAE)is proposed for unmixing.Firstly,the traditional AE architecture is specifically improved as an unsupervised unmixing framework.Secondly,hypergraph learning is employed to reformulate the loss function,which facilitates the expression of high-order similarity among locally neighboring pixels and promotes the consistency of their abundances.Moreover,L1/2 norm is further used to enhance abundances sparsity.Finally,the experiments on simulated data,real hyperspectral remote sensing images,and textile cloth images are used to verify that the proposed method can perform better than several state-of-the-art unmixing algorithms.

In hyperspectral unmixing, dealing with nonlinear mixing effects and spectral variability (SV) is a significant challenge. Traditional linear unmixing can be seriously deteriorated by the coupled residuals of nonlinearity and SV in remote sensing scenarios. For the simplification of calculation, current unmixing studies usually separate the consideration of nonlinearity and SV. As a result, errors individually caused by the nonlinearity or SV still persist, potentially leading to overfitting and the decreased accuracy of estimated endmembers and abundances. In this paper, a novel unsupervised nonlinear unmixing method accounting for SV is proposed. First, an improved Fisher transformation scheme is constructed by combining an abundance-driven dynamic classification strategy with superpixel segmentation. It can enlarge the differences between different types of pixels and reduce the differences between pixels corresponding to the same class, thereby reducing the influence of SV. Besides, spectral similarity can be well maintained in local homogeneous regions. Second, the polynomial postnonlinear model is employed to represent observed pixels and explain nonlinear components. Regularized by a Fisher transformation operator and abundances’ spatial smoothness, data reconstruction errors in the original spectral space and the transformed space are weighed to derive the unmixing problem. Finally, this problem is solved by a dimensional division-based particle swarm optimization algorithm to produce accurate unmixing results. Extensive experiments on synthetic and real hyperspectral remote sensing data demonstrate the superiority of the proposed method in comparison with state-of-the-art approaches.

Plant density is a significant variable in crop growth. Plant density estimation by combining unmanned aerial vehicles (UAVs) and deep learning algorithms is a well-established procedure. However, flight companies for wheat density estimation are typically executed at early development stages. Further exploration is required to estimate the wheat plant density after the tillering stage, which is crucial to the following growth stages. This study proposed a plant density estimation model, DeNet, for highly accurate wheat plant density estimation after tillering. The validation results presented that (1) the DeNet with global-scale attention is superior in plant density estimation, outperforming the typical deep learning models of SegNet and U-Net; (2) the sigma value at 16 is optimal to generate heatmaps for the plant density estimation model; (3) the normalized inverse distance weighted technique is robust to assembling heatmaps. The model test on field-sampled datasets revealed that the model was feasible to estimate the plant density in the field, wherein a higher density level or lower zenith angle would degrade the model performance. This study demonstrates the potential of deep learning algorithms to capture plant density from high-resolution UAV imageries for wheat plants including tillers.

As a representative structural property, the sparsity of ground covers' distribution in hyperspectral images (HSIs) has been extensively applied to improve spectral unmixing in years. It is worth leveraging the close relationship between the sparsity of abundances and the spatial information of HSIs to obtain more reasonable sparse unmixing results. In this letter, a novel multilevel reweighted sparse unmixing method using superpixel segmentation and particle swarm optimization (MRSUPSO) is proposed. Three sparse reweighted factors are finely designed at different local and global spatial levels. The first two local reweighted factors are constructed according to the sparseness and the low-rank property of pixels' abundances in the generated superpixels. The third global reweighted factor is given by considering the change of the sparseness of each material abundance map in the entire HSI. Then, a new sparse constraint is imposed, which can effectively facilitate the correct expression of abundances' sparsity during unmixing. Moreover, PSO based on double swarms with dimension division is employed to solve the unmixing problem and enhance the unmixing robustness. Experimental results of both simulated and real hyperspectral data validate that the proposed method can produce accurate unsupervised sparse unmixing results.

The presence of shadows has always been a troublesome problem in image processing and can also affect spectral unmixing with hyperspectral remote sensing images. Traditional unmixing algorithms regard shadows as a special type of ground cover, so they can only estimate real materials' sunlit abundances, and materials with low reflectance may be wrongly recognized as shadows. Without regarding shadows as ground cover, we propose a supervised nonlinear unmixing method to accurately estimate real materials' total abundances inside and outside shadow areas. First, sunlit and shadowed constituents in every pixel of an image are modeled explicitly and integrated by a tractable bilinear mixing mechanism. Second, based on the constructed model, the strong sparsity of shadow spatial distribution and the spatial correlation among neighboring material abundances are exploited to produce a constrained optimization problem for nonlinear unmixing. Third, an existing unmixing framework based on particle swarm optimization is extended to calculate unknown variables of the optimization problem. Three alternatingly updated swarms using improved dimensional division strategies are designed to accordingly address the unmixing subproblems with respect to variables to be estimated during the solution search. This process has the potential to be generalized to solve complex nonlinear unmixing optimization problems. Finally, model-based simulated data, virtual citrus orchard data, and real hyperspectral remote sensing images are used in experiments to evaluate the proposed method and compare it with traditional and state-of-the-art nonlinear unmixing algorithms. Experimental results verify that the proposed method can achieve acceptable unmixing performance when managing nonlinear mixing effects and shadows.

Due to noisy acquisition and atmospheric effects, some spectral bands in hyperspectral images (HSIs) suffer from low signal-to-noise ratios, thus requiring robust techniques to tackle unmixing problems. Besides, integrating the spatial information of HSIs into the nonlinear unmixing framework remains a challenge. To cope with the above problems, first, the $\ell_{2.1}$ norm-based objective function is adopted to suppress the influence of noisy bands. Furthermore, to fully exploit the spatial-spectral information of HSIs, a reweighted collaborative sparse regularizer imposed on the abundances enforces that the pixels in a superpixel-based neighborhood share the same set of endmembers and have similar abundances, and a reweighted spectral total variation regularizer is employed to enhance the spatial-spectral smoothness of the nonlinear parameters. Extensive experiments conducted on the simulated and real datasets verify the superiority of the proposed algorithm over other state-of-the-art ones.

Due to the presence of multiple scatterings, linear unmixing methods may not perform well in practical applications, and thus nonlinear unmixing has become an urgent problem to be solved. Usually, the mixing process in the observed scenarios is physically based, and many well-designed models have been proposed to interpret it. Recently, kernel-based nonlinear unmixing methods have been popularly studied to achieve a model-free and flexible representation of the nonlinearity. However, the existing kernel-based methods are mainly data-driven, which could make them fail to match the real physical mixing mechanism and result in the occurrence of overfitting. In this article, a kernel-based bilinear unmixing (KBU) method was proposed to transform the classic bilinear mixing models into their equivalent kernel forms that are more general and effective in expressing second-order scatterings. Two specific types of kernel transformations were designed, and the alternating direction method of multipliers (ADMM) was used to solve the kernel-transformed model-based optimization problem for unmixing. Moreover, the spatial prior was exploited to further improve the unmixing accuracy, and here we employ the total variation (TV) regularization as a paradigm. Experiments on synthetic data sets, physics-based simulated data sets, and real data were conducted to evaluate the algorithms. It is validated that our methods have better performance in abundance estimation and nonlinear reconstruction compared with other nonlinear unmixing methods.

Unmanned Aerial Vehicle (UAV) borne hyperspectral remote sensing system can provide high spatial and high spectral resolution dataset for precise crops classification in a flexible and effective way. However, the spatial heterogeneity and spectral variability of these images bring serious challenges for precise crops classification. The deep learning (DL) methodologies show great advantages and have demonstrate promising results in this field. The objective of this study is to evaluate the impact of spatial and spectral resolution on classification accuracy with DL-based methodologies. Using the public benchmark dataset WHU-Hi, the results show that: excessive higher or coarser spatial resolution is unnecessary, and 0.5-1.0 m spatial resolution is appropriate for precise crop classification; reducing the number of bands slightly changes classification results but significantly reducing computations; there are distinct performances among selected DL-based methods, and the SSRN demonstrate the most stable and highest accuracies under different degrading simulations.