MFT-GAN: A Multiscale Feature-guided Transformer Network for Unsupervised Hyperspectral Pansharpening

Unsupervised learning, learning data distributions without needing labeled samples, is a particularly promising approach for solving the challenging task of hyperspectral pansharpening. Inspired by the above, we introduce an innovative generative adversarial network framework (named MFT-GAN) that incorporates a transformer network and multiscale interaction technology. Specifically, MFT-GAN is constructed of one generator and two discriminators. The generator is composed of a multiscale feature guidance branch (MFGB) and a feature interaction fusion branch (FIFB). The former aims at extracting spectral and spatial information branches from low-resolution hyperspectral (LRHS) and panchromatic (PAN) images by convolutional sampling operations, respectively. The latter accomplishes the interaction of spectral and spatial information at different scales through multiscale interaction technology. Then, the multiscale interaction results are fused by level-by-level convolutional sampling and pixel summation operations to generate high-resolution hyperspectral (HRHS) images. The discriminators are composed of a spectral transformer discriminator and a spatial transformer discriminator, designed to maintain structural and parametric balance and learn long-range spectral and spatial correlation relationships. In addition, hybrid loss functions are employed to complete the adversarial training of the MFT-GAN to improve its performance. The experiments on simulated and real datasets further demonstrated the effectiveness of the proposed MFT-GAN method.