The paper presents an empirical study concerning filters size adjustment in 3D Convolutional Neural Networks (CNN) for hyperspectral data classification. Based on Heisenberg's uncertainty principle, constraints on the depth-dependent relationships between spatial and spectral filter sizes have been defined. The benefits of CNNs satisfying these constraints have been evaluated for raw and transformed input data, in terms of optimization of both classification accuracy and training time. Experimental results show that the filter setting based on Heisenberg principle provides a significant reduction in training time and high classification accuracy. In particular, it can be a viable and feasible alternative to depth reduction in the case of transformed input data.
Multimodal remote sensing land cover classification is a significant challenging task. Current methods predominantly rely on deep semantic segmentation models; however, the lack of sufficient representative data and class imbalance among samples obstacles the application of these models on a large-scale area. To overcome these challenges, this paper proposes a semantic segmentation framework based on diffusion model-based data augmentation, leveraging existing classification maps to address data scarcity. The framework comprises four components: pseudo-label generation, multispectral image translation, segmentation model training, and post-processing. Experimental results on the Yangtze River Economic Belt dataset (MMSeg-YREB) from the 2024 IEEE WHISPERS Data Fusion Competition demonstrate the effectiveness of the proposed approach. Additionally, our method achieved second place in the 2024 IEEE WHISPERS competition.
Hyperspectral imaging in combination with microscopy can increase material discrimination possibilities with respect to regular microscopy imaging. We explore this discrimination potential to assess exposure to particle contamination. We focus on discriminating health relevant particles such as silica, in the respirable size fraction. To do so, multispectral imaging in combination with transmission microscopy is used for particle material identification. We use two compact snapshots near-infrared (NIR) cameras providing 16–25 spectral bands in the 600–960 nm range. The multispectral microscopy system has been tested for discrimination of silica among fourteen different particle materials. The analysis performed shows potential to accurately discriminate the 14 tested particle materials. In addition, the band relevance analysis shows that only a few specific bands are needed to provide accurate discrimination. The multispectral method presented could therefore enable a faster exposure assessment than traditional techniques for occupational exposure estimation.
Hyperspectral data, with its rich material information, offers significant advantages over RGB data, leading to an increased focus on hyperspectral object tracking. However, the limited availability of hyperspectral tracking datasets presents a major challenge, hindering the development of robust deep learning-based tracking algorithms. To address this issue, we propose a novel method called Low-Rank Adaptation-based Hyperspectral Tracking (LA-HST). Initially, we propose a spectral enhancement module with a low intrinsic dimension to enhance spectral details in images. Additionally, we introduce an adapted transformer encoder that effectively extracts hyperspectral features with rich spectral information. During training, we freeze the parameters of the pre-trained network and only train the parameters of the proposed modules, which account for just 0.5% of the total parameters. This approach allows the model to fully leverage the tracking capabilities of the pre-trained network while significantly enhancing training efficiency and improving generalization. Extensive experiments on the VIS, NIR, and RedNIR videos from the HOTC2024 dataset validate the effectiveness of LAHST, demonstrating its ability to deliver robust and efficient tracking performance across diverse spectral bands.
This paper analyses the challenge of using the same autoencoder model for lossy compression on different hyperspectral sensors and its impact on the complex application of camouflaged target detection. We propose an adaptive 1D convolutional autoencoder architecture for lossy hyperspectral data compression with the property of portability to unknown spectral signatures of different sensors. In our experiment, a model is pre-trained on PRISMA satellite data from around the world and fine-tuned with a small amount of target HySpex sensor data. The comparison model with the same architecture is trained from scratch with a larger amount of target sensor data. Compression rates of 4, 8, and 16 are implemented and evaluated. The evaluation discusses the reconstruction accuracy measured by the SAM, PSNR, and SSIM metrics. In addition, the reconstruction error is evaluated using the camouflaged target detection application. We show that fine-tuning the pretrained model with a small amount of data results in higher reconstruction accuracy than building the model from scratch with a larger amount of target data. This observation also applies to the target detection application.
