Light field data hold great potential in immersive media, virtual reality, and 3D imaging. However, their inherent high-dimensional characteristics result in huge data volumes, posing significant challenges for efficient storage and real-time transmission. To address this, this paper proposes a Implicit Neural Representation with State Space Model for Light Field Image Compression. By leveraging the advantages of state space models in long-sequence modeling, the proposed method employs a selective scanning mechanism to effectively capture long-range dependencies in light fields, thereby enhancing the modeling capacity of implicit representations. Moreover, to further improve reconstruction quality, a global attention convolutional residual block is introduced in the decoding stage, which enhances the model's representational power while maintaining computational efficiency. Additionally, a Temporal-aware Affine Transform residual block is incorporated to harness the potential of feature alignment. Experiments conducted on the EPFL light field dataset and HCI light field dataset demonstrate that the proposed method is competitive with existing NeRV-based compression approaches in terms of both rate-distortion performance and subjective visual quality. On the EPFL and HCI light field datasets, this method delivers average BD-rate (PSNR) savings of 4.74% and 8.62% over NeRV++.
Hyperspectral image (HSI) classification has emerged as a critical area of research in remote sensing, computer vision, and pattern recognition due to its ability to capture rich spectral-spatial information. This review systematically explores the evolution of HSI classification techniques, from traditional handcrafted feature extraction methods to advanced deep learning models. Initially, handcrafted approaches such as texture analysis, spectral indices, and feature selection methods were employed to extract meaningful patterns from high-dimensional hyperspectral data. These methods, however, struggled with scalability and generalization across diverse datasets. Machine learning techniques, including support vector machines (SVM), k-nearest neighbors (KNN), and random forests (RF), addressed some limitations but required extensive feature engineering and lacked adaptability to complex spectral variations. The advent of deep learning revolutionized HSI classification, with convolutional neural networks (CNNs), graph convolutional networks (GCNs), and attention-based transformers demonstrating superior accuracy by learning spatial–spectral correlations automatically. Additionally, transfer learning and self-supervised learning have improved classification in low-data scenarios, making deep models more practical. Despite these advancements, challenges such as computational complexity, the requirement for large labeled datasets, and domain adaptation issues persist. This survey critically examines existing approaches, highlights their advantages and limitations, and identifies research gaps that can drive future developments in HSI classification. Finally, the review provides insights into emerging trends, including hybrid models and efficient lightweight architectures, to enhance the scalability and real-time applicability of HSI classification models. This comprehensive study serves as a foundation for researchers aiming to develop robust and efficient hyperspectral image classification methodologies.
LiDAR-based 3D object detection plays an essential role in autonomous driving, while existing pillar-based detectors still suffer from insufficient height representation and limited contextual modeling under sparse point clouds, leading to degraded performance for small, distant, and partially occluded objects. To address these issues, this paper proposes PillarH-CNSNet, a parameter-efficient pillar-based 3D detection framework that enhances PointPillars through improved geometric representation and multi-scale feature learning. Specifically, a height-aware dual-branch encoding module is introduced to preserve vertical geometric information by combining normalized height features and height distribution characteristics. In addition, a spatial attention module (SAM) is designed to enhance foreground responses and suppress irrelevant background features, while a parameter-efficient ConvNeXt-based backbone and Simplified Spatial Pyramid Pooling – Fast (SimSPPF) neck are employed to strengthen contextual representation and multi-scale feature aggregation. Extensive experiments on the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) benchmark demonstrate the effectiveness of the proposed method. Compared with PointPillars, PillarH-CNSNet achieves improvements of 2.21%, 0.48%, and 2.53% in car 3D AP40, 3.22%, 5.12%, and 4.65% in cyclist 3D AP40, and 6.28%, 6.12%, and 5.16% in pedestrian 3D AP40 under easy, moderate, and hard settings, respectively. The proposed model achieves an end-to-end inference latency of approximately 22.22 ms (45 FPS) on an NVIDIA RTX 5090D. These results indicate that PillarH-CNSNet effectively improves the robustness of pillar-based 3D detection for challenging scenarios while achieving a favorable balance between accuracy and efficiency.
Intensity inhomogeneity caused by uneven illumination, surface reflectance variation, and imaging noise often weakens the reliability of active-contour image segmentation. This paper proposes a bias-correcting variational level set evolution (BCVLSE) model for two-phase segmentation under nonuniform intensity conditions. Instead of explicitly estimating a slowly varying bias field in the image-intensity domain, BCVLSE works in the logarithmic additive domain and uses second-order differential processing to attenuate low-frequency bias variations. The resulting structural response is normalized by a sign-preserving bounded mapping and then coupled with the signed level set function through a gradient-guided fitting energy. Gaussian smoothing is used before differential processing to reduce noise amplification, and nonlinear regularization with local averaging is applied during iteration to stabilize contour evolution and suppress small spurious structures. Experiments on selected natural images, a public steel-defect benchmark, and self-collected industrial steel-surface defect images show that BCVLSE achieves competitive segmentation accuracy, robustness to initialization perturbations, and relatively low computational cost under the reported comparison protocols.
Urban sewage pipe network health assessment is vital for public safety. However, traditional defect detection models, which rely on extensive expert annotations and operate under the closed-set assumption, fail to generalize when confronted with novel or unseen defects. Furthermore, real-world images often contain multiple co-occurring defects, highlighting the need for Multi-Label Zero-Shot Learning (ML-ZSL). We introduce the Dual Adaptive Prompting Network (DAP-CLIP), which builds on Contrastive Language–Image Pre-training (CLIP) and employs a Dual Adaptation (DA) strategy to systematically bridge the fine-grained industrial domain gap. This strategy includes Full Fine-tuning of the Visual Encoder (FFE) for domain-specific visual adaptation and a Multi-Granularity Prompting (MGP) mechanism that uses Domain Description Generation (DDG) for static local alignment and Vision-guided Prompt Fine-tuning (VPF) for dynamic global alignment. This DA mechanism ensures precise knowledge transfer from seen to unseen defect categories in sewer pipe scenes. Experiments on the Sewer-ML and TransPipe datasets demonstrate that DAP-CLIP outperforms existing ML-ZSL methods across key metrics, including mean average precision (mAP) and F1 score, confirming its robustness and practical value in pipe defect detection.
Reliable provenance verification for diffusion-generated and diffusion-processed images requires watermarks that remain detectable while avoiding visible distortion in semantically sensitive regions. This paper presents a semantic-aware attention-guided watermarking framework for post-hoc diffusion watermarking. Given an image, prompt, binary message, and secret key, the message-key pair is deterministically mapped to a complex annular template and embedded in the Fourier domain of the DDIM-inverted latent. Cross-attention responses from multiple U-Net layers and denoising timesteps are aggregated to estimate prompt-aligned semantic importance. This attention prior is converted into a tolerance map, and its mean-normalized inverse is used in a semantic-weighted perceptual objective, constraining foreground objects and faces more strongly while allowing less perceptually salient background regions to absorb more watermark perturbation. Experiments on MS-COCO, DiffusionDB, and FFHQ demonstrate reliable verification performance, with WDR ranging from 0.964 to 0.989 and FPR no higher than 0.004 at P*=0.99. Across the same datasets, the proposed method further achieves global LPIPS values of 0.037, 0.036, and 0.031, respectively, together with an FID of 18.24, suggesting favorable perceptual fidelity and limited distributional deviation under the evaluated setting.However, the method remains sensitive to prompt-image misalignment, DDIM inversion quality, and geometric desynchronization. Future work will focus on prompt-independent semantic priors and synchronization-aware detection.
Image denoising aims to recover clean images from noisy observations while preserving fine structural details and texture information. Existing convolutional neural network (CNN)-based methods effectively capture local image structures but have limited capability for modeling long-range contextual dependencies, whereas Vision Transformer (ViT)-based approaches excel at global context modeling but often lack strong spatial inductive bias for fine texture preservation. To address these complementary limitations, we propose Convformer, a hybrid CNN–Transformer architecture for both synthetic additive white Gaussian noise (AWGN) and real-world image denoising. The proposed framework follows a residual learning paradigm and consists of a multi-domain Shallow Feature Extractor (SFE) and a Transformer-based Deep Feature Extractor (DFE). The SFE jointly exploits three complementary representations: the original spatial image, an intensity-inverted (negative) representation that enhances complementary structural and contrast information, and a frequency-domain representation obtained using DCT/IDCT transformations. These complementary features are extracted using the proposed Multi-Scale Residual Attention (MSRA) module and subsequently refined by the DFE. The DFE employs an encoder–decoder Transformer architecture equipped with the proposed Convolution-enhanced Window Multi-Head Self-Attention (Conv-WMHSA) and Convolution-enhanced Feed-Forward Network (Conv-FFN), which effectively combine long-range dependency modeling with local texture preservation through convolutional inductive bias. The network predicts the residual noise, which is subtracted from the noisy input to obtain the restored image. Extensive experiments on grayscale, color, and real-world denoising benchmarks demonstrate that Convformer achieves superior PSNR and SSIM with visually faithful reconstructions compared with representative methods. Ablation studies further verify the effectiveness of multi-domain feature extraction and CNN–Transformer integration.
Semantic segmentation of urban scenes is essential for autonomous driving and smart city applications. However, segmenting small, fine-grained objects like traffic signs and poles remains challenging due to spatial detail loss during downsampling and inefficient multi-scale feature fusion. To address these limitations, we propose a synergistically optimized model: the Gated Fusion Feature Pyramid Network (GF-FPN). Our model employs a ConvNeXt-Tiny backbone and a reconstructed FPN to establish a top-down pathway for systematic spatial restoration. Furthermore, we introduce a Gated Fusion (GF) unit at each FPN node. This unit utilizes a content-conditioned gating mechanism to dynamically modulate shallow-level features, proactively suppressing noise and refining object boundaries. Experiments on Cityscapes and CamVid validate the method's effectiveness. Achieving 76.64% MIoU with only 31.87 M parameters on Cityscapes, GF-FPN outperforms heavyweight counterparts in small-object categories, providing an efficient solution for practical urban scene understanding.
Publicly shared remote sensing images may expose sensitive targets such as aircraft and vessels. Existing sanitization methods can remove targets but do not support authorized recovery, whereas conventional hiding methods are limited by capacity and fidelity when embedding image-level oriented targets. This paper proposes a fixed-decoder neural steganography framework with scene-level restoration for sensitive target protection. The sender constructs a secret atlas by orientation-normalized cropping from quadrilateral annotations, generates a target-removed cover by inpainting, and embeds the atlas into texture-rich regions using mask-guided robust perturbation optimization. The receiver reproduces a fixed random decoder from a shared decoder seed, recovers the secret atlas from the stego-cover residual, and restores targets to their original positions using Record metadata. Experiments on a DOTA-derived dataset with a 128 × 128 atlas show that the proposed method achieves 43.82 dB PSNR and 0.9878 SSIM between stego images and covers without attack. Under JPEG compression (QF=80), it obtains 30.03 dB masked PSNR and 0.9221 masked SSIM for valid-region recovery, with stable performance under Gaussian and salt-and-pepper noise. The results indicate that the proposed framework balances public-image imperceptibility, recovery robustness, and scene-level usability for secure remote sensing image sharing.
Recent composite degradation restoration methods have enabled the handling of degradations with varying degrees and combinations via a unified framework. Some studies employ high-quality priors to replace degraded information. Nevertheless, key challenges lie in how to design a strategy to map degraded codes to clean codes and compensate for the information loss caused by mapping. To tackle these challenges, we create our fine-grained structure-preserving composite degradation image restoration model (FSPIR). At first, a high-low frequency mapping (HLFM) group is designed to fulfill the desired mapping. We then propose a gated cross-attention (GCA) module, which leverages residual valid information in the degraded images to compensate for information loss. Additionally, a discrete wavelet transform (DWT) loss is introduced, which simultaneously constrains the high and low-frequency components to regularize the image structure. Extensive experiments confirm that our method eliminates degradations while preserving the fine-grained structures of images. The proposed FSPIR outperforms state-of-the-art methods both in simulated and real-world degradation scenarios.
Modern Visual Place Recognition (VPR) methods highly rely on Deep Learning (DL) models. These DL-based models offer rigid representations with fixed complexity. That is, a single representation is forced to capture task-specific patterns. However, in practice, different application scenarios may require varying computational resources. Dimensionality reduction techniques are commonly used to match available computing resources, which is time-consuming. In this work, we propose a representation learning framework, ReLATE, that generates flexible and efficient features. To achieve this, the ReLATE framework employs a multi-Pareto objective that, in addition to optimizing the complete representation, also optimizes its subsets for the downstream task. In other words, along with the original features, this enables the low-dimensional subsets drawn from the complete representation to capture essential features. Extensive experiments on popular aggregators such as NetVLAD demonstrate competitive performance with the baseline VPR model. This is while offering significant computational and storage savings. For instance, the NetVLAD aggregator trained in the ReLATE setting produces 16× smaller descriptors at the cost of ∼1.62% 𝑅@1 on the Pittsburgh 30k test data. This further reduces the storage of 10k representations from 655MB to 41MB in float32 format. Importantly, once the aggregators are trained in the proposed framework, they can serve VPR at different computational budgets, offering adaptive test-time efficiency.
Intelligent scene analysis is a crucial activity that aims to analyze and comprehend visual situations in computer vision and artificial intelligence using machine learning algorithms and computer vision techniques. The research introduces a novel concept for Intelligent Scene Analysis, named "Graph Convolutional Efficient Det and Curve-Skeleton Deep Gaussian Processes," to address challenges in object recognition from complex scenes. The research focuses on the recognition of objects from complex scene images using a Graph Convolutional Network with Panoptic Efficient Det. As the object in the scene deforms or has diverse poses, the key points of the objects alter making the object detection challenging. To overcome this challenge, a novel algorithm Self-Supervised Attention Region Graph Convolution Network (SA-RGCN) is employed, which captures the shift key points of objects with articulated structures, such as humans by exploiting the dependencies and relationships among key points, effectively capturing them regardless of deformation or diverse poses. Panoptic Feature Pyramid Efficient Det (PFP-Efficient Det) is also utilized for object detection from complex scenes, providing semantic labels to every pixel. Then to generate the textual descriptions for the generated output, a Sequential Recurrent Transformer-XL Net is developed. Furthermore, to address difficulties in determining object orientations and depths, Probabilistic Curve-Skeleton Deep Gaussian Processes (PCS-DGP) are introduced, which extract the central axis of objects and establish relationships between extracted features and central angles, aiding in accurate alignment, orientation, and depth estimation. The experimental outcomes of the proposed framework from the evaluations effectively validate the outperformance in detecting the object from the complex scene with their alignment and depth in terms of accuracy, recall, precision, specificity, sensitivity, FPR, FNR, F1-score, mean accuracy, and mAP.
Multimodal emotion recognition (MER) is crucial for empathetic human-computer interaction, yet faces challenges such as modality imbalance, data scarcity, noise, and achieving robust generalization. This paper introduces a novel framework designed to address these challenges by integrating advanced sequential modelling with counterfactual learning. Our architecture employs modality-specific feature extraction (leveraging RoB-ERTa, Wav2Vec2, ResNet-50, and LSTM) followed by dedicated Progressive BiLSTM modules incorporating internal self-attention and adaptive gating to capture complex temporal dynamics within text, audio, and visual streams. It utilizes a bidirectional cross-attention mechanism across all modalities to facilitate rich inter-modality fusion, generating contextually enhanced representations. These are subsequently standardized and fused through a shared MLP feature stack. Central to our contribution is a three-phase counterfactual training framework defining a counterfactual generator and intention predictor with the application of Focal Loss, an alignment loss to enhance cross-modal consistency and robustness. It allows explicit modelling of counterfactual relationships, a concept explored for explainability but novel as a direct training strategy for MER. Comprehensive experiments on publicly available MELD and IEMOCAP datasets demonstrate the effectiveness of our approach, showcasing significant improvements in emotion recognition accuracy and robustness compared to baseline and state-of-the-art methods. The proposed counterfactual training strategy proves particularly beneficial in learning discriminative and generalizable multimodal representations. The code is available at: http s://github.com/BLANK-2340/A-Unified-Approach-for-Multimodal-Emotion-Recognition-Using-Counterfactual-Learning
