In this paper we explore the impact of differentiable JPEG configuration on the performance of a sandwiched image compression framework by considering both fixed and learnable quantization tables. To better align with real JPEG behavior, we introduce a conditional penalty term in the loss function. Preliminary results on CLIC dataset demonstrate that the improved differentiable codec significantly enhances rate–distortion performance, particularly at low and medium bitrates. These findings highlight the critical role of differentiable codec design and configuration in enabling effective end-to-end learning of pre- and post-processing modules.

No-reference image quality assessment (NR-IQA) is a pivotal technique for evaluating visual content quality across a wide spectrum of applications, from social media and remote education to autonomous driving and medical imaging. Despite significant advancements in deep learning-based NR-IQA methods, two critical challenges persist: heavy reliance on large-scale labeled datasets and insufficient ability to perceive diverse distortions. To address these limitations, we propose a two-stage framework integrating degradation-aware self-supervised contrastive pre-training with frequency-spatial domain feature learning. In the first stage, a MoCo-based contrastive learning strategy trains a momentum encoder to learn robust, generalizable distortion representations by distinguishing between differently degraded image pairs, eliminating the need for subjective quality labels. The second stage introduces a frequency-spatial perception module that conducts complementary analysis: in the frequency domain, a contrast sensitivity function-weighted spectral analysis emphasizes perceptually relevant artifacts, while in the spatial domain, multi-scale features are extracted via parallel convolutional branches, with adaptive fusion of all features through an attention mechanism. Here we show that extensive experiments on seven benchmark datasets (LIVE, CSIQ, TID2013, KADID-10K, LIVEC, KonIQ-10K, LIVE-FB) demonstrate superior performance, achieving a maximum PLCC of 0.958 and SROCC of 0.955 on KADID-10K, and 0.955 PLCC and 0.939 SROCC on KonIQ-10K. This framework not only mitigates the reliance on labeled data but also enhances the generalization across synthetic and authentic distortions, offering a reliable solution for NR-IQA in real-world multimedia systems and advancing the alignment between computational quality assessment and human visual perception. The source code is available at https://github.com/huqigong/DFS .

With the rapid advancement of Artificial Intelligence Generated Content (AIGC) techniques, AI generated images (AIGIs) have attracted widespread attention, among which AI generated omnidirectional images (AIGODIs) hold significant potential for Virtual Reality (VR) and Augmented Reality (AR) applications. AI generated omnidirectional images exhibit unique quality issues, however, research on the quality assessment and optimization of AI-generated omnidirectional images is still lacking. To this end, this work first studies the quality assessment and distortion-aware saliency prediction problems for AIGODIs, and further presents a corresponding optimization process. Specifically, we first establish a comprehensive database to reflect human feedback for AI-generated omnidirectionals, termed OHF2024, which includes both subjective quality ratings evaluated from three perspectives and distortion-aware salient regions. Based on the constructed OHF2024 database, we propose two models with shared encoders based on the BLIP-2 model to evaluate the human visual experience and predict distortion-aware saliency for AI-generated omnidirectional images, which are named as BLIP2OIQA and BLIP2OISal, respectively. Finally, based on the proposed models, we present an automatic optimization process that utilizes the predicted visual experience scores and distortion regions to further enhance the visual quality of an AI-generated omnidirectional image. Extensive experiments show that our BLIP2OIQA model and BLIP2OISal model achieve state-of-the-art (SOTA) results in the human visual experience evaluation task and the distortion-aware saliency prediction task for AI generated omnidirectional images, and can be effectively used in the optimization process. The database and codes will be released on https://github.com/IntMeGroup/AIGCOIQA to facilitate future research.

As the complexity of 3D digital content grows exponentially, understanding human visual attention is critical for optimizing rendering and processing resources. Therefore, reliable 3D mesh saliency ground truth (GT) is essential for human-centric visual modeling in virtual reality (VR). However, existing VR eye-tracking frameworks are fundamentally bottlenecked by their underlying acquisition and generation mechanisms. The reliance on zero-area single ray sampling (SRS) fails to capture contextual features, leading to severe texture aliasing and discontinuous saliency signals. And the conventional application of Euclidean smoothing propagates saliency across disconnected physical gaps, resulting in semantic confusion on complex 3D manifolds. This paper proposes a robust framework to address these limitations. We first introduce a view cone sampling (VCS) strategy, which simulates the human foveal receptive field via Gaussian-distributed ray bundles to improve sampling robustness for complex topologies. Furthermore, a hybrid Manifold-Euclidean constrained diffusion (HCD) algorithm is developed, fusing manifold geodesic constraints with Euclidean scales to ensure topologically-consistent saliency propagation. We demonstrate the improvement in performance over baseline methods and the benefits for downstream tasks through subjective experiments and qualitative and quantitative methods. By mitigating "topological short-circuits" and aliasing, our framework provides a high-fidelity 3D attention acquisition paradigm that aligns with natural human perception, offering a more accurate and robust baseline for 3D mesh saliency research.

Few-shot Action Recognition (FSAR) aims to recognize novel actions from only a few labeled examples, posing challenges due to limited supervision and complex temporal dynamics. Existing methods often adopt a unified motion modeling strategy for both short- and long-term dynamics, overlooking the need to adapt motion pattern extraction to the specific temporal properties inherent to different timescales. This forces models to hedge against multi-scale relevance through exhaustive searches over temporal tuples, followed by heavy spatio-temporal fusion, which substantially increases parameters and computation and ultimately limits efficiency. To this end, we propose the efficient Temporal Consistency and Variation-Guided Spatio-Temporal Aggregation Network (TCV-STA), which comprises four key components: the Temporal Consistency Module (TCM), the Temporal Variation Module (TVM), the Spatio-Temporal Aggregation attention (STA), and the Shifted Window Temporal Attention (SWTA). The TCM captures stable motion patterns to suppress short-term perturbations and enhance temporal consistency for robust motion representation, while the TVM models dynamic motion patterns to highlight long-term variations that improve inter-class discriminability and facilitate intra-class alignment. Built upon these complementary motion cues, the STA selectively aggregates spatial and temporal representations under the guidance of the learned stable and dynamic motion patterns, avoiding global dense fusion. Finally, to address the limited receptive field and discontinuous modeling caused by frame grouping in TCM and TVM, we adapt a SWTA to capture longer-range temporal dependencies and ensure smooth transitions across subaction segments for few-shot action recognition. Experiments demonstrate that TCV-STA achieves competitive accuracy across four widely-used FSAR benchmarks while reducing parameters by up to 27.9% and computational cost by 21.3%, striking a favorable balance between accuracy and efficiency for deployment in resource-constrained scenarios.

The skeleton representation is currently the emphasis of human action recognition research due to its excellent robustness and lightweight nature. Additionally, popular benchmark models mostly employ multi-stream skeleton fusion strategies to enhance performance. However, current multi-stream strategies typically fuse various features at the end of the model, thereby neglecting the interaction between information from different streams. To solve this problem, we propose the Global Information Fusion Network (GIFN) for Human Action Recognition. In GIFN, we first propose Intermediate Features Bidirectional Fusion (IFBF) to aggregate and reshape intermediate modality features to construct richer graph representations. Then the model extracts multiple perspective interactions among the graph representations through the proposed Multiple Interactive Information Extractor (MIIE) module. Finally, Three-stage Hierarchical Supervision (THS) is designed to supervise the training process to address the training instability issues caused by pre-fusion methods. Besides, we construct the shallow network Multi-field Spatial Temporal Network (MSTN) after pre-fusion stage to further extract spatio-temporal joint features and let the action recognition task more efficient. The whole model excellently accomplishes the task of human action recognition based on skeleton sequences, and achieves the state-of-the-art performance on popular action recognition datasets such as NTU RGB+D 60, NTU RGB+D 120, and Northwestern-UCLA.

Recent advances in multimodal large language models (MLLMs) have greatly improved image understanding and captioning capabilities. However, existing image captioning benchmarks typically suffer from limited diversity in caption length, the absence of recent advanced MLLMs, and insufficient human annotations, which potentially introduces bias and limits the ability to comprehensively assess the performance of modern MLLMs. To address these limitations, we present a new large-scale image captioning benchmark, termed, ICBench, which covers 12 content categories and consists of both short and long captions generated by 10 advanced MLLMs on 2K images, resulting in 40K captions in total. We conduct extensive human subjective studies to obtain mean opinion scores (MOSs) across fine-grained evaluation dimensions, where short captions are assessed in terms of fluency, relevance, and conciseness, while long captions are evaluated based on fluency, relevance, and completeness. Furthermore, we propose an automated evaluation metric, ITIScore, based on an image-to-text-to-image framework, which measures caption quality through reconstruction consistency. Experimental results demonstrate strong alignment between our automatic metric and human judgments, as well as robust zero-shot generalization ability on other public captioning datasets. Both the dataset and model will be released upon publication.

Although semi-supervised semantic segmentation ($\text{S}^4$) utilizes abundant unlabeled data to reduce manual labeling burdens, independent training of labeled and unlabeled data causes the former to dominate, which severely degrades pseudo-label quality. To address this challenges, we propose a novel remote sensing (RS) $\text{S}^4$ method via unified flow with feature memory bank (UFFM). Specifically, UFFM comprises two key innovations: unified flow (UF) and feature memory bank (FMB). The UF is a new training flow that generates less biased pseudo-labels by combining an external visual foundation model (VFM) with an RS domain teacher, and jointly optimizes labeled and pseudo-labeled data under a unified training objective. The FMB is a novel memory module for $\text{S}^4$ that dynamically updates class-specific features during training and reduces the feature discrepancy between labeled and unlabeled data through class-feature alignment. To verify the effectiveness of our model, we conduct extensive experiments on RS datasets. The experimental results show the superiority of our method over SOTA $\text{S}^4$ methods. Moreover, the results demonstrate the effectiveness of our contributions in bridging the optimization and feature representation gap between labeled and unlabeled data. Our code is released at \href{https://github.com/wangshanwen001/RS-UFFM}{https://github.com/wangshanwen001/RS-UFFM}.

Deep neural network (DNN)-based image quality assessment (IQA) methods have achieved remarkable progress in recent years. However, existing DNN-based IQA approaches still face two major challenges. First, IQA datasets usually suffer from insufficient training samples. Second, substantial differences in quality scoring standards and scales among different datasets limit the unified training of IQA models. In this paper, we propose a novel objective IQA training framework based on federated learning (FL). To tackle the problem of insufficient labeled training samples, the proposed FL-based IQA framework conducts collaborative training by uniting samples from different datasets. Instead of simply mixing multiple datasets, we train the IQA model on each dataset individually and aggregate the results of different models. Meanwhile, considering the inconsistencies in the scoring standards and scales of different datasets, we propose a model decoupling strategy to divide the model into shared and personalized components: the former maintains global knowledge across all models, while the latter allows flexible adjustments tailored to the specific characteristics of individual datasets. Experiments on five IQA datasets show that our proposed FL-based IQA framework has improvements in accuracy and robustness on existing IQA models.

This paper presents the results of the Grand Challenge on Video Quality Assessment for Asymmetric Encoded Videos, held at QoMEX 2026 in Cardiff, UK. The challenge addresses the growing need for video quality metrics (VQM) capable of accurately predicting the perceptual quality of asymmetrically encoded videos, where saliency-driven or semantic-based encoding allocates different quality levels to different spatial regions. Participants were provided with the Sport-ROI dataset containing subjective quality scores and were invited to develop both full-reference (FR) and no-reference (NR) VQM models. We describe the challenge design, the dataset, the evaluation methodology, and summarize the submitted approaches and their performance.

Deep neural networks have shown remarkable progress in blind image quality assessment. However, accurately modeling human visual perception remains challenging due to the wide variations in image content and the complex interplay of distortion types. Existing methods, relying on content-agnostic or fixed receptive field approaches, struggle to capture adaptive perceptual features linking semantic regions and distortion perception. To address these limitations, we propose the dual perception-aware model, a two-stage framework integrating semantic-and distortion-aware representations, and then exploring dynamic global-local feature extraction. First, our method leverages superpixel similarity indicators as semantic-aware representations that capture perceptually coherent regions, enabling subsequent content-adaptive feature extraction beyond traditional grid-based methods. A cross-attention mechanism then facilitates mutual modulation between semantic importance and distortion sensitivity, allowing the model to focus on perceptually critical areas while maintaining distortion awareness. Second, we design an adaptive parallel feature extraction unit combining vision transformer blocks with enhanced adaptive filtering residual blocks, achieving comprehensive global-local feature representation that adapts to image-specific characteristics, followed by a weighted dual-pathway regressor for content-tailored quality predictions. Extensive experiments on benchmark datasets containing both synthetic and authentic distortions demonstrate superior performance compared to state-of-the-art methods, with comprehensive ablation studies validating the effectiveness of each proposed component.

Underwater image quality assessment (UIQA) is a critical research area, challenged by underwater environments such as wavelength-dependent light attenuation, scattering, and non-uniform illumination. Existing deep learning-based UIQA methods often address these degradations in isolation, neglecting their complex interplay with human perception and lacking explicit modeling of underwater optical phenomena. To address this, we propose PhysIQ-Net, a novel framework that integrates physics-driven principles with progressive multi-prior interaction modeling through three key innovations: First, introduce dual physics-based decomposition that separates images into Backscatter, Transmission, Reflectance, and Illuminance components to capture distinct degradation mechanisms; Second, propose prior-guided dynamic filtering that adapts convolutional kernels to image-specific content using physical priors; and Third, propose physic-informed Cross-Domain Feature Interaction that enables bidirectional collaboration between color-aware and structure-aware representations to model their perceptual inter-dependencies. Extensive experiments across multiple benchmark datasets demonstrate that PhysIQ-Net significantly outperforms existing methods, with ablation studies validating each component’s contribution, providing a robust solution for UIQA.

Face retouching is a composite image editing task, and is challenging in simultaneously accomplishing multiple editing targets, such as removing acne and adjusting skin tone. Existing face retouching methods typically perform global editing without target-specific analysis, which leads to the overlook of certain targets and under-edited output. In this work, we propose DFRetouch, a multi-task face retouching framework that enables multi-target editing by explicitly modeling target-related sub-tasks and progressively modulating target-oriented features. Specifically, we introduce a Face-aware Task Embedding (FTE) module that decomposes user instructions into target-associated sub-tasks and generates task-specific retouching embeddings, which encode the information of the requested operations and the corresponding facial structures. To accomplish multiple retouching objectives, we further design a Sub-task Feature Refinement (SFR) module that stacks task fusion cross-attention blocks to aggregate task-specific embeddings and modulates features conditioned on the resulting representation, thereby synthesizing high-quality retouching results. Extensive experiments demonstrate that DFRetouch outperforms state-of-the-art face retouching methods in terms of visual fidelity and multi-task flexibility.

Existing supervised and unsupervised shadow removal methods often suffer from limited generalization due to the insufficient diversity of available training datasets, while zero-shot methods tend to produce artifacts and require time-consuming test-time optimization. To address these issues, we propose FreeShadow, a training-free shadow removal method built upon pretrained diffusion models, which exploits diffusion priors for shadow removal without any training or optimization. For illumination recovery, we propose an illumination transfer attention (ITA), which re-weights the self-attention maps in diffusion model to transfer illumination cues from non-shadow to shadow regions. For content preservation, we analyze the effects of illumination variations on self-attention maps and latent high-frequency features in diffusion model, and selectively preserve illumination-invariant components to maintain content fidelity while suppressing residual shadows. We further propose local texture-preserving relighting (LTPR) to mitigate local texture misalignment caused by VAE compression. Extensive experiments demonstrate that our method achieves strong generalization and produces realistic shadow-free images.

AI-generated human-centric videos play a crucial role in a wide range of modern applications. However, they often suffer from quality issues and semantic mismatches, underscoring the importance of effective quality assessment for such videos. To this end, we extend our previous dataset HVEval with pairwise preference annotations, resulting in HVEval+, the largest holistic quality assessment dataset for AI-generated human-centric videos, which comprises 1k prompts based on a comprehensive taxonomy, 20k videos generated by 24 text-to-video (T2V) models, and extensive human annotations, including 60k mean opinion scores (MOSs) and 60k preference pairs across 3 dimensions (i.e., spatial quality, temporal quality, and text-video correspondence), as well as 20k category-specific question-answer (Q A) pairs. Along with the HVEval+ dataset, we further propose MoE-Rater, a Mixture-of-Experts (MoE)-inspired and multimodal large language model (MLLM)-based all-in-one method that supports multi-dimensional quality rating, multi-dimensional pairwise comparison, and category-specific question answering within a single model. Specifically, we introduce Mixture of Projector Experts (MoPE) and Mixture of LoRA Experts (MoLE), together with a three-stage training strategy consisting of task-aware pre-training, task-specific adaptation, and adaptive routing optimization, to effectively unify multiple tasks, resulting in superior performance on both HVEval+ and Human-AGVQA datasets. Extensive experiments and comprehensive analysis demonstrate the significant potential of the HVEval+ dataset and the MoE-Rater method in advancing AI-generated video quality assessment and further facilitating the evaluation and optimization of T2V models.

In spike camera shooting, stray light accumulation, thermal noise, and light intensity loss make pixel-generated spike intervals inaccurately reflect original light source info, posing two video salient object detection (VSOD) challenges in spike streams: 1) Noisy spikes hinder long-term video info capture; 2) Noise makes sparse spikes struggle to capture salient object texture details. To address these issues, we propose a noise-robust VSOD model based on spiking neural networks (SNNs) with a gradual fusion strategy: the spatial-channel cross-perception module (SCPM) enhances attention to salient regions and filters spatial noise; the local deformable cross-attention module (LDCM) strengthens local feature correlations for temporal denoising; the global information-enhanced self-attention module (GISM) models global context and extracts fine-grained textures. Extensive experiments show that it achieves an average relative improvement 9.5% across all metrics compared to state-of-the-art methods.

Facial Expression Recognition (FER) adds a layer of emotional intelligence to technology, enabling digital systems to respond to user emotional states. While deep learning has advanced FER, practical application is constrained by the long-tail distribution of real-world data, where underrepresented minority classes (e.g., Fear, Disgust) are frequently misclassified due to feature ambiguity. To address this limitation, we present EMVMamba, a novel hybrid architecture designed to resolve the part-whole ambiguity in rare expressions. Specifically, EMVMamba framework employs a dual-stream paradigm: a Local Hierarchical Branch (CNN) to capture multi-level compositional structure and textural details, and a Global Topological Branch (VMamba) to model long-range semantic dependencies. We also introduce an Adaptive Fusion Gate that leverages global context to bridge these distinct feature domains. Combining this architectural advantage with a class-weighted loss, EMVMamba achieves a competitive 80.61% accuracy on RAF-DB. Notably, its performance surpasses DDAMFN and marginally exceeds the heavy Poster++ model with 40% fewer FLOPs (5.06G vs 8.4G), offering a state-of-the-art trade-off between computational efficiency and rare-class recognition performance. Additional experiments on the FER-2013 dataset further demonstrate its robust generalization capabilities.

The rapid advancement of AI generation technologies has led to the widespread use of AI-generated multimedia content, including images, videos, and 3D contents, across various applications. While significant progress has been made in quality evaluation for 2D content, evaluating the quality of 3D content synthesized from single image remains an underexplored problem. To bridge this gap, we introduce the first comprehensive subjective evaluation database tailored for assessing the quality of 3D content generated from single image. Our database, named AIGC-SI23DCQA, includes three distinct categories of input images, i.e., realistic images, AI-generated images, and computer graphic (CG) images, with 100 images in each category. Using five representative single-image-to-3D algorithms, we produce 1,500 3D contents and collect 94,500 annotations across three quality dimensions, including texture fidelity, shape accuracy, and overall quality. Based on the constructed database, we first benchmark and evaluate the performance of existing quality assessment methods revealing their limitations in addressing this novel task. Thus, we further propose a novel objective quality assessment method, termed I3DQA, for effective single-image-to-3D content quality assessment. Specifically, I3DQA first extracts the reference features from the source image, and the multi-modal features from the generated 3D content, including the projected video, patches, and large-multimodal model (LMM) features. These features are integrated through symmetric transformer blocks, enabling effective quality-related feature fusion and score prediction. Extensive experiments demonstrate the superior performance of our method and validate the effectiveness of its components. This work provides a foundational resource and a robust framework for advancing research in this emerging field, and our database and model are released at https://github.com/ZedFu/SI23DCQA.

Image or video composition aims to combine the foreground object and the background content to generate a realistic composite image or video, in which harmonization algorithms play a crucial role in adjusting the color or light of the foreground object to make the entire image or video harmonious. However, existing image quality assessment (IQA) and video quality assessment (VQA) methods often fail to align with human preference in evaluating harmonization algorithms due to insensitivity to minor color or light inconsistencies. To address the issue and facilitate the advancement of harmonization, we introduce the first harmonization quality assessment database, termed HarmonyQAD, which consists of Harmony-IQAD comprising 1,800 harmonized images generated by 9 different image harmonizaion algorithms (IHAs), and HarmonyVQAD comprising 1,600 harmonized videos generated by 2 IHAs and 6 video harmonization algorithms (VHAs). All harmonized images and videos are accompanied by mean opinion scores (MOSs), collected through a meticulously designed subjective evaluation experiment. Based on the HarmonyQAD, we propose a unified framework for harmonization quality assessment, termed HarmonyQA, to predict the harmonization quality of both harmonized images and videos. Extensive experiments show that HarmonyQA achieves state-of-the-art performance on our datasets, and also achieves competing results on traditional IQA and VQA tasks. Furthermore, cross-dataset evaluation also shows that HarmonyQA exhibits better generalization ability than self-supervised learning-based IQA methods. The dataset and code are available at https://github.com/IntMeGroup/HarmonyQA.

Existing shadow removal methods often suffer from inconsistent illumination restoration across shadowed regions and conspicuous artifacts near shadow boundaries. Although several approaches attempt to address these issues via divide-and-conquer strategies, they inevitably incur substantial computational overhead and increased model complexity. Observing the distinct spatial distributions of low- and high-frequency degradations in shadowed images, we propose a region-aware frequency divide-and-conquer (RFDC) strategy for shadow removal. RFDC decomposes shadow images into low- and high-frequency components for targeted processing, thereby mitigating the additional computational burden typically introduced by conventional divide-and-conquer schemes. Specifically, the low-frequency branch focuses on illumination recovery in shadowed regions, while the high-frequency branch aims to suppress artifacts along shadow boundaries. For low-frequency restoration, a region-adaptive dual-scope self-attention (RDSA) is proposed to adaptively capture local and global non-shadow illumination cues for shadow boundaries and central regions, respectively. For high-frequency restoration, a matching-enhanced self-attention (MESA) is proposed to strengthen query-key matching for sparse high-frequency feature using restored low-frequency features. In addition, a boundary high-frequency (BHF) loss is introduced to explicitly guide the details recovery at shadow boundaries. Extensive experiments on benchmark datasets demonstrate that our method outperforms state-of-the-art approaches with fewer parameters and FLOPs.