Image Quality Assessment (IQA) with references plays an important role in optimizing and evaluating computer vision tasks. Traditional methods assume that all pixels of the reference and test images are fully aligned. Such Aligned-Reference IQA (AR-IQA) approaches fail to address many real-world problems with various geometric deformations between the two images. Although significant effort has been made to attack Geometrically-Disparate-Reference IQA (GDR-IQA) problem, it has been addressed in a task-dependent fashion, for example, by dedicated designs for image super-resolution and retargeting, or by assuming the geometric distortions to be small that can be countered by translation-robust filters or by explicit image registrations. Here we rethink this problem and propose a unified, non-training-based Deep Structural Similarity (DeepSSIM) approach to address the above problems in a single framework, which assesses structural similarity of deep features in a simple but efficient way and uses an attention calibration strategy to alleviate attention deviation. The proposed method, without application-specific design, achieves state-of-the-art performance on AR-IQA datasets and meanwhile shows strong robustness to various GDR-IQA test cases. Interestingly, our test also shows the effectiveness of DeepSSIM as an optimization tool for training image super-resolution, enhancement and restoration, implying an even wider generalizability.

Radar-based human activity recognition has gained attention as a privacy-preserving alternative to vision and wearable sensors, especially in sensitive environments like long-term care facilities. Micro-Doppler spectrograms derived from FMCW radar signals are central to recognizing dynamic activities, but their effectiveness is limited by noise and clutter. In this work, we use a benchmark radar dataset to reimplement and assess three recent denoising and preprocessing techniques: adaptive preprocessing, adaptive thresholding, and entropy-based denoising. To illustrate the shortcomings of conventional metrics in low-SNR regimes, we evaluate performance using both perceptual image quality measures and standard error-based metrics. We additionally propose a novel framework for static activity recognition using range-angle feature maps to expand HAR beyond dynamic activities. We present two important contributions: a temporal tracking algorithm to enforce consistency and a no-reference quality scoring algorithm to assess RA-map fidelity. According to experimental findings, our suggested techniques enhance classification performance and interpretability for both dynamic and static activities, opening the door for more reliable radar-based HAR systems.

The aesthetic quality of a scene depends strongly on camera viewpoint.Existing approaches for aesthetic viewpoint suggestion are either single-view adjustments, predicting limited camera adjustments from a single image without understanding scene geometry, or 3D exploration approaches, which rely on dense captures or prebuilt 3D environments coupled with costly reinforcement learning (RL) searches. In this work, we introduce the notion of 3D aesthetic field that enables geometry-grounded aesthetic reasoning in 3D with sparse captures, allowing efficient viewpoint suggestions in contrast to costly RL searches.We opt to learn this 3D aesthetic field using a feedforward 3D Gaussian Splatting network that distills high-level aesthetic knowledge from a pretrained 2D aesthetic model into 3D space, enabling aesthetic prediction for novel viewpoints from only sparse input views. Building on this field, we propose a two-stage search pipeline that combines coarse viewpoint sampling with gradient-based refinement, efficiently identifying aesthetically appealling viewpoints without dense captures or RL exploration. Extensive experiments show that our method consistently suggests viewpoints with superior framing and composition compared to existing approaches, establishing a new direction toward 3D-aware aesthetic modeling.

Transformer-based models have demonstrated great promises in single image super-resolution (SISR), but our investigations find significant redundancy in terms of high mutual information across the attention maps, which is associated with reduced efficiency and degraded performance of SOTA models. To address the problem, here we propose a low redundancy attention network (LRAN). First, to mitigate the redundancy among heads, we introduce in the self-attention computation a multi-element mechanism, which allows for the incorporation of various types of self-attention, thus increasing inter-head diversity. Second, to address the redundancy among blocks, we propose the encapsulated architecture, in which enhanced local perception unit and gated multi-layer perceptron are designed to capture local information. Specifically, this architecture incorporates a single self-attention layer between several MLP layers. Subsequently, the proposed gated multi-layer perceptron significantly enhances the SR quality. Extensive experiments demonstrate that LRAN outperforms SOTA models in the task of lightweight SR, achieving a better trade-off between quality and speed. For instance, the proposed LRAN-light surpasses SwinIR-light by 0.32dB PSNR in $\times 4$ SR on Urban100, while running $\times 4$ faster.

No-Reference Image Quality Assessment (NR-IQA) aims to estimate perceptual quality without access to a reference image of pristine quality. Learning an NR-IQA model faces a fundamental bottleneck: its need for a large number of costly human perceptual labels. We propose SHAMISA, a non-contrastive self-supervised framework that learns from unlabeled distorted images by leveraging explicitly structured relational supervision. Unlike prior methods that impose rigid, binary similarity constraints, SHAMISA introduces implicit structural associations, defined as soft, controllable relations that are both distortion-aware and content-sensitive, inferred from synthetic metadata and intrinsic feature structure. A key innovation is our compositional distortion engine, which generates an uncountable family of degradations from continuous parameter spaces, grouped so that only one distortion factor varies at a time. This enables fine-grained control over representational similarity during training: images with shared distortion patterns are pulled together in the embedding space, while severity variations produce structured, predictable shifts. We integrate these insights via dual-source relation graphs that encode both known degradation profiles and emergent structural affinities to guide the learning process throughout training. A convolutional encoder is trained under this supervision and then frozen for inference, with quality prediction performed by a linear regressor on its features. Extensive experiments on synthetic, authentic, and cross-dataset NR-IQA benchmarks demonstrate that SHAMISA achieves strong overall performance with improved cross-dataset generalization and robustness, all without human quality annotations or contrastive losses.

No-reference bitstream-layer point cloud quality assessment (PCQA) can be deployed without full decoding at any network node to achieve real-time quality monitoring. In this work, we develop the first PCQA model dedicated to Trisoup-Lifting encoded 3D point clouds by analyzing bitstreams without full decoding. Specifically, we investigate the relationship among texture bitrate per point (TBPP), texture complexity (TC) and texture quantization parameter (TQP) while geometry encoding is lossless. Subsequently, we estimate TC by utilizing TQP and TBPP. Then, we establish a texture distortion evaluation model based on TC, TBPP and TQP. Ultimately, by integrating this texture distortion model with a geometry attenuation factor, a function of trisoupNodeSizeLog2 (tNSL), we acquire a comprehensive NR bitstream-layer PCQA model named streamPCQ-TL. In addition, this work establishes a database named WPC6.0, the first PCQA database dedicated to Trisoup-Lifting encoding mode, encompassing 400 distorted point clouds with 4 geometry multiplied by 5 texture distortion levels. Experiment results on M-PCCD, ICIP2020 and the proposed WPC6.0 database suggest that the proposed streamPCQ-TL model exhibits robust and notable performance in contrast to existing advanced PCQA metrics, particularly in terms of computational cost.

Novel view synthesis methods, such as neural radiance fields and 3D Gaussian splatting, offer a promising solution for photorealistic rendering. However, they remain challenged in few-shot settings, where models tend to overfit the limited supervised views, leading to artifacts such as quality fluctuations, degradation in distant views, and geometric inconsistencies. To address these issues, we introduce Human Perceptual Preference Optimization (HuPPO), a framework that incorporates human perceptual guidance into model training. HuPPO mitigates distortions by regularizing training dynamics with perceptual preference cues, thereby reducing the reliance on extensive supervised views. Specifically, HuPPO leverages human perception to identify and select candidate novel views, and introduces a corresponding objective function that steers optimization toward perceptually preferred outcomes. In addition, a meta-learning pipeline is integrated to promote the learning of generalizable representations. The framework is flexible and can be seamlessly applied to a wide range of neural rendering models without incurring additional inference overhead. Extensive experiments and analyses demonstrate that HuPPO achieves consistent improvements over state-of-the-art baselines.

The evaluation of perceptual quality in 3D mesh compression, particularly for Video-based Dynamic Mesh Coding (V-DMC), is challenged by the scarcity of subject-rated datasets and the high computational cost of full-mesh decoding and the sophisticated visual feature extraction steps. To bridge this gap, we first introduce a novel V-DMC distortion dataset, comprising 16 high-quality original meshes and 400 compressed, textureless variants. We conducted a subjective quality assessment study with 30 participants using the Double Stimulus Impairment Scale (DSIS) method to collect reliable Mean Opinion Scores (MOS). We then propose streamMQ, the first-of-its-kind noreference, bitstream-layer model for perceptual quality assessment of V-DMC compressed meshes. By extracting key geometric features such as quantization parameters and triangle count directly from the compressed bitstream, streamMQ predicts perceptual quality without full decoding. Experimental evaluation and comparison with state-of-the-art methods demonstrate that streamMQ achieves highly competitive quality assessment performance at tiny fractions of computational and storage costs, facilitating real-time and low-storage application environments. The dataset and source code will be made publicly available at https://github.com/HFL01/QDU-GDM.

The rapid growth of Augmented Reality (AR) in a variety of applications such as mobile and retail sectors has intensified the need for perceptually accurate Quality of Experience (QoE) assessment. However, existing image quality assessment (IQA) models often fail to account for the unique degradations found in AR, particularly the interaction between virtual objects and real-world backgrounds. Existing works are largely limited by a lack of comprehensive datasets and an inability to jointly model object-level fidelity and photometric harmony. In this paper, we propose the AR composition quality assessment (AR-COMPQ) model, a novel dual-path deep learning framework designed to evaluate AR image composition quality by explicitly separating and then integrating object-level structural fidelity and appearance-level harmony. To support this research, we introduce the ARC-IQA dataset, a curated benchmark featuring 193 AR scenes with controlled geometric, texture, and illumination distortions, accompanied by Mean Opinion Scores (MOS) derived from a systematic subjective study. Experimental results demonstrate that AR-COMPQ achieves superior alignment with human perception, yielding higher correlation and lower prediction error with respect to MOS. The proposed method significantly outperforms traditional metrics and recent perceptually driven deep models, thereby facilitating robust automated quality control in AR rendering pipelines.

Machine-learning models working on skin images often have degraded performance when the skin colour captured in images (SCCI) differs between training and deployment. Such differences arise from entangled environmental factors (e.g., illumination, camera settings), and intrinsic factors (e.g., skin tone) that cannot be accurately described by a single "skin tone" scalar. To mitigate such colour mismatch, we propose a skin-colour disentangling framework that adapts disentanglement-by-compression to learn a structured, manipulable latent space for SCCI from unlabelled dermatology images. To prevent information leakage that hinders proper learning of dark colour features, we introduce a randomized, mostly monotonic decolourization mapping. To suppress unintended colour shifts of localized patterns (e.g., ink marks, scars) during colour manipulation, we further propose a geometry-aligned post-processing step. Together, these components enable faithful counterfactual editing and answering an essential question: "What would this skin condition look like under a different SCCI?", as well as direct colour transfer between images and controlled traversal along physically meaningful directions (e.g., blood perfusion, camera white balance), enabling educational visualization of skin conditions under varying SCCI. We demonstrate that dataset-level augmentation and colour normalization based on our framework achieve competitive lesion classification performance.

Recently, Video-based Point Cloud Compression (V-PCC) has demonstrated significant advantages in the field of dynamic point cloud (DPC) compression due to its excellent rate-distortion performance, laying the technical foundation for the large-scale application of DPC in immersive media. However, existing quality assessment studies for V-PCC compressed DPC primarily focus on compression distortion, with insufficient attention given to perceptual quality degradation caused by packet loss during transmission. To address this, this paper introduces streamDPCQ-PL—the first no-reference bitstream-level quality assessment model specifically designed for packet loss scenarios. This model efficiently predicts the subjective quality of V-PCC compressed DPC under transmission distortion. Specifically, we introduce the compression rate (CR) as a tuning parameter to quantify the dynamic changes of the model slope and intercept based on the linear relationship between the packet loss rate (PLR) and the mean opinion score (MOS). Meanwhile, considering the human visual system’s (HVS) high sensitivity to texture complexity, we propose a novel attribute measure—total attribute (TAE). TAE represents the cumulative value of attribute residual encoding payloads for all encoding units in a bitstream. It effectively quantifies differences in DPC across various texture representations and demonstrates significant advantages in capturing subtle changes in DPC under different levels of texture complexity. This enables the model to more accurately characterize texture distortion. Finally, with the help of empirical correction coefficients the model can distinguish the distortion differences caused by packet loss locations. Experimental results show that the model exhibits excellent performance on the DPC datasets proposed by both UNIN and UC labs, which further validates the reliability and effectiveness of the proposed method. Notably, the proposed model does not require full decoding the bitstream, which provides the possibility of real-time quality monitoring during transmission. To facilitate further research, the source code will be publicly released at https://github.com/qduywh/streamDPCQ-PL-model

Ultra-high-speed cameras frequently suffer from severe overexposure in scenarios involving extreme brightness transitions, significantly degrading image quality and obscuring critical visual details. To address this issue, we propose a novel reconstruction method combining neuromorphic sensors with state-of-the-art diffusion models. Our approach leverages the asynchronous, high-temporal-resolution, and high-dynamic-range capabilities of neuro-morphic sensors to capture rapid brightness variations, subsequently utilizing conditional diffusion models to reconstruct high-quality frames from sparse event data. We validated the proposed method through experiments conducted under three challenging lighting conditions. The results demonstrate that our approach effectively recovers detailed visual content in severely overexposed frames, significantly outperforming traditional frame-based imaging techniques.

The fast growing application of omnidirectional images calls for effective approaches for omnidirectional image quality assessment (OIQA). Existing OIQA methods have been developed and tested on homogeneously distorted omnidirectional images, but it is hard to transfer their success directly to the heterogeneously distorted omnidirectional images. In this paper, we conduct the largest study so far on OIQA, where we establish a large-scale database called OIQ-10K containing 10,000 omnidirectional images with both homogeneous and heterogeneous distortions. A comprehensive psychophysical study is elaborated to collect human opinions for each omnidirectional image, together with the spatial distributions (within local regions or globally) of distortions, and the head and eye movements of the subjects. Furthermore, we propose a novel multitask-derived adaptive feature-tailoring OIQA model named IQCaption360, which is capable of generating a quality caption for an omnidirectional image in a manner of textual template. Extensive experiments demonstrate the effectiveness of IQCaption360, which outperforms state-of-the-art methods by a significant margin on the proposed OIQ-10K database. The OIQ-10K database and the related source codes are available at https://github.com/WenJuing/IQCaption360.

AI models for skin cancer diagnosis often underperform on darker skin tones due to imbalanced training datasets that predominantly feature lighter skin. In this study, we investigate whether lightweight, textual input can mitigate this disparity in a low-data setting. We use a dataset of only 4,311 clinical dermatology images—3,900 from lighter skin tones and just 411 from darker tones—to train Vision Transformers (ViTs) enhanced with text input including skin tone and generated lesion descriptions from Gemini and MONET. These textual inputs are fused with visual features via late fusion strategies. Among all configurations, ViT-B/32 combined with BERT-encoded skin tone using Element-Wise Fusion achieved the most balanced results, with AUCs of 0.822 (light) and 0.825 (dark), and matched accuracies of 0.823. This setup reduced the AUC gap to 0.003 and the accuracy gap to 0.0001. Our findings show that incorporating simple and domain-specific textual input can substantially reduce skin tone bias in ViT-based diagnosis offering a practical solution for building fairer medical AI.

Image degradation synthesis is highly desirable in a wide variety of applications ranging from image restoration to simulating artistic effects. Existing models are designed to generate one specific or a narrow set of degradations, which often require user-provided degradation parameters. As a result, they lack the generalizability to synthesize degradations beyond their initial design or adapt to other applications. Here we propose the first universal degradation model that can synthesize a broad spectrum of complex and realistic degradations containing both homogeneous (global) and inhomogeneous (spatially varying) components. Our model automatically extracts and disentangles homogeneous and inhomogeneous degradation features, which are later used for degradation synthesis without user intervention. A disentangle-by-compression method is proposed to separate degradation information from images. Two novel modules for extracting and incorporating inhomogeneous degradations are created to model inhomogeneous components in complex degradations. We demonstrate the model's accuracy and adaptability in film-grain simulation and blind image restoration tasks. The demo video, code, and dataset of this project will be released at github.com/yangwenbo99/content-degradation-disentanglement.

The scope of point cloud (PC) applications is expanding. We propose a no-reference bitstream-layer quality assessment model that eliminates the need for full decoding of the PC, providing quality evaluation scores during the V-PCC decoding process. Specifically, we illustrate the relationship between content diversity (CD) and perceptual coding distortion in lossless geometric coding. Subsequently, we model attribute distortion by predicting CD using transform energy (TE) and texture quantization parameter (TQP). By combining the geometric distortion model with geometry quantization parameters (GQP) and the attribute distortion model, we derive comprehensive quality prediction results. Our experimental results on four PC databases (WPC2.0, M-PCCD, VSENSE VVDB and VSENSE VVDB2) show that the proposed energy-adaptive bitstream-layer model (EABL) delivers competitive quality prediction performance in comparison with existing full-reference, reduced-reference and no-reference PC quality assessment models that require full decoding, and meanwhile exhibits large speed advantage. The source code will be made publicly available for repeatability research at https://github.com/arthas-sws/EABL_model.

Mesh quality assessment (MQA) models play a critical role in the design, optimization, and evaluation of mesh operation systems in a wide variety of applications. Current MQA models, whether model-based methods using topology-aware features or projection-based approaches working on rendered 2D projections, often fail to capture the intricate interactions between texture and 3D geometry. We introduce HybridMQA, a first-of-its-kind hybrid full-reference colored MQA framework that integrates model-based and projection-based approaches, capturing complex interactions between textural information and 3D structures for enriched quality representations. Our method employs graph learning to extract detailed 3D representations, which are then projected to 2D using a novel feature rendering process that precisely aligns them with colored projections. This enables the exploration of geometry-texture interactions via cross-attention, producing comprehensive mesh quality representations. Extensive experiments demonstrate HybridMQA's superior performance across diverse datasets, highlighting its ability to effectively leverage geometry-texture interactions for a thorough understanding of mesh quality. Our project website is available at https://arshafiee.github.io/hybridmqa/.

Despite substantial efforts dedicated to the design of heuristic models for omnidirectional (i.e., 360°) image quality assessment (OIQA), a conspicuous gap remains due to the lack of consideration for the diversity of viewing behaviors that leads to the varying perceptual quality of 360° images. Two critical aspects underline this oversight: the neglect of viewing conditions that significantly sway user gaze patterns and the overreliance on a single viewport sequence from the 360° image for quality inference. To address these issues, we introduce a unique generative scanpath representation (GSR) for effective quality inference of 360° images, which aggregates varied perceptual experiences of multi-hypothesis users under a predefined viewing condition. More specifically, given a viewing condition characterized by the starting point of viewing and exploration time, a set of scanpaths consisting of dynamic visual fixations can be produced using an apt scanpath generator. Following this vein, we use the scanpaths to convert the 360° image into the unique GSR, which provides a global overview of gazed-focused contents derived from scanpaths. As such, the quality inference of the 360° image is swiftly transformed to that of GSR. We then propose an efficient OIQA computational framework by learning the quality maps of GSR. Comprehensive experimental results validate that the predictions of the proposed framework are highly consistent with human perception in the spatiotemporal domain, especially in the challenging context of locally distorted 360° images under varied viewing conditions. The code will be released at https://github.com/xiangjieSui/GSR.

Implicit Neural Representations (INRs) represent data as continuous functions using the parameters of a neural network, where data information is encoded in the parameter space. Therefore, modeling the distribution of such parameters is crucial for building generalizable INRs. Existing approaches learn a joint distribution of these parameters via a latent vector to generate new data, but such a flat latent often fails to capture the inherent hierarchical structure of the parameter space, leading to entangled data semantics and limited control over the generation process. Here, we propose a ontrollable ierarchical mplicit eural epresentation () framework, which explicitly models conditional dependencies across layers in the parameter space. Our method consists of two stages: In Stage-1, we construct a Layers-of-Experts (LoE) network, where each layer modulates distinct semantics through a unique latent vector, enabling disentangled and expressive representations. In Stage-2, we introduce a Hierarchical Conditional Diffusion Model (HCDM) to capture conditional dependencies across layers, allowing for controllable and hierarchical data generation at various semantic granularities. Extensive experiments across different modalities demonstrate that CHINR improves generalizability and offers flexible hierarchical control over the generated content.

With the rapid development of immersive multimedia technology, the growing demand for high-quality visual experiences has driven the emergence of point cloud quality assessment (PCQA). While current deep learning-based PCQA models have achieved breakthroughs in performance, problems such as high computational complexity and limited model generalization ability still need to be solved. In this study, focusing on compression distortion, we analyzed and verified that the compression quantization parameter (QP) can be used as a key feature for predicting perceptual quality. Based on this, a novel no-reference point cloud perceptual quality assessment metric, DQP-PCQA, is proposed. Unlike existing PCQA models that only use mean opinion score (MOS) as a supervisory label, this study proposes a multi-objective constrained optimization scheme that adds geometric quantization parameter (GQP) and texture quantization parameter (TQP) as auxiliary supervisory labels to help the model can learn robust perceptual features that take into account both subjective quality and objective distortion. We conducted comparative experiments with other advanced PCQA models on several mainstream PCQA datasets. The results show that the DQP-PCQA model achieves fast convergence speed, excellent and stable performance, low complexity and strong generalization. Further migration experiments show that after applying our proposed method to other advanced PCQA models, the performance of the improved model is further improved. Our discovery provides new insight for PCQA research. To facilitate future reproducible research, the source code will be publicly released at https://github.com/Dds46/DQP-PCQA.