In recent years, various studies have focused on developing and refining image-processing pipelines for RGB-IR imagery. In this paper, we address a central bottleneck in such pipelines: the removal of infrared (IR) contamination from the RGB channels. The IR crosstalk leakage leads to colour degradation, desaturation, and hue shifts, especially under high IR illumination. We study the spatial characteristics of this crosstalk in controlled laboratory conditions across diverse illumination intensities, and introduce a lightweight U-Net-based convolutional model with a task-specific loss that prioritizes spatially challenging, IR-contaminated pixels. Building on the standard linear contamination hypothesis, our model predicts per-channel coefficients that can integrate with conventional ISP stages, while adapting the correction to spatial outliers such as radial vignetting and edge artefacts.
In remote deception detection scenarios, where data are scarce, modalities are restricted to audio and video, and facial cues are subtle and highly individual, modeling expressions via Action Units (AUs) offers a focused and interpretable approach. This paper investigates why many methods still rely on OpenFace for AU prediction and compares the performance of CNN-based AU prediction across different scenarios. For this study, we acquired a new dataset collected in a controlled setting where participants were instructed to deceive during mock online sales meetings. For comparison, we also use the Real-life Trial dataset. We first demonstrate that our CNN-based approach outperforms OpenFace on the EmotioNet AU dataset. When used as a feature set for deception detection, the CNN approach also performs better on the challenging, in-the-wild data of the Real-life Trial. However, OpenFace, that covers also AUs not included within EmotioNet, performs better in the semi-controlled mock sales meeting scenario.
Interpretability is essential in security-critical applications such as Morph Attack Detection (MAD), where model transparency supports trust, auditability, and bias assessment. This work investigates the use of occlusion mapping to explain the decision process of a Vision Transformer (ViT)-based MAD system combined with a Support Vector Machine (SVM) classifier. Two complementary occlusion strategies were explored: landmark-based occlusion, targeting semantically relevant facial regions (eyes, nose, mouth), and grid-based occlusion, dividing images into spatial cells. Experiments were conducted on cropped and aligned versions of the FRLL-Morphs and FFHQ-Morphs datasets. The results, visualized through heatmaps, reveal that the model consistently assigns highest importance to the nose and mouth regions, with strong sensitivity to depth transition areas prone to morphing artifacts. These findings provide actionable insights for improving the robustness and bias mitigation of MAD systems.
As Virtual Reality technologies become increasingly accessible, their application within the educational and cultural heritage sectors is rapidly advancing. This study presents a multiuser Social Extended Reality application designed for guided virtual tours of a historically reconstructed site: the presbytery of the Church of Santa Maria Antiqua in Rome. The experience integrates spatial audio, avatars, and multilingual narration to promote immersive and social learning. Fifty-two participants took part in the tour in groups of two or three, accompanied by a guide. Using a mixed methods approach, we evaluated simulator sickness, usability, social presence, learning outcomes, and spatial behavior based on proxemics theory. The results show that the users maintained interpersonal distances consistent with the social zone, aligning with real-world patterns. The comprehension scores did not vary significantly with language familiarity, suggesting that visual and spatial cues effectively supported understanding. Overall, the application was well received, with favorable usability and presence ratings, highlighting the potential of multilingual Social Extended Reality as a tool for accessible and engaging cultural education.
The usage of advanced computational methods in clinical diagnostic procedures is a long-studied field. In this area, deep learning models show the highest degree of performance, but they require large amounts of annotated training data to achieve that performance. However, the process of annotating biomedical data requires time and is expensive since it must be done by medical professionals. For example, synchrotron X-ray tomographic imaging has been used for endomyocardial biopsy (EMB) sample scanning, which greatly increases resolution and clarity of acquired samples compared to other tomographic imaging techniques. The problem is that in this case, the amount of annotated data is several orders of magnitude smaller than the amount of raw data. This is especially evident in the area of image segmentation, where class annotations need to be accurately done for each pixel in the image, greatly increasing the annotation effort needed. Recent advances in foundational segmentation models offer us a way to partially automate the segmentation using limited oversight, allowing for the creation of partially supervised segmentation masks, especially in volumetric data such as CT or synchrotron X-ray images. In this work, we present a method for robust tissue segmentation in synchrotron X-ray images of endomyocardial biopsy samples in the case of a sparse set of tissue annotations. We employ a large foundational segmentation model from the Segment Anything 2 family to create a set of annotations on a small subset of tissues, using only limited supervision on a few key frames. We then use those frames to propagate annotations through a tissue volume, allowing us to get a large set of initial noisy annotations. These annotations are then used to train a smaller deep segmentation model. The final results are compared against the small set of annotations done by medical professionals to determine model performance. These results show that this approach greatly improves performance when compared to traditional image processing techniques or to models trained only on a small set of professionally annotated images.
The development of detailed fascial tissue models is an essential aspect in research of medical suture failure. These models can be created on the basis of tensile test data and require accurate deformation measurement. Typically, this is done by applying colored dots or a speckle pattern to simplify the tracking with digital image correlation (DIC). This method leads to time-consuming preparation of the test objects and a computationally intensive analysis. This paper presents an alternative approach to automated strain analysis, using opensource tracking algorithms to measure the deformation of natural textures on fascial tissue. As part of the evaluation, a straindependent error in the displacement of individual tracking points was identified using artificial videos of idealized stretched fascial textures. By automating and simplifying the deformation measurement, the described approach enables researchers to easily investigate various different fascial tissue samples in order to generate more characteristic data and improve existing models.
Denoising speech signals can be a difficult task especially if the background noise is non-stationary. Deep Neural Network (DNN) models provide an excellent performance in speech enhancement, yet they require a substantial computational effort during the inference process, as well as an enormous data basis. We propose a novel approach to optimize real-time supervised learning algorithms semi-supervised by adding a discriminator to the optimization process as well as a non-feature based objective function with phase awareness. We use a Convolutional Recurrent U-net for Speech Enhancement (CRUSE) to build up a conditional generative adversarial training framework. To reduce the computational complexity we train the generator exclusively on the magnitude spectrum and the metric discriminator returns the influence of the disturbed phase. Our findings show that CRUSE-cMGAN outperforms the existing supervised trained CRUSE based on the same architecture. Furthermore the semi-supervised learning manner allows a reduced training database for the optimization process. Experimental results are provided to prove the advantages of the conditional metric GAN framework.
Automatic pollen detection has recently gained significant research attention due to its importance for public health. Numerous computer vision methods have been proposed for this task, with the majority relying on two-stage object detection models applied to optical microscopy images. However, twostage approaches generally suffer from higher inference times, making them less suitable for real-time pollen monitoring tasks. Moreover, model performance on measurements from outdoor data samples remains largely unexplored because they introduce additional challenges, including visually complex backgrounds and the presence of other non-pollen particles such as dust, spores, and debris. In this work, we investigate the limits of the one-stage detection model YOLOv11 on Ambrosia pollen detection in images of outdoor and laboratory-prepared samples, acquired with the AeroTape device. We systematically evaluate training strategies, analyzing the effects of using either laboratory or outdoor images, their combinations, and also the use of data augmentation. The results show that incorporating augmented lab-prepared pollen images into training achieves the best mAP@50-95, highlighting the benefits of controlled laboratory data and one-stage detectors for improved generalization.
Precise segmentation of hepatic vascular structures is a critical prerequisite for liver surgery planning, transplant evaluation, and interventional procedures. However, publicly available datasets offering complete and consistent vessel annotations remain extremely limited. The 3DIRCADb dataset is widely used for liver imaging research, but the vascular annotations it provides are partial and heterogeneous, impeding effective deep learning model training. In this work, we present 3DIRCADb-Vessel+, an open and standardized extension of the 3DIRCADb dataset, specifically focused on hepatic vessels. We provide manually curated binary segmentation masks that combine the hepatic arteries, portal vein, hepatic veins, and inferior vena cava into a unified and consistent format. These masks were obtained by merging and refining the existing individual vessel annotations available in the original dataset. In addition to this dataset resource, we propose an enhanced 3D segmentation architecture based on 3D U-Net, in which we integrate a residual block to improve feature propagation and capture fine vascular structures more effectively. This adaptation is particularly beneficial for the recovery of small and low-contrast vessels, which remain challenging for conventional architectures. Together, the proposed dataset and the improved architecture aim to support the development of robust and generalizable liver vessel segmentation models, addressing a key bottleneck in open and reproducible research on vascular analysis in abdominal CT.
Current day vehicles are equipped with a high number of cameras in order to assist the driver. Each camera consists of an image signal processing pipeline, which is responsible for creation of digital images. Color correction, a fundamental step in many image signal processing pipelines, is especially important because of the dynamic nature of driving environments. In this work, we explored many different deep neural network architectures and observed that they are well suited for the task of color correction in driving environments. Instead of estimating illumination, we generate the color correction matrix directly from the given image, simplifying the image signal processing pipeline. We tested our methods on our own dataset that was made in collaboration with Xylon, an electronics company focused on the design of flexible automotive data logging solutions, which was recorded in urban driving environments.
Radiomics, the extraction of features from radiographic images using data-characterization algorithms, has gained considerable attention in prostate cancer research. Traditionally, manual segmentation conducted by experts is regarded as the gold standard. However, this method is time-consuming, subjective, and susceptible to variability, affecting the performance of classification models. Deep learning-based segmentation presents a promising alternative by offering fast and reproducible results but the impact of this approach on radiomics-based classification models remains unexplored. In this study, we evaluated the differences between manually drawn contours from an External Beam Radiotherapy (EBRT) planning system with automatically generated You Only Look Once (YOLO) segmentations and their effects on radiomic-based prostate cancer classification. Radiomic features were extracted from both segmentation approaches, followed by feature selection and classification using 14 different machine learning models combined with 7 feature selection techniques. Performance was assessed through Area Under the Receiver Operating Characteristic (AUC) and Accuracy (ACC) metrics with five-fold cross-validation. Results showed that for manual segmentation, the best model used Percentile feature selection with a Bagging(ExtraTree) classifier, achieving an AUC of 0.78 +/- 0.12. For YOLO-based segmentation, the best model employed Variance Threshold feature selection with a Gradient Boosting classifier, obtaining an AUC of 0.75 +/- 0.04. These findings suggest that automatic segmentation with YOLO yields comparable classification performance to manual delineation while improving efficiency and reproducibility.
Recent Weakly-Supervised Video Anomaly Recognition approaches still face critical limitations, mainly due to two reasons: (1) Existing methods often struggle to capture both subtle local changes and long-range temporal dependencies, which are essential to understand complex anomaly dynamics; (2) The semantic information is constrained to class names or a small number of learnable tokens, which is not discriminative enough. To address these issues, we propose ESTER-VAR, a novel architecture composed of two main modules: The Temporal Adapter that captures both short- and long-range temporal dependencies through local and global patterns, sequence modelling, and multiscale temporal information; and the Textual Module, which combines LLM-generated anomaly descriptions with learnable prompts to produce more discriminative anomaly information. Experiments conducted on the XD-Violence and UCF-Crime datasets demonstrate state-of-the-art performance in fine-grained anomaly recognition and competitive results in coarse-grained anomaly detection. Specifically, our framework achieves 84.42% AP and 26.55% AVG mAP on XD-Violence, and 88.06% AUC and 7.49% AVG mAP on UCF-Crime.
Immersive (XR) technologies are rapidly expanding, but their evaluation poses significant challenges. Standard User Experience (UX) questionnaires, such as the UEQ, fail to capture immersive qualities, while presencespecific tools, such as the PQ, are often verbose and, when used in combination, risk inducing survey fatigue. This paper addresses this methodological gap by proposing a comprehensive, user-driven framework for the development of integrated evaluation tools. We illustrate this framework through the case study of the development of the eXtended User Experience Questionnaire (X-UEQ), a novel 12-item module. Its six dimensions-perceived realism, agency, sensory engagement, interface quality, active exploration, and selfevaluation of performance-are theoretically grounded in established presence models and empirically derived from a qualitative study with 25 participants. The main contribution is twofold: a replicable, user-centered methodology and the resulting validation-ready tool that provides a necessary bridge between traditional usability evaluation and immersive presence evaluation.
Artificial Bandwidth Extension (ABE) improves narrowband (NB) speech by restoring high-frequency components crucial for intelligibility and naturalness. We introduce a method integrating the constant-Q Transform (CQT) in a lightweight neural architecture framework. Departing from short-time Fourier transform (STFT) pipelines, our technique exploits CQT's logarithmic spacing and finer low-frequency resolution to better align with human audition. During preprocessing, NB bins are extracted from wideband (WB) recordings, then stacked and masked to produce a WB-shaped input, while targets derive from the CQT of the WB signals. A compact Multi-Layer Perceptron (MLP), MLPCQT, is trained to estimate the full WB magnitude spectrum. Phase is recovered through spectral folding with an inverse CQT (iCQT), enabling time-domain reconstruction. Evaluations on the TIMIT dataset show that MLPCQT outperforms Gaussian Mixture Models (GMMs) and STFT-based models in objective and subjective metrics, validating improvements in speech quality.
This paper introduces an optimal filtering framework, in the mean-squared-error sense, for signals defined on directed acyclic graphs (DAGs), enabled by the zero-padding construction. By expanding the original DAG with padding vertices, the extended shift operator becomes diagonalizable, which enables spectral filter design. The Wiener solution is derived in the spectral domain, and approximated in the vertex domain, using a polynomial graph filter fitted through least-squares. Numerical experiments demonstrate consistent signal-to-noise ratio (SNR) improvements across a wide range of noise levels. Statistical evaluation over multiple noise realizations confirms that the designed polynomial filters achieve average SNR gains of 4-6 dB, while also reducing mean-squared-error (MSE). These results validate the feasibility of optimal filtering on DAGs via zero-padding and demonstrate the effectiveness of polynomial vertex-domain implementations.
In this paper, we evaluate three AI-generated video models (Sora, CogVideoX and Text2Video-Zero) using both subjective quality assessment and objective quality measures. A dataset of videos is generated based on ten prompts per each model and evaluated by 15 human participants along two perceptual dimensions: overall visual quality and text-to-video alignment. Objective quality is assessed using a wide range of no-reference image, video, and text-to-video alignment measures. Our analysis reveals a significant gap between subjective scores and current objective measures, indicating that existing tools are insufficient for reliably evaluating AI-generated video content. The results emphasize the need for improved objective quality assessment measures and frameworks of AI-generated video sequences. All subjective scores and tested video sequences are publicly available at http://aigvdataset.dynalias.com.
Training noise-aware speech enhancement models is inherently tricky due to the implicit coupling with the estimator of noise characteristics. Thereby, it is difficult to simultaneously investigate robust noise representations to encode noise-related information while independently optimising the enhancement network architecture for incorporating noise-awareness. The systematic effect of different noise representations on a given noise-aware architecture or the effect of different noise-aware architectures for a given noise representation can, therefore, not be easily studied. To decouple the two aspects, we first propose a novel, tunable data-generation paradigm to generate reference noise signals with varying degrees of accuracy - reflecting estimation errors expected in practice. Next, we incorporate this framework to train a straightforward noise-aware speech enhancement model. Through a systematic ablation study, we show that the performance of the noise-aware framework steadily improves with the accuracy of the background noise information. Also, when the noise-reference is minimally informative, the noise-aware framework performs on par with the baseline (noiseblind) speech enhancement model. Applying this trained noiseaware framework to realistic and real-world captures, we further demonstrate the generalisability of the trained model to even completely unseen and mismatched scenarios - convincingly establishing the validity of the data-generation paradigm. This work, thereby, paves the way for systematic exploration of the benefit of noise-aware speech enhancement, permitting simultaneous investigation into network architecture and noiserepresentations, without imposing a limiting interconnection between these two aspects.
Separating competing speakers in noisy and reverberant environments remains a fundamental challenge in speech signal processing. In this study, we present PEASE-Net (Phase-aware EAr-conditioned Separation Network), a DNN-based binaural, two-channel speaker separation system featuring ear-conditioning and a novel skip-connection design that bridges the raw input and decoder stages to improve reconstruction fidelity, and a regime without Permutation Invariant Training (PIT). The end-to-end model uses complex short-time Fourier transform (STFT) representations from the left and right channels, non-causal processing, and direct-path waveform targets to jointly perform separation and dereverberation for two speakers positioned at a fixed azimuth. Experiments were conducted on spatialized two-speaker mixtures under both anechoic and reverberant and noisy conditions in this study. Results demonstrate that the proposed system consistently outperforms Conv-TasNet, iFaSNet, and DPTNet baselines, achieving up to 23.1 dB SI-SDRi in anechoic and 18.2 dB in reverberant mixtures, with corresponding gains in STOI and PESQ. An ablation study further demonstrates that ear-conditioned skip connections, as proposed, provide the largest SI-SDRi and perceptual gains. These findings confirm that phase-aware, ear-conditioned modeling substantially improves separation fidelity and perceptual quality, offering a promising approach for robust binaural source separation.
Fault detection and diagnosis (FDD) is a cornerstone of predictive maintenance, preventing costly, unscheduled downtime by detecting incipient faults before they escalate. However, FDD for rotating machinery still hinge on where sensors are mounted, which axes are used, how long the input window is, and how vibration is transformed into images. Previous studies have generally fixed these factors rather than quantifying them. We close this gap with a 2x4x3x5 full-factorial evaluation on MAFAULDA, varying bearing location (inner/outer), sensor axis (axial, radial, tangential, tri-axial), input length (256/1024/4096), and Signal-to-Image Algorithms (STIM, GADF, MTF, RP, SDP) across 120 controlled configurations under a fixed ResNet-50 pipeline. We report both fault classification accuracy and processing time required for Signal to Image algorithms respectively. From that result, we analyze main and interaction effects, and derive actionable design rules. Results reveal factor level trends: tri-axial sensing outperforms single-axis inputs (with tangential strongest among singles); longer windows improve accuracy; STIM/GADF lead on average; and outer placement yields a modest gain. Latency-wise, STIM is most efficient and nearly length- invariant, GADF is moderate, while MTF/RP grow steeply with longer windows. Rather than emphasizing a single best configuration, this work provides the first systematic quantification of how sensing geometry and Signal to Image choices jointly affect FDD performance and cost, offering reproducible, evidence-based guidelines.
Cameras in vehicles must handle highly variable lighting, from bright sunlight to dark garages, making automatic exposure (AE) algorithms difficult to design and validate. Capturing the right lighting conditions for testing AE algorithms is often unpredictable, which motivates the usage of offline approaches. In this work, we study how exposure metadata can guide mapping between high dynamic range (HDR) and standard dynamic range (SDR) imagery. We first present a paired HDR-SDR dataset collected with synchronized automotive cameras in semi-open garage scenarios, together with per-frame metadata including exposure time, analog gain, and digital gain. To ensure spatial alignment, we then introduce a preprocessing pipeline based on uniform ORB feature matching and homography estimation across multiple frames. Finally, we propose a lightweight differentiable HDR to SDR tone mapping model that predicts mapping parameters directly from exposure metadata and is trained end-to-end using a differentiable histogram loss. Our results show that metadata can effectively guide tone mapping, providing a practical step toward offline analysis of AE behavior.
