As vehicles transition toward higher levels of automation, Driver Monitoring Systems (DMS) have become essential for ensuring human oversight, safety, and regulatory compliance in a vehicle. These systems rely on multimodal sensing and AI-driven inference to assess driver attention, cognitive state, and readiness to take control. While technologically promising, their deployment introduces a complex set of ethical and legal challenges - ranging from privacy and consent to data ownership and algorithmic fairness. While overarching frameworks such as the GDPR, EU AI Act, and IEEE standards offer important guidance, they lack the specificity required for addressing the unique risks posed by in-cabin sensing technologies. This paper adopts a review-to-design perspective, critically examining existing regulatory instruments and ethical frameworks -- such as the GDPR, the EU AI Act, and IEEE guidelines -- and identifying gaps in their applicability to the distinctive risks posed by multimodal, AI-enabled in-cabin monitoring. Building on this review, we propose a modular ethical design framework tailored specifically to Driver Monitoring Systems. The framework translates high-level principles into actionable design and deployment guidance, including user-configurable consent mechanisms, fairness-aware model development, transparency and explainability tools, and safeguards for driver emotional well-being. Finally, the paper outlines a risk analysis and failure mitigation strategy, emphasizing proactive incident response and accountability mechanisms tailored to the DMS context. Together, these contributions aim to inform the development of transparent, trustworthy, and human-centered driver monitoring systems for next-generation autonomous vehicles.

Federated learning (FL) enables privacy-preserving video human activity recognition without centralizing data. Despite its potential, FL remains fundamentally constrained by the high communication overhead incurred during training. Existing methods, such as model quantization, gradient sparsification and low-rank approximation, aim to mitigate this overhead but often at the cost of model expressivity, leading to reduced classification accuracy. In this work, we take a representation-centric perspective and show that for video-base activity recognition using pretrained embeddings, class-discriminative information lies in a very low-dimensional subspace, that preserves linear seperability between activity classes. Motivated by this observation, we propose Federated discriminative subspace discovery, a framework that collaboratively identify and operate within such a subspace. Instead of communicating full classifier updates, each client first projects its local video embeddings using a shared random projection matrix. It then computes the covariance of the projected embeddings and transmits this statistic to the server for aggregation. The server reconstructs a global covariance matrix, performs singular value decomposition, and selects the top-k eigenvectors that define the global class-discriminative subspace. Subsequent communication occurs only in this compact subspace. Extensive experiments on UCF101, HMDB51, and Toyota-SmartHome demonstrate that, learning and sharing the classifier in the discovered subspace, reduces communication costs by up to 61%, with a moderate drop in activity recognition accuracy compared to the full model.

Neuromorphic vision sensors offer significant advantages for real-time embedded perception due to their microsecond latency, high temporal resolution, wide dynamic range, and low power consumption. However, deploying multi-stage event perception pipelines onto edge hardware remains fundamentally constrained by memory, throughput, and execution bottlenecks on resource-limited silicon. In this work, we introduce an end-to-end, real-time perception pipeline deployed on a Raspberry Pi 5 platform, establishing a hardware-aware reference baseline for future Neural Processing Unit (NPU) and neuromorphic architectures. Our system integrates a custom attention-enhanced YOLOv8-nano model for joint person and face detection, a parameter-free ByteTrack multi-object tracking framework, and a lightweight kinematic feature extraction engine for behavioral triage and velocity-based activity classification. To evaluate localized spatial trade-offs—particularly critical for in-cabin automotive application domains such as Driver and Occupant Monitoring Systems (DMS/OMS) we systematically quantify the sensor bias configurations and optical geometries (76° vs. 104° Fields of View) across detection accuracy, temporal consistency, face region extraction, and kinematic activity mapping. The detection backbones are fine-tuned on a custom indoor event dataset and exported using static ONNX execution graphs optimized for edge deployment. Operating strictly on sparse spatiotemporal event representations without intermediate RGB frame reconstruction, our pipeline maintains robust, privacy-responsive performance under extreme lighting and motion dynamics, sustaining 19–24 FPS end-to-end throughput. By providing complete quantitative and qualitative benchmarks, this framework establishes a reproducible baseline for low-latency edge-AI deployment in automotive cabin monitoring, autonomous robotics, and assistive healthcare. The project details, fine-tuned models, empirical results, and curated event datasets are publicly available at https://mali-farooq.github.io/NeuroVision.

Microsaccades – rapid, fixational eye movements typically smaller than 1° – play a crucial role in maintaining visual stability and are increasingly recognized as indicators of internal neurological states. Classical studies and recent evidence indicate that microsaccade timing and rate are closely linked to attentional shifts and task demands, highlighting their functional significance. However, methodological inconsistencies in definitions and detection parameters continue to challenge interpretation. Converging findings across behavioral, neural, and computational studies support the use of microsaccades as precise, noninvasive probes of perception, attention, and cognition. This review synthesizes decades of research along four main dimensions: the kinematics and definition of microsaccades, their neural control mechanisms, links to cognitive and affective processes, and methods for their detection and analysis, including emerging high-speed and event-based sensors. We also identify open challenges, including the need for standardized criteria, robust model-based detection at kilohertz rates, and advanced event-based tracking. Addressing these challenges will enable microsaccades to serve as a reliable window into the dynamics of neural and cognitive processes. We conducted a systematic search to identify papers for this review. Sources: Google Scholar, Scopus, PubMed. Search terms included "microsaccade", "fixational eye movements", AND "eye tracking", and combinations with "detection", "Engbert", "Bayesian", "event camera" "SPAD". We included peer-reviewed articles, conference proceedings, and publicly available benchmarks. Inclusion: empirical studies reporting microsaccade kinematics, neural mechanisms, detection algorithms, or cognitive/clinical applications; review/editorial pieces only when they provided synthesis. Exclusion: studies that did not report primary microsaccade data or were peripheral to fixation (e.g., large saccade-only papers). This procedure was chosen to maximize coverage of influential works while maintaining transparency for replication.

Objective. This driving simulator experiment was conducted to examine the effect of prolonged automation on fatigue during conditionally automated driving (CAD). Driver fatigue, which can be distinguished from drowsiness and already accounts for as many as 20% of all road traffic accidents, is likely to remain a pervasive problem during CAD, as reductions in attention and vigilance due to fatigue could imperil safe transitions of control between automated system and user. While considerable research exists concerning drivers' responses under states of drowsiness or distraction, fewer studies have investigated the effect of automation on fatigue.Approach. Drivers' self-reported fatigue and workload, physiological responses, and takeover performance were examined across three driving conditions: a baseline period of manual driving, an automated driving condition in which drivers interacted with theN-back task, and a 50 min automated drive with no secondary task.Main results. Findings show that fatigue was significantly higher following 50 min of automated driving than at baseline, or while participants performed a non-driving-related task. Strikingly, 80% of participants experienced signs of sleepiness, and almost half had to exercise effort to stay awake. Fatigue also resulted in decreases in heart rate (HR) and relative beta power derived from electroencephalography, and an increase in blink rate and HR variability.Significance. Overall, the findings advance knowledge in this area by supporting the idea of fatigue as failure to adequately self-regulate during automation. Several physiological measures have also been identified as possible markers of fatigue to inform emerging monitoring technology.

Traditional OCR and Automatic Number Plate Recognition (ANPR) systems typically perform well under controlled conditions, such as clean, well-lit, high-resolution environments. However, their reliability diminishes in real-world scenarios involving visual degradations like dust, occlusions, rust, glare, or faded characters. To address this, we present a novel imaging pipeline that synthetically renders number plates under challenging conditions using diffusion-based generative models. Our approach integrates edge-conditioned ControlNet with inpainting-driven diffusion to generate spatially precise and semantically realistic distortions. The process is guided by using tailored text prompts generated via a large language model, enabling contextual degradation synthesis. A hybrid architecture combines SDXL and a stable inpainting backbone to operate on full-frame vehicle images, capturing variability in viewpoint, distance, and background clutter. We validate the effectiveness of our synthetic dataset by evaluating the performance of a mature OCR engine (EasyOCR), showing that standard models struggle with complex distortions—highlighting the potential of our method for training and benchmarking robust OCR systems. The complete synthetic number plate degradation dataset and the fine-tuned model, is available at (https://github.com/MAli-Farooq/Prompt-Guided-Diffusion-Inpainting-for-License-Plate-Degradation/).

This article presents a C3I-SynMicrosaccade dataset: a synthetic microsaccade dataset designed to enable event-based modelling and classification of microsaccadic eye movements. Using Blender, we generated high-resolution RGB sequences of microsaccades, characterized by small, transient eye rotations around a fixed head pose. Each microsaccade follows a horizontal-boomerang-like trajectory, simulating the natural back-and-forth displacement of the eye during visual fixation. Seven distinct angular classes, ranging from 0.5° to 2.0°, capture varying motion amplitudes while maintaining consistent scene, lighting, and texture conditions. The rendered RGB frames were converted into event-based data streams using the v2e simulator, which replicates the asynchronous behaviour of neuromorphic vision sensors. Temporal durations and event counts were carefully controlled and resampled to ensure class balance and eliminate bias toward motion magnitude. The resulting dataset comprises 175,000 event sequences (87,500 per eye), providing a large-scale, balanced foundation for microsaccade recognition, neuromorphic vision research, and synthetic-to-real transfer learning. This work offers a controlled, reproducible framework for studying fixational eye movements and evaluating event-based algorithms under fine motion dynamics.

ThermVision-DB presents a synthetic long-wave infrared (LWIR) facial dataset designed to support research in privacy-preserving vision, thermal perception, and multimodal facial analysis. The dataset builds upon generative diffusion models to create photorealistic thermal facial images and video sequences capturing controlled variations in facial expression and head pose. Each synthetic identity is generated using text-to-image conditioning followed by video retargeting module, enabling precise control over pose angles, expression intensity, and frame-to-frame consistency. The dataset includes a diverse set of synthetic adult identities of both male and female genders with multiple facial expressions - such as neutral, smile, frown, and surprise and head-pose rotations spanning yaw, pitch, and roll. Data are provided in both image and video formats, accompanied by face localization annotations, landmark detections and identity labels. To ensure reusability and scalability, all samples are generated through a standardized pipeline using open-source models, allowing researchers to easily expand the dataset with additional synthetic identities while maintaining consistent thermal appearance and scene illumination. The synthetic generation process avoids the use of any personally identifiable visual data, ensuring compliance with FAIR and GDPR principles.ThermVision-DB is intended for use in developing and benchmarking algorithms for facial detection, landmark localization, expression recognition, and head-pose estimation in the thermal domain. It also provides a foundation for research in synthetic-to-real transfer learning, privacy-safe biometric analysis, and cross-spectrum data fusion. The dataset is released for open research purposes under a non-commercial license, with full documentation and metadata available to facilitate reproducibility and integration with existing thermal vision benchmarks.

This paper presents a lightweight three-dimensional convolutional neural network (3D-CNN) for human activity recognition (HAR) using event-based vision data. Privacy preservation is a key challenge in human monitoring systems, as conventional frame-based cameras capture identifiable personal information. In contrast, event cameras record only changes in pixel intensity, providing an inherently privacy-preserving sensing modality. The proposed network effectively models both spatial and temporal dynamics while maintaining a compact and computationally efficient design. To address class imbalance and enhance generalization, focal loss with class reweighting and targeted data augmentation strategies are employed. The model is trained and evaluated on a composite dataset derived from the Toyota Smart Home and ETRI datasets. Experimental results demonstrate an F1-score of 0.9415 and an overall accuracy of 94.17%, outperforming benchmark 3D-CNN architectures such as C3D, ResNet3D, and MC3_18 by up to 3%. It also has approximately 10 times fewer parameters than the second best benchmark network. These results highlight the potential of event-based deep learning for developing accurate, efficient, and privacy-aware human action recognition systems suitable for real-world applications.

Event cameras record luminance changes with microsecond resolution, but converting their sparse, asynchronous output into dense tensors that neural networks can exploit remains a core challenge. Conventional histograms or globally-decayed time-surface representations apply fixed temporal parameters across the entire image plane, which in practice creates a trade-off between preserving spatial structure during still periods and retaining sharp edges during rapid motion. We introduce Locally Adaptive Decay Surfaces (LADS), a family of event representations in which the temporal decay at each location is modulated according to local signal dynamics. Three strategies are explored, based on event rate, Laplacian-of-Gaussian response, and high-frequency spectral energy. These adaptive schemes preserve detail in quiescent regions while reducing blur in regions of dense activity. Extensive experiments on the public data show that LADS consistently improves both face detection and facial landmark accuracy compared to standard non-adaptive representations. At 30 Hz, LADS achieves higher detection accuracy and lower landmark error than either baseline, and at 240 Hz it mitigates the accuracy decline typically observed at higher frequencies, sustaining 2.44

Time event cameras are a novel technology for recording scene information at extremely low latency and with low power consumption. Event cameras output a stream of events that encapsulate pixel-level light intensity changes within the scene, capturing information with a higher dynamic range and temporal resolution than traditional cameras. This study investigates the contact-free reconstruction of an individual's cardiac pulse signal from time event recording of their face using a supervised convolutional neural network (CNN) model. An end-to-end model is trained to extract the cardiac signal from a two-dimensional representation of the event stream, with model performance evaluated based on the accuracy of the calculated heart rate. The experimental results confirm that physiological cardiac information in the facial region is effectively preserved within the event stream, showcasing the potential of this novel sensor for remote heart rate monitoring. The model trained on event frames achieves a root mean square error (RMSE) of 3.32 beats per minute (bpm) compared to the RMSE of 2.92 bpm achieved by the baseline model trained on standard camera frames. Furthermore, models trained on event frames generated at 60 and 120 FPS outperformed the 30 FPS standard camera results, achieving an RMSE of 2.54 and 2.13 bpm, respectively.

Automating visual inspection in medical device manufacturing remains challenging due to small and imbalanced datasets, high-resolution imagery, and stringent regulatory requirements. This work proposes two attention-guided autoencoder architectures for deep anomaly detection designed to address these constraints. The first employs a structural similarity-based anomaly score (4-MS-SSIM), offering lightweight and accurate real-time defect detection, yielding ACC 0.903 (unsupervised thresholding) and 0.931 (supervised thresholding) on the - Surface Seal Image - Test split with only 10

Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) are key elements of speech based features and user interfaces (UI) for many modern consumer technologies, ranging from embedded AI assistants to AI chatbots and most recently the latest generation of wearables such as smart-glasses. Growing demand for privacy-preserving, speech-enabled consumer devices is pushing ASR, TTS and speaker verification from the cloud to the edge, where memory and compute budgets are tight. This paper introduces a single, joint speech-text architecture that unifies acoustic and textual representations in a shared embedding space and supports three downstream tasks: ASR, TTS and speaker recognition, without task-specific encoders. The model replaces the Connectionist Temporal Classification (CTC) decoder used in earlier work with an RNN-Transducer, boosting ASR accuracy while reducing overall parameter count to 86.08 M. Trained on 460 h of LibriTTS, it attains 10.8% word error rate (WER) / 3.96% character error rate (CER), 78.1% speaker-ID accuracy, and a 7.14 dB MCD with predicted mean opinion score (MOS) on naturalness of 2.9. On a Raspberry Pi 5 (CPU-only) the system runs close to real time (RTF 0.83 for ASR; 1.34 for TTS), demonstrating practical on-device deployment. Beyond the primary tasks, the learned embeddings achieve 90% audio-text retrieval accuracy on clean LibriTTS/Librispeech splits and 98.3% digit classification on AudioMNIST, highlighting their versatility. These results show that a single, compact model can deliver competitive multimodal speech performance while staying within the strict resource envelope of embedded hardware.

We propose SynAdult, a multimodal synthetic data generation framework designed to address the scarcity of diverse and privacy-compliant senior adult face datasets for biometric applications and facial analysis. Our pipeline begins with the rendering of high-fidelity 2D adult facial images using parameter-efficient LoRA-based tuning of the state-of-the-art hyperrealism Stable Diffusion XL (SDXL) model, producing photorealistic outputs across diverse ethnicities and age-specific features. Next, we integrate a video retargeting pipeline to synthesize temporally consistent head pose and facial expression sequences, ensuring naturalistic dynamics crucial for downstream video-based facial analysis. In the third stage, we generate neuromorphic event data to introduce a privacy-preserving modality aligned with real-world edge deployment scenarios, such as ambient monitoring and in-vehicle sensing, where high temporal resolution and minimal identity leakage are beneficial. Finally, we reconstruct detailed 3D facial meshes from single 2D frames using 2D-to-3D morphing techniques to capture fine-grained structural details. This modality enhances geometric understanding and supports applications in AR/VR and affective computing. To validate the robustness and utility of the generated dataset, we perform a comprehensive evaluation using Kernel Inception Distance (KID), BRISQUE, CLIP score, and identity similarity metrics. We further assess downstream applicability employing the state-of-the-art facial expression classification networks and event facial landmarks tests for downstream machine learning tasks. As a key contribution, we open-source a large-scale, multimodality, multi-race adult dataset, enabling future research in secure and ethically grounded synthetic data for facial biometrics and facial analysis applications. The project website, along with the complete adult multimodality dataset and the fine-tuned model, is available at https://mali-farooq.github.io/SynAdult/

As immersive 360 degrees video experiences through head-mounted displays (HMDs) gain widespread adoption, the need for real-time, fine-grained assessment of Quality of Experience (QoE) becomes increasingly critical for optimising user engagement and system performance. This paper introduces RCQoEA-360VR, a novel multi-modal dataset designed for continuous QoE evaluation in virtual reality (VR) environments. In a controlled study (N=32), participants watched five selected 360 degrees video sequences across eight different video quality configurations (from the VQEG database) using a Vive Pro Eye while providing continuous QoE annotations via a touchpad-based input method, enhanced by the DotMorph peripheral visualisation technique. The dataset also includes synchronised physiological signals (electrocardiogram and galvanic skin response), behavioural data (eye and head movements) and post-viewing QoE ratings gathered through a within-VR interface. RCQoEA-360VR addresses a critical gap in existing public datasets by providing a fine-grained, synchronised multimodal data for immersive QoE analysis. It offers a unique and valuable resource for the research community, supporting a wide range of research applications, including QoE prediction, behavioural modelling, adaptive streaming, and implicit perceptual analysis.

This study introduces a novel approach to enhance the spatial-temporal resolution of time-event pixels based on luminance changes captured by event cameras. These cameras present unique challenges due to their low resolution and the sparse, asynchronous nature of the data they collect. Current event super-resolution algorithms are not fully optimized for the distinct data structure produced by event cameras, resulting in inefficiencies in capturing the full dynamism and detail of visual scenes with improved computational complexity. To bridge this gap, our research proposes a method that integrates binary spikes with Sigma Delta Neural Networks (SDNNs), leveraging spatiotemporal constraint learning mechanism designed to simultaneously learn the spatial and temporal distributions of the event stream. The proposed network is evaluated using widely recognized benchmark datasets, including N-MNIST, CIFAR10-DVS, ASL-DVS, and Event-NFS. A comprehensive evaluation framework is employed, assessing both the accuracy, through root mean square error (RMSE), and the computational efficiency of our model. The findings demonstrate significant improvements over existing state-of-the-art methods, specifically, the proposed method outperforms state-of-the-art performance in computational efficiency, achieving a 17.04-fold improvement in event sparsity and a 32.28-fold increase in synaptic operation efficiency over traditional artificial neural networks, alongside a two-fold better performance over spiking neural networks.

In this work, we present an innovative approach utilizing ControlNet-based diffusion models along with upscaling capabilities for domain adaptation and quality refinement of 3D modelled synthetic datasets, focusing on autonomous vehicle applications. A significant domain gap often exists between synthetic and real-world data, hindering the applicability of deep learning models trained on synthetic data for real-world scenarios. Our methodology leverages the strengths of Controlled Augmentation by simultaneously utilizing multiple ControlNet signals, including edge detection, depth information, segmentation maps, and tile resampling. To improve how synthetic data aligns with the desired domain specifications, these signals guide the generative process, and we also incorporate text-guided prompts extracted via Large Language Models (LLMs), to improve control over the synthesis of desired features and attributes. We test the approach on diverse environmental conditions from the VKITTI dataset, a well-known 3D modelled synthetic dataset generated in Unity for autonomous driving research. The refined data is validated using quantitative metrics including FID, SSIM, and LPIPS, and is also evaluated on downstream machine learning tasks of object detection and classification, using YOLO-v8 to ensure its utility and effectiveness. Experimental analysis demonstrates the effectiveness of this method in improving the realism and usability of synthetic data. Our approach contributes to fields that require high-quality data synthesis and domain adaptation. The experimental work, along with ControlNet models used in this project is available online.

The evaluation of diffusion-based image sampling methods is pivotal in improving the quality and reliability of synthetic data generation, particularly in medical imaging applications. Medical imaging requires high precision and fidelity, as even subtle artifacts or inconsistencies can significantly impact clinical decision-making. This study examines the effectiveness of four different image sampling techniques across various medical imaging modalities, focusing on dermoscopic skin lesion data, computed tomography angiography for Type B Aortic Dissection, and chest X-ray imaging. By systematically assessing these methods, we aim to enhance the fidelity of synthetic datasets, ensuring they more closely resemble real-world clinical data thereby supporting more accurate diagnostics, treatment planning, and prognostic predictions. In this work, we evaluate the performance of four different sampling techniques by incorporating Euler, Euler A, Denoising Diffusion Implicit Mode (DDIM), and Pseudolinear Multistep (PLMS) approaches for medical image synthesis. The study utilizes quantitative metrics including Structural Similarity Index Measure (SSIM), and Learned Perceptual Image Patch Similarity (LPIPS) to assess the realism and structural integrity of the generated images. Additionally, we employed t-SNE visualization to illustrate the latent feature representations of rendered synthetic medical images, providing an intuitive understanding of the underlying structure. We also analyzed and compared the computational complexity associated with each image sampling technique, offering insights into the efficiency of different approaches. The generated medical images are available at Diffusion-Sampling-for-Medical-Image-Synthesis (GitHub Link: https://github.com/MAli-Farooq/Diffusion-Sampling-for-Medical-Image-Synthesis-).

This paper tackles the challenges of face detection, a vital computer vision task with wide-ranging applications, particularly in driver monitoring systems, where both accuracy and computational efficiency are crucial. Traditional frame-based methods often suffer from high computational complexity and under-sampling issues. In turn this limits their effectiveness in real-time applications and increase energy budget. To overcome these limitations, this paper explores the benefits of neuromorphic event cameras, which capture asynchronous pixel-level changes, offering lower data processing demands and reduced latency. The paper proposes a hybrid architecture combining Spiking Neural Networks (SNNs) and Artificial Neural Networks (ANNs). This approach leverages the energy efficiency and low-latency of SNNs while maintaining the high accuracy of ANNs, resulting in a highly efficient and accurate face detection system. The proposed Spiking-Face approach demonstrates a 57.76% improvement in mean average precision (mAP) over state-of-the-art methods. Additionally, the paper provides a comprehensive analysis of the system’s performance across different temporal resolutions, showing that the system performs robustly and adapts effectively to varying conditions. This hybrid SNN-ANN architecture achieves up to three times higher computational efficiency compared to equivalent traditional ANN methods, significantly reducing computational complexity while maintaining accuracy. These findings underscore the potential of this hybrid architecture for real-time, energy-constrained applications.

Understanding and mitigating flicker effects caused by rapid variations in light intensity is critical for enhancing the performance of event cameras in diverse environments. This paper introduces an innovative autonomous mechanism for tuning the biases of event cameras, effectively addressing flicker across a wide frequency range -25 Hz to 500 Hz. Unlike traditional methods that rely on additional hardware or software for flicker filtering, our approach leverages the event cameras inherent bias settings. Utilizing a simple Convolutional Neural Networks -CNNs, the system identifies instances of flicker in a spatial space and dynamically adjusts specific biases to minimize its impact. The efficacy of this autobiasing system was robustly tested using a face detector framework under both well-lit and low-light conditions, as well as across various frequencies. The results demonstrated significant improvements: enhanced YOLO confidence metrics for face detection, and an increased percentage of frames capturing detected faces. Moreover, the average gradient, which serves as an indicator of flicker presence through edge detection, decreased by 38.2 percent in well-lit conditions and by 53.6 percent in low-light conditions. These findings underscore the potential of our approach to significantly improve the functionality of event cameras in a range of adverse lighting scenarios.