Workplace stress significantly affects employee well-being and task efficiency yet current stress recognition approaches remain impractical for real-world deployment. Existing methods mostly depend on intrusive physiological signals which require sensors attached to the body. Inspired by how humans naturally infer stress through observable behavior this PhD proposes a non-intrusive multimodal deep learning approach. This method uses overt expressive cues combining facial expressions to identify emotional valence with voice tone to capture emotional intensity and eye movements to monitor mental fatigue. To ensure reliability the system incorporates facial thermal imaging as a covert but non-intrusive cue to detect involuntary heat changes that reveal genuine stress even when a user attempts to mask their outward expression. The project develops a framework where contact-based physiological signals and user self-assessments are captured as a dual ground truth reference. By using these internal signals as a benchmark during the training phase the predictive model can learn to identify stress using only remote modalities. A critical component of this work is the development of a synchronized multimodal recording platform that ensures the accurate alignment of diverse data sources. This synchronization is essential for training reliable models that can predict stress levels in real-world settings without requiring the physical sensors. The expected outcomes include a synchronized recording platform and a multimodal dataset with aligned behavioral and physiological labels. By shifting the focus toward a combination of observable behavior and remote thermal monitoring this project seeks to advance practical multimedia systems that support workplace well-being.
Delivering immersive 6-DoF experiences over capacity-constrained networks requires balancing visual fidelity, interactive capabilities, and streaming efficiency. Existing polygonal mesh or 3D Gaussian Splatting (3DGS) approaches exhibit fundamental trade-offs among these requirements, whereas prior hybrid methods have largely focused on reconstruction quality rather than end-to-end delivery. In this work, we introduce LMG, the first end-to-end streaming system utilizing a Layered Mesh-Gaussian representation for the efficient and interactive delivery of static 3D scenes. Our model organizes textured meshes into a streamable base layer to ensure structural interactivity and attaches surface-aligned Gaussians into an enhancement layer for high-fidelity, photorealistic rendering. To enable effective deployment, we develop a complete system comprising a training server, a streaming server, and an XR client. The training server features a joint training pipeline with a custom hybrid rasterizer and budget-aware allocation algorithms that concentrate Gaussians in perceptually important regions. The streaming server employs a layered and tile-based pipeline tailored for LMG, prioritizing the delivery of the base layer for a quick scene overview, followed by the progressive refinement of visible regions using Gaussians. Evaluations of our systems demonstrate that the LMG training server significantly outperforms the state-of-the-art baseline in visual quality-improving PSNR by up to 10.99 dB (8.89 dB on average) and SSIM by up to 0.293 (0.255 on average). Furthermore, our results highlight the efficacy of the LMG streaming system, showing that layered and tiled streaming yields quality improvements of up to 19.81 dB in PSNR (5.80 dB on average), 0.143 in SSIM (0.06 on average), and 40.86 in VMAF (16.06 on average), while enabling low-latency and interactive XR experiences.
In HTTP Adaptive Streaming (HAS), the bitrate ladder and the Adaptive Bitrate (ABR) algorithm are two fundamental components that jointly determine user Quality of Experience (QoE). While prior work has extensively studied ABR algorithms, bitrate ladder design remains largely driven by encoding efficiency, with limited understanding of how the selection and distribution of bitrate levels affect client-side QoE. Besides, existing approaches typically optimize bitrate ladder at the encoder side and apply it uniformly across players, ignoring the diverse and dynamic network conditions faced by individual players during playback. We characterize how bitrate ladder structure affects QoE across diverse networks, showing that providing more bitrate levels does not necessarily improve QoE and that no single static ladder is globally optimal. Motivated by these findings, we present ReLadder, a practical system that dynamically adapts the set of available bitrate levels at the player side based on runtime network conditions. By integrating encoding efficiency with network dynamics, ReLadder enables ABR algorithms to operate within a more suitable decision space and improve the overall QoE. We evaluate ReLadder through large-scale trace-driven simulations using multiple ABR algorithms. Results show that ReLadder consistently outperforms static bitrate ladders: in a global network environment, it reduces rebuffer ratio by 1-10%, lowers average bitrate per-session by 100-530 Kbps, corresponding to 1-8% CDN traffic savings, while maintaining comparable visual quality within 0.5 VMAF difference, allowing users to enjoy the same high-quality video with fewer interruptions and lower bandwidth consumption.
Shot Boundary Detection (SBD), which identifies scene (or "shot") changes, is a core step in video analysis pipelines such as summarization and highlight generation. Yet, it remains challenging in sports broadcasts because rapid camera motion, frequent camera switches, sport-specific transitions and graphic overlays often cause false detections and poor cross-domain generalization. In this paper, we address shot boundary detection in professional sports broadcasts, focusing on ice hockey and soccer. We propose a sports-oriented model based on a fine-tuned R(2+1)D 3D CNN, trained to detect hard cuts, gradual transitions, and logo-based replay effects. The model is evaluated on both goal-centered clips and full-match broadcast footage, and is benchmarked against two state-of-the-art baselines: TransNetV2 for accuracy and PySceneDetect for efficiency. Our approach consistently achieves higher precision, recall, and F1-score across all evaluation settings, while demonstrating strong cross-league generalization within ice hockey and cross-sport generalization to soccer. We release our pretrained sports-specific SBD model as an open-source Python package, enabling straightforward integration into existing video analysis pipelines.
Chinese landscape painting seeks a poetic harmony between humanity and nature, an ideal encapsulated by the concept of "wandering within the painting." However, its appreciation, whether in traditional museums or through digital reconstructions, often remains a passive, visually oriented experience. In response, Ink Voyage draws on this classical aesthetic to create an immersive and interactive system. Utilizing inertial motion capture, it allows participants to navigate a virtual boat through digitally reimagined ink landscapes. This project transforms passive viewing into embodied exploration, enabling a more natural and active encounter with the spatial poetics of Chinese landscape art.
