Human activity is a fundamental component of intelligent interactive systems. Existing human activity recognition (HAR) follows a closed-ended assumption—defining a fixed set of activity classes and classifying each sensor signal into one of them—yet real-world human behavior is complex and ever-changing and thus cannot be fully represented by any fixed set. We present a new paradigm for open-ended human activity understanding (HAU), shifting the goal from closed-set classification to understanding activities via their meta-motions. To achieve this, we design MOSAIC, a motion decomposition and semantic fusion framework that transforms raw sensor streams into structured meta-motion representations and subsequently generates fine-grained natural-language descriptions. Additionally, we propose an LLM-driven training strategy for sensor-language alignment, enabling effective cross-modal supervision in the absence of real-world training data. Finally, we leverage LLM-assisted reasoning to recognize complex high-level activities based on the understanding of underlying motions and their composition. Our model demonstrates strong generalization and performance across 18 public HAR datasets, outperforming the best baseline by up to 19.6% in unseen scenarios.
Understanding Cognitive Load (CL) is essential to advance our knowledge of human cognitive resources and improve performance in demanding environments. Studying CL helps prevent overload and improve learning, productivity, and well-being in daily and professional life, which is particularly useful in the context of human computer interaction. This study presents a comprehensive multi-modal dataset from 55 participants across two sites, examining CL under relaxed, easy, and difficult task conditions. The dataset features synchronized, high-resolution recordings from multiple devices: Emteq OCOsense smartglasses for facial and head measurements; Empatica E4 wristband for heart rate variability, skin temperature and electrodermal activity; Tobii eye-tracker for gaze dynamics; and camera recordings (depth, RGB, and thermal) for behavioral and thermal measurements. Participants also provided subjective CL ratings, enabling richer analysis by integrating self-reports with sensor data. To validate data quality, the study provides statistical analyses of perceived difficulty and task performance. Machine learning experiments for CL estimation include person-dependent, person-independent, and task-independent models, highlighting the dataset's potential for assessing model generalization. Feature importance analysis across models identifies the most informative signals for CL modeling. This resource supports interdisciplinary research, enabling applications from CL modeling and machine learning personalization, to adaptive interface design and aims to advance CL assessment through robust, reproducible methods.
Photoplethysmography (PPG) has become a cornerstone of physiological sensing in wearable devices, enabling non-invasive monitoring of heart rate and related biomarkers. However, its reliability deteriorates sharply under dynamic, high-intensity, or non-periodic motions such as those in sports, where existing datasets fail to capture realistic wrist dynamics. To address this gap, we introduce PPG-Sport, the first large-scale dataset designed for heart rate monitoring from wrist-worn PPG under real sports conditions. The PPG-Sport dataset includes synchronized PPG, inertial measurement unit (IMU), and electrocardiography (ECG) recordings from both wrists of 30 participants across six representative activities: stationary, walking, running, badminton, table tennis, and basketball, amounting to 48 hours of multimodal data. PPG-Sport uniquely captures three critical properties absent in prior datasets: (1) non-periodicity, reflecting irregular and broadband motion patterns; (2) high intensity, with frequent, large-magnitude accelerations; and (3) bilateral asymmetry, caused by distinct functional roles of the dominant and non-dominant hands during sports. We further establish a deep-learning-based benchmark that combines both temporal and spectral representations of PPG and IMU signals to evaluate heart rate estimation performance. Experimental results show that models trained only on conventional periodic activities fail drastically in sports scenarios. Although incorporating sports data mitigates the degradation, significant errors remain. These findings highlight PPG-Sport as an essential and challenging benchmark for developing motion-robust physiological sensing algorithms during sports activities. Dataset and benchmark code are available at: https://github.com/LaserHu/PPG-Sport.
Prolonged desk work has become a major source of musculoskeletal strain, leading to fatigue, discomfort, and reduced mobility among sedentary professionals. Although prior interventions such as standing desks and exergames promote movement, most overlook cumulative postural imbalance and environmental constraints that shape ergonomic risk. We present Adaptive Stretching Pipeline (ASP), a mixed-reality (MR) system that delivers personalized stretching guidance during work breaks. ASP continuously monitors posture through joint-angle and duration analysis and leverages a vision-language model (VLM) to assess workspace supports such as desks or walls. Based on an integrated need-scoring model, the system generates adaptive MR stretching sessions targeting fatigued body regions during microbreak. A three-phase formative study informed the system design, including expert interviews on clinical stretching practices, workplace observations to identify environmental supports, and stretch video analysis to construct a stretching library. A user study with 15 participants compared ASP to a Bend and randomised condition. Our results provide initial evidence that such adaptive mechanisms can shape perceived effort, engagement, and time perception in short-term interactions.
Smart homes are receiving growing interest in global markets. However, current smart home assistant systems either control appliances solely through users' explicit commands or rely on cloud-based large language models (LLMs) for vague command understanding, which brings drawbacks such as high latency, privacy concern, and high cost. In this work, we propose VCU-LLM, the first system to deploy LLMs on edge devices for local vague command understanding and smart device control plan generation. VCU-LLM introduces a novel vague command knowledge retrieval algorithm that refines device-related information in the input prompt, thereby accelerating the LLM's on-device inference and reducing task complexity. We further construct a dataset for LLM fine-tuning to simulate the use of smart home assistants in controlling devices across different households. During inference, a customized KV-cache technique is applied for further inference acceleration. Our evaluations with both human-based and LLM-based scoring demonstrate that VCU-LLM improves the quality of generated control plans by an average of 43.3% compared with SOTA baselines, while reducing time overhead by an average of 8.44x compared with other on-device baselines. We also implement VCU-LLM through a case study in a real home environment, demonstrating its feasibility in real-world application.
