Self-touch gestures (e.g., nuanced facial touches and subtle finger scratches) provide rich insights into human behaviors, from hygiene practices to health monitoring. However, existing approaches fall short in detecting such micro gestures due to their diverse movement patterns. This paper presents μTouch, a novel magnetic sensing platform for self-touch gesture recognition. μTouch features (1) a compact hardware design with low-power magnetometers and magnetic silicon, (2) a lightweight semi-supervised framework requiring minimal user data, and (3) an ambient field detection module to mitigate environmental interference. We evaluated μTouch in two representative applications in user studies with 11 and 12 participants. μTouch only requires three-second fine-tuning data for each gesture, and new users need less than one minute before starting to use the system. μTouch can distinguish eight different face-touching behaviors with an average accuracy of 93.41

Background Team leadership during medical emergencies like cardiac arrest resuscitation is cognitively demanding, especially for trainees. These cognitive processes remain poorly characterized due to measurement challenges. Using virtual reality simulation, this study aimed to elucidate and compare communication and cognitive processes-such as decision-making, cognitive load, perceived pitfalls, and strategies-between expert and novice code team leaders to inform strategies for accelerating proficiency development.Methods A simulation-based mixed methods approach was utilized within a single large academic medical center, involving twelve standardized virtual reality cardiac arrest simulations. These 10- to 15-minutes simulation sessions were performed by seven experts and five novices. Following the simulations, a cognitive task analysis was conducted using a cued-recall protocol to identify the challenges, decision-making processes, and cognitive load experienced across the seven stages of each simulation.Results The analysis revealed 250 unique cognitive processes. In terms of reasoning patterns, experts used inductive reasoning, while novices tended to use deductive reasoning, considering treatments before assessments. Experts also demonstrated earlier consideration of potential reversible causes of cardiac arrest. Regarding team communication, experts reported more critical communications, with no shared subthemes between groups. Experts identified more teamwork pitfalls, and suggested more strategies compared to novices. For cognitive load, experts reported lower median cognitive load (53) compared to novices (80) across all stages, with the exception of the initial presentation phase.Conclusions The identified patterns of expert performance — superior teamwork skills, inductive clinical reasoning, and distributed cognitive strategiesn — can inform training programs aimed at accelerating expertise development.

Acoustic sensors are now integrated into nearly every wearable device, valued for their affordability, low power consumption, and unobtrusiveness. By design, microphones and speakers are traditionally used for speech interactions and sound playback, respectively. However, recent advancements in artificial intelligence, advanced signal processing, and high-fidelity compact sensors have enabled researchers in Ubiquitous and Wearable Computing and HumanComputer Interaction (HCI) to significantly expand the sensing capabilities of these acoustic components. They are now being repurposed as minimally obtrusive, low-power, and privacy-aware sensing units on wearables to capture high-quality information about users and their surrounding environments. This information can be intelligently interpreted for seamless interaction, contextual awareness, health monitoring, and activity recognition-highlighting the tremendous potential of acoustic sensing for the future of wearable technologies. To fully explore the opportunities and challenges of applying acoustic sensing in real-world applications in the age of AI, this workshop invites researchers and practitioners from academia and industry to share insights, identify key challenges, and discuss emerging developments in intelligent acoustic sensing and interaction technologies for everyday wearable devices. Through collaborative discussion and exploration, participants will address critical issues and propose innovative solutions to advance the field. Topics of interest include, but are not limited to, acoustic sensing system development, open-source tools and datasets, signal processing, AI-driven approaches, privacy concerns, deployment challenges, and novel applications.

Enabling arbitrary surfaces to understand the properties and locations of objects in their surrounding environments can facilitate a wide range of applications, including collaborative tangible interactions and mixed-reality 3D interfaces. However, limitations of current tracking mechanisms, e.g., line-of-sight occlusion for vision-based approaches, requiring instrumenting the objects with tags for RF-based tracking, ultimately restrict their adoption. This paper introduces MagDeck, an object recognition and tracking approach based on passive magnetic sensing. Utilizing an array of 112 low-cost magnetometers positioned underneath a table, our custom-designed signal processing and machine-learning pipeline enables accurate and robust identification and localization of unmodified daily objects. MagDeck is able to classify 10 household and workplace objects in real-time with 99.0% accuracy while tracking them with a 3.27 cm average error. Through instrumenting a regular table surface, MagDeck presents a low-cost and accurate approach to detecting passive objects and a step forward toward context-aware computing in real-world environments.

Enabling computing systems to detect the objects that people hold and interact with provides valuable contextual information that has the potential to support a wide variety of mobile applications. However, existing approaches either directly instrument users' hands, which can reduce tactile sensation, or are limited in the types of objects and interactions they can detect. This work introduces HandSAW, a wireless wrist-worn device incorporating a Surface Acoustic Wave (SAW) sensor with enhanced bandwidth and signal-to-noise ratio while rejecting through-air sounds. The device features a sealed mass-spring diaphragm positioned on top of the sound port of a MEMS microphone, enabling it to capture SAWs generated by objects and through touch interaction events. This custom-designed wearable platform, paired with a real-time ML pipeline, can distinguish 20 passive object events with >99% per-user accuracy and a 91.6% unseen-user accuracy, as validated through a 16-participant user study. For devices that do not emit SAWs, our active tags enable HandSAW to detect those objects and transmit encoded data using ultrasonic signals. Ultimately, HandSAW provides an easy-to-implement, robust, and cost-effective means for enabling user-object interaction and activity detection.

Internal and external rotation of the shoulder is often challenging to quantify in the clinic. Existing technologies, such as motion capture, can be expensive or require significant time to setup, collect data, and process and analyze the data. Other methods may rely on surveys or analog tools, which are subject to interpretation. The current study evaluates a novel, engineered, wearable sensor system for improved internal and external shoulder rotation monitoring, and applies it in healthy individuals. Using the design principles of the Japanese art of kirigami (folding and cutting of paper to design 3D shapes), the sensor platform conforms to the shape of the shoulder with four on-board strain gauges to measure movement. Our objective was to examine how well this kirigami-inspired shoulder patch could identify differences in shoulder kinematics between internal and external rotation as individuals moved their humerus through movement patterns defined by Codman’s paradox. Seventeen participants donned the sensor while the strain gauges measured skin deformation patterns during the participants’ movement. One-dimensional statistical parametric mapping explored differences in strain voltage between the rotations. The sensor detected distinct differences between the internal and external shoulder rotation movements. Three of the four strain gauges detected significant temporal differences between internal and external rotation (all p < .047), particularly for the strain gauges placed distal or posterior to the acromion. These results are clinically significant, as they suggest a new class of wearable sensors conforming to the shoulder can measure differences in skin surface deformation corresponding to the underlying humerus rotation.

Accurate and responsive 3D tracking enables interactive and context-aware workspaces, including mixed reality 3D interfaces and collaborative tangible interactions. However, limitations of current tracking mechanisms - line-of-sight occlusion, drifting errors, small working volumes, or instrumentation that requires maintenance - ultimately restrict their adoption. This paper introduces MagDesk, an interactive tabletop workspace capable of real-time 3D tracking of passive magnets embedded in objects. Using a sensing array of 112 low-cost magnetometers underneath a table, our custom-designed signal processing and localization engine enables simultaneous tracking of multiple magnets in 5 degrees of freedom with millimeter accuracy. MagDesk can continuously and robustly track magnets at a maximum height of 600 mm over a 1750 mm (L) × 950 mm (W) table, while achieving an average positional and orientational error of 2.49 mm and 0.72 ° near the table surface and 14.40 mm and 2.25° across the entire sensing range. To demonstrate MagDesk's object-tracking capabilities, this work presents a series of magnetic widgets for tangible interactions and explores two applications - a 3D drawing interface and augmented-reality tabletop games. By instrumenting a regular table surface, MagDesk presents a low-cost and accurate approach to 3D tracking passive objects for home and office environments.

Internet-connected cameras support many useful home monitoring and health applications. However, these same cameras indiscriminately capture sensitive and Personally Identifiable Information (PII), limiting their acceptance in certain settings, such as the home. Prior works removed Region of Interest (ROI) to secure images and improve privacy. However, the methods that rely solely on RGB information to find persons are susceptible to environmental and lighting conditions, causing them to fail and leak PII. From our deployment study, nearly half of the images containing persons had a PII leakage when using RGB-only methods. Furthermore, ROI removal is often performed off-device, requiring the server performing these operations to be trustworthy. This work presents the PrivacyLens system, where with the addition of thermal sensing, our system has a significantly enhanced ability to find persons in RGB images and video and efficiently remove them on the device before any data is stored or transmitted, all while staying under typical IoT power constraints. From our aforementioned deployment study in an office-building atrium, family home, and outdoor park environment, the PrivacyLens prototype effectively removes PII with a sanitization rate of 99.1%. Additionally, PrivacyLens can use its embedded GPU to generate on-device features for downstream CV/ML tasks, as shown in three illustrative applications, further reducing the collection and storage of PII.

Pairing real-time ML with sensor data drives many interactive applications. However, the tools to prototype these applications are often proprietary and not open-source. This course instructs how to build interactive sensing applications using T4Train, an open-source and user-friendly framework for rapid prototyping. Participants will learn sensor interfaces (e.g., on a laptop/Arduino), signal processing, ML, and the T4Train tool. Afterward, they will build real-time, interactive sensing systems, such as LED lighting that reacts to different sounds or hand movements. This course builds on 4 semesters of instruction using T4Train and multiple HCI research contributions, including 3 award-winning papers at CHI.

Effective means of enabling single-lead, non-intrusive, and dry electrocardiogram (ECG) measurements offer the potential for prolonged cardiac rhythm monitoring of mobile users in non-clinical environments. However, existing ECG measurement approaches require accurate electrode placement, cumbersome wiring, and require users to be stationary. Alternatively, current heart sound-based approaches such as phonocardiograms lack the sensitivity and precision to detect crucial cardiac rhythm features and are vulnerable to environmental noise. This work utilizes a wide bandwidth surface-acoustic-wave microphone on the neck to capture heart sounds via the carotid artery. A cross-modal autoencoder, a state-of-the-art algorithm for signal modality conversion, is proposed to transform heart acoustic signals into corresponding ECG waveforms. Results from a 9 participant study demonstrate the effectiveness of constructing a PQRST waveform from acoustic heart sounds and accurately determining critical PQRST metrics. Finally, mobile acoustic ECG wave construction of a user walking is demonstrated, laying the groundwork for unobtrusive, long-term, low-cost daily cardiac rhythm monitoring.Clinical relevance—Transforming heart sound signals to produce prominent ECG metrics enables low-cost daily cardiac rhythm monitoring using a single-node dry wearable device

The proliferation of mobile devices in consumer electronics, IoT, and healthcare sectors has sparked considerable interest in wireless localization. While antenna array systems have demonstrated promise for wireless localization, they often entail high costs, intricate system designs, lengthy integration periods, and specialized packet formats. This study employs a 16-element L-shaped antenna array paired with a 2-channel 2MHz receiver, utilizing affordable and readily available components to localize incoming packets. The proposed approach calculates the 2D Angle of Arrival (AoA) of incoming signals using a custom Phase Difference Matching (PDM) algorithm. Additionally, a non-parallel wave depth estimator infers depth information of the signal source by learning phase difference trends. The result shows the system achieves median AoA error of 2.53 degrees horizontally and 1.88 degrees vertically, with an average depth estimation error of 1.07 m. This approach demonstrates the potential for 3D wireless localization of commonly available RF devices, through an N-element 2D phased array paired with a cost-effective commodity receiver.

Recent advancements in object-tracking technologies can turn mundane constructive assemblies into Tangible User Interfaces (TUI) media. Users rely on instructions or their own creativity to build both permanent and temporary structures out of such objects. However, most existing object-tracking technologies focus on tracking structures as monoliths, making it impossible to infer and track the user's assembly process and the resulting structures. Technologies that can track the assembly process often rely on specially fabricated assemblies, limiting the types of objects and structures they can track. Here, we present StructureSense, a tracking system based on passive UHF-RFID sensing that infers constructive assembly structures from object motion. We illustrated StructureSense in two use cases (as guided instructions and authoring tool) on two different constructive sets (wooden lamp and Jumbo Blocks), and evaluated system performance and usability. Our results showed the feasibility of using StructureSense to track mundane constructive assembly structures.

Abstract Visual impairments in older adults are associated with declines in physical, cognitive and social functioning, often leading to a loss of independence and quality of life. Increased age is a major risk factor, with conditions such as glaucoma having a 7-fold increase in prevalence from the 50s to the 80s (Klein & Klein, 2013). Treatment in the form of eye drops is commonly prescribed yet suboptimal adherence can significantly impact outcomes (Newman-Casey et al., 2015). The purpose of this preliminary study was to determine the effects of body posture and hand use on eye drop instillation success in a sample of 20 healthy, non-eye drop users (mean age: 70.0 +/- 3.7 y). Participants administered eyes drops without instruction while seated, standing and in a supine position. Video analysis was used to determine the number of attempts before successful instillation occurred (failed attempts) as well as whether participants used one or two hands. Despite published recommendations to use both hands (Davis et al, 2018), only 65% adopted a bimanual approach. The percentage of failed attempts was lowest in the supine (11.6%) compared to the seated (25.7%) or standing (21.2%) positions. Pinch strength was not correlated with successful instillation as has been previously reported (Naito et al., 2021), possibly due to participants’ overall level of physical functioning. Based on these results, a supine position is recommended for eye drop instillation where the head and neck are stabilized.

Touchscreen devices, designed with an assumed range of user abilities and interaction patterns, often present challenges for individuals with diverse abilities to operate independently. Prior efforts to improve accessibility through tools or algorithms necessitated alterations to touchscreen hardware or software, making them inapplicable for the large number of existing legacy devices. In this paper, we introduce BrushLens, a hardware interaction proxy that performs physical interactions on behalf of users while allowing them to continue utilizing accessible interfaces, such as screenreaders and assistive touch on smartphones, for interface exploration and command input. BrushLens maintains an interface model for accurate target localization and utilizes exchangeable actuators for physical actuation across a variety of device types, effectively reducing user workload and minimizing the risk of mistouch. Our evaluations reveal that BrushLens lowers the mistouch rate and empowers visually and motor impaired users to interact with otherwise inaccessible physical touchscreens more effectively.

Emerging ultra-low-power tiny scale computing devices run on harvested energy, are intermittently powered, have limited computational capability, and perform sensing and actuation functions under the control of a dedicated firmware operating without the supervisory control of an operating system. Wirelessly updating or patching firmware of such devices is inevitable. We consider the challenging problem of simultaneous and secure firmware updates or patching for a typical class of such devices—Computational Radio Frequency Identification (CRFID) devices. We propose Wisecr, the first secure and simultaneous wireless code dissemination mechanism to multiple devices that prevents malicious code injection attacks and intellectual property (IP) theft, whilst enabling remote attestation of code installation. Importantly, Wisecr is engineered to comply with existing ISO compliant communication protocol standards employed by CRFID devices and systems. We comprehensively evaluate Wisecr's overhead, demonstrate its implementation over standards compliant protocols, analyze its security, implement an end-to-end realization with popular CRFID devices and open-source the complete software package on GitHub.

Device-free localization methods allow users to benefit from location-aware services without the need to carry a transponder. However, conventional radio sensing approaches using active wireless devices require wired power or continual battery maintenance, limiting deployability. We present TomoID, a real-time multi-user UHF RFID tomographic localization system that uses low-level communication channel parameters such as RSSI, RF Phase, and Read Rate, to create probability heatmaps of users' locations. The heatmaps are passed to our custom-designed signal processing and machine learning pipeline to robustly predict users' locations. Results show that TomoID is highly accurate, with an average mean error of 17.1 cm for a stationary user and 18.9 cm when users are walking. With multiuser tracking, results showing an average mean error of <72 cm for five individuals in constant motion. Importantly, TomoID is specifically designed to work in real-world multipath-rich indoor environments. Our signal processing and machine learning pipeline allows a pre-trained localization model to be applied to new environments of different shapes and sizes, while maintaining good accuracy sufficient for indoor user localization and tracking. Ultimately, TomoID enables a scalable, easily deployable, and minimally intrusive method for locating uninstrumented users in indoor environments.