Remote emotional connection and internal emotional regulation play a critical role in addressing emotional isolation and mental stress in modern society. This study presents "Ga'u Echo," a wearable device grounded in biosensing and affective computing technologies, specifically developed to facilitate remote emotional connection and support individual emotional regulation. Functioning as a "digital ga'u box" for the user, "Ga'u Echo" transforms the physiological rhythms of distant loved ones and the user's own internal states into tangible tactile feedback. This enables individuals to form deeper, more authentic emotional connections with both others and themselves.
We introduce VisRing, the first smartring incorporating a bendable 160 x 32 4-bit grayscale organic light-emitting diode display. VisRing stands out by displaying nano visualizations while maintaining a compact design and minimal weight of 6.6 g, with an overall cost of around $35. We exploit opportunities for a system-on-a-chip architecture to tightly integrate an inertial measurement unit, a photoplethysmograph sensor, a temperature sensor, Bluetooth, a microcontroller, and a display unit that spans 270 degrees to 360 degrees, depending on finger size. Our contributions include the hardware design and implementation of VisRing, along with a software library that supports visualizing various data types. A qualitative study with 12 participants demonstrated the comfort, likability, and social acceptance of VisRing's hardware and software. The participants liked the visualizations and found the ring lightweight, but also pointed out possible improvements. All materials are shared under an open-source license to enable the community to extend and improve VisRing.
This paper introduces a novel authoring method for swarm user interfaces that combines hands-on shape manipulation and speech to convey intent for generative motion and interaction. We refer to this authoring method as shape-aware generative authoring, which is generalizable to actuated tangible user interfaces. The proof-of-concept Shape n' Swarm tool allows users to create diverse animations and interactions with tabletop robots by hand-arranging the robots and providing spoken instructions. The system employs multiple script-generating LLM agents that work together to handle user inputs for three major generative tasks: (1) thematically interpreting the shapes created by users; (2) creating animations for the manipulated shape; and (3) fexibly building interaction by mapping I/O. In a user study (n = 11), participants could easily create diverse physical animations and interactions without coding. To lead this novel research space, we also share limitations, research challenges, and design recommendations.
Large language models (LLMs) enable the rapid generation of data wrangling scripts based on natural language instructions, but these scripts may not fully adhere to user-specified requirements, necessitating careful inspection and iterative refinement. Existing approaches primarily assist users in understanding script logic and spotting potential issues themselves, rather than providing direct validation of correctness. To enhance debugging efficiency and optimize the user experience, we develop ViseGPT, a tool that automatically extracts constraints from user prompts to generate comprehensive test cases for verifying script reliability. The test results are then transformed into a tailored Gantt chart, allowing users to intuitively assess alignment with semantic requirements and iteratively refine their scripts. Our design decisions are informed by a formative study (N=8) that explores user practices and challenges. We further evaluate the effectiveness and usability of ViseGPT through a user study (N=18). Results indicate that ViseGPT significantly improves debugging efficiency for LLM-generated data-wrangling scripts, enhances users' ability to detect and correct issues, and streamlines the workflow experience.
We present UltraEdit, an in-situ design environment that enables users to directly edit ultrasound haptic sensations in VR using barehand interactions. UltraEdit represents haptic sensations as tangible objects, called blobs, which users can modify or apply to 3D objects through intuitive hand gestures. Users can create, edit, copy, and apply blobs, allowing them to work with multiple haptic sensations to design virtual objects with diverse tactile feedback. Additionally, UltraEdit allows users to create spatial and temporal haptic patterns using time blobs and spatial drawing features. Users can draw custom-shaped haptic patterns directly on their hands, adapting to comprehensive design scenarios. To evaluate UltraEdit, we conducted an exploratory user study assessing its usability, effectiveness, and ease of learning. We also compared its performance to an existing desktop-based haptic editing tool. Participants found UltraEdit intuitive to learn, enjoyable to use, and effective for adding haptic feedback to virtual objects.
This study presents EarPressure VR, a system that modulates ear canal pressure to simulate atmospheric pressure changes in virtual reality (VR). EarPressure VR employs sealed earbuds and a linear stepper motor-driven syringe to generate controlled pressure variations within safe limits (+/- 40 hPa relative to ambient pressure). Through two user studies, we evaluate (1) perceptual thresholds for detecting ear pressure in terms of direction (inward or outward) and intensity differences, and (2) the effect of ear pressure feedback on users' sense of environmental presence across two VR scenarios involving gradual and discrete changes in ambient pressure. Results show that participants reliably identified pressure direction at thresholds of +14.4 hPa (inward) and -23.8 hPa (outward), and intensity differences at +/- 14.6% and +/- 34.9%, respectively. Pressure feedback significantly improved presence ratings, particularly when pressure variation was continuously adjusted to reflect environmental transitions. We conclude by discussing the broader applicability of ear canal pressure feedback in areas such as training, simulation, and everyday experiences.
2D cartoon-style digital characters represent an important art form in games, animation, and virtual livestreaming. However, traditional 2D character creation workflows involve tedious manual layering, complex skeleton rigging, and professional animation skills, posing challenges for independent studios and non-professional users. While existing AI generation technologies can quickly create visual content, they typically produce non-layered, difficult-to-edit composite images that cannot be integrated into current workflows.This paper presents Spiritus, a semantic-driven 2D character generation framework. Unlike existing text-based AI animation workflows, Spiritus integrates mixed text and sketch inputs, achieving character image generation and automatic component segmentation through an open mask library and semantic matching. The system implements automatic skeleton rigging of character components through a shape-compatible mesh system and supports the export of Spine-compatible skeletal animations, enabling driving and rendering in web pages or game engines. We validated the system’s effectiveness in character generation freedom, character animation quality, and technical barrier reduction through comparative evaluation of user experiment results. Finally, we explored the possibilities of applying generated characters to various workflows and scenarios, including game development, animation production, and interactive illustrations.
Individual fagship conferences today can have over a thousand papers; even reading just the abstract of every paper at the latest relevant conference to keep up with the research is time and memory prohibitive. Previous visualizations in this domain have ubiquitously followed Shneiderman's Visual Information-Seeking Mantra, with details available on demand. However, recently in other domains, system designers have leveraged Structure-Mapping Theory (SMT) to facilitate seeing both the overview and the details at the same time, facilitating abstraction without losing context. We compose and evaluate a system, called ABSTRACTEXPLORER, with analogous SMT-derived characteristics for the domain of scientific abstract corpus familiarization. ABSTRACTEXPLORER has a unique combination of LLM-powered (1) faceted comparative close reading with (2) role highlighting enhanced by (3) structure-based ordering and (4) alignment. An ablation study (N=24) validated that these features work best together. A summative study (N=16) describes how these features support users in familiarizing themselves with a corpus of paper abstracts from a single large conference with over 1000 papers.
Our brain’s plasticity rapidly adapts our senses in VR, a phenomenon leveraged by techniques such a redirected walking, hand-redirection, etc. However, while most of HCI is interested in how users adapt to VR, we turn our attention to how users need to adapt their senses when returning to the real-world. We report cases where, even after leaving VR, users experience unintended, lingering side-effects: distortions in proprioception or memory that may pose safety or usability risks. To investigate, we conducted two studies examining (1) proprioceptive side-effects from altered hand movements (retargeting), and (2) memory distortions arising from spatial mismatches between the virtual and real-world locations of the same object. We found that, after leaving VR, (1) participants’ hands remained redirected by up to 7cm, indicating residual proprioceptive distortion; and (2) participants incorrectly recalled the virtual location of objects rather than their actual real-world locations (e.g., remembering the location of a VR-extinguisher, even when trying to recall the real one). Finally, we discuss the implications of these findings for VR and propose a call-to action for a deeper study of these side-effects within HCI.
Mobile eye tracking plays a vital role in capturing human visual attention across both real-world and extended reality (XR) environments, making it an essential tool for applications ranging from behavioural research to human-computer interaction. However, missing values due to blinks, pupil detection errors, or illumination changes pose significant challenges for further gaze data analysis. To address this challenge, we introduce HAGI - a multi-modal diffusion-based approach for gaze data imputation that, for the first time, uses the integrated head orientation sensors to exploit the inherent correlation between head and eye movements. Our method includes a head-movement feature extraction module alongside a novel hybrid feature fusion mechanism that effectively integrates gaze and head motion features at multiple levels. Additionally, we introduce a tailored loss function to enhance gaze imputation accuracy further. Extensive evaluations on the large-scale Nymeria, Ego-Exo4D, and HOT3D datasets demonstrate that HAGI consistently outperforms conventional interpolation methods and deep learning-based time-series imputation baselines, reducing mean angular error by up to 22%. Furthermore, statistical analyses confirm that HAGI produces gaze velocity distributions that more closely match actual human gaze behaviour than baselines, ensuring more realistic gaze imputations. Our method paves the way for more complete and accurate eye gaze recordings in real-world settings and has significant potential for enhancing gaze-based analysis and interaction across various application domains.
Recent advances in 2D generative AI are finding applications in highly specialized fields, such as car exterior design. However, current 2D-centric approaches have several limitations: each viewpoint requires a new sketch; consistency across different viewpoints is difficult to maintain; and steering design development in the desired direction is challenging. To address these limitations, we propose a novel workflow that integrates 3D sketching with 2D generative AI for car exterior design. This workflow enables car designers to seamlessly transition between expressive 3D sketching, detailed 2D drawing, and realistic 2D generation. In a formal user study, professional car designers used our system to create new exterior concepts across all major car body types. They produced high-quality, view-consistent sets of 2D renderings in a short time, demonstrating diverse patterns of progressive design development.
Communication barriers between Deaf and Hard-of-Hearing (DHH) individuals and hearing individuals remain a major challenge, highlighting the need for technologies that enable seamless sign language interpretation. However, current American Sign Language (ASL) recognition and translation systems face key limitations, including poor portability, complex usage settings, incomplete capture of essential components, and weak generalization, reducing practicality and user acceptance. To address these challenges, we present SignGlass, the first smart glasses equipped with three wearable cameras for comprehensive capture of both manual and non-manual ASL markers, supported by advanced algorithms for real-time recognition and translation into English. Specifically, SignGlass integrates a jitter-aware spatio-temporal attention mechanism for robust recognition of hand movement patterns. Complementing this, a dual-camera, temporally-aware facial module captures the subtle facial expressions essential for ASL comprehension. To support diverse signing styles across individuals, we further introduce a cascaded data augmentation strategy to improve model generalization. In a user study with 14 Deaf participants, SignGlass achieved high translation accuracy and was well-received, demonstrating its effectiveness in bridging communication gaps. This work highlights the promise of multi-camera wearable systems in advancing ASL translation and promoting more accessible communication.
We propose ModalPlayground, a collaborative design tool for authoring rich, multi-modal haptic feedback directly onto a physical space. Location-specific haptic feedback design faces significant barriers for creating immersive experiences that respond to users’ positions in real-world spaces without tethering them to XR environments. The authoring process is often disconnected from the physical experience, leading to a gap between designer intent and user perception. ModalPlayground allows designers to use their own bodies as a ’paintbrush’, physically walking through a space to define and layer haptic zones. Using high-precision Ultra-wideband (UWB) indoor positioning and the multi-modal haptic feedback capabilities of the DataFeel DevKit, this tool enables multiple designers to co-create and modify a shared haptic canvas in real time. This embodied, in-situ approach targets applications in immersive and interactive storytelling as well as spatial guidance.
Micro-gesture recognition and fine-grain pinch press enables intuitive and discreet control of devices, offering significant potential for enhancing human-computer interaction (HCI). In this paper, we present EI-Lite, a lightweight wrist-worn electrical impedance sensing device for micro-gesture recognition and continuous pinch force estimation. We elicit an optimal and simplified device architecture through an ablation study on electrode placement with 13 users, and implement the elicited designs through 3D printing. We capture data on 15 participants on (1) six common micro-gestures (plus idle state) and (2) index finger pinch forces, then develop machine learning models that interpret the impedance signals generated by these micro-gestures and pinch forces. Our system is capable of accurate recognition of micro-gesture events (96.33% accuracy), as well as continuously estimating the pinch force of the index finger in physical units (Newton), with the mean-squared-error (MSE) of 0.3071 (or mean-force-variance of 0.55 Newtons) over 15 participants. Finally, we demonstrate EI-Lite’s applicability via three applications in AR/VR, gaming, and assistive technologies.
Narrative-based education engages children in learning, but traditional approaches offer limited adaptability to individual preferences. Although large language models (LLMs) offer promising opportunities for interactive narratives, balancing their unpredictability with structured learning objectives remains challenging. To answer this challenge, we designed and built Oak Story, an educational mobile application for 4th-6th graders centered on local oak woodland ecosystems. Oak Story employs a learning-goal-directed LLM architecture that adapts the narrative, as well as multimodal real-world activities, to each individual student while ensuring that learning goals are met. In a between-participants study (N = 47), we find that Oak Story produces statistically significant increases in learning gains, engagement, and perceived agency compared to a control with static sequencing within and between scenes. These findings demonstrate an effective architectural approach for LLM-based educational systems that successfully balances learner agency with pedagogical structure.
Dynamic physical interfaces are often dedicated devices designed to adapt their physical properties to user needs. In this paper, we present an actuation system that allows users to transform their existing objects into dynamic physical user interfaces. We design our actuation system to integrate as a self-contained locomotion layer into existing objects that are small-scale, i.e., hand-size rather than furniture-size. We envision that such objects can act as collaborators: as a studio assistant in a painter’s palette, as tutors in a student’s ruler, or as caretakers for plants evading direct sunlight. The key idea is to decompose the actuation into (1) energy input and (2) steering to achieve a flat form factor. The energy input is provided by simple vibration. We implement steering through differential friction controlled by flat-foldable compliant structures that can be activated electrically. We study the mechanism and its performance, and show its application scenarios enabling dynamic interactions with objects.
While electronics miniaturization has propelled the evolution of technology from desktops to compact wearables, most devices are still rigid and bulky, often leading to abandonment. To enable interfaces that can truly disappear and seamlessly integrate into daily life, the next evolutionary leap will require further miniaturization to achieve full conformability. With FiberCircuits, we ofer design and fabrication guidelines for the manufacturing of high-density circuits that are thin enough for full encapsulation within fbers. Our demonstrations include a 1.4 mm-wide ARM microcontroller with sensors as small as 0.9 mm-wide and arrays of 1 mm-wide addressable LEDs, which were woven into our interactive textiles. We provide example applications from ftness to VR, and propose a scalable fabrication process to enable large-scale deployment. To accelerate future research in HCI, we also made our platform Arduino-compatible, created custom libraries, and open-sourced all the materials. Finally, our technical characterizations demonstrate FiberCircuits' durability, thanks to its silicone encapsulation for waterproofness and braiding for robustness. From wearables to insertables or even implantables, we believe that by making miniature circuits accessible to researchers and beyond, FiberCircuits will open possibilities for new scalable interfaces that embody imperceptible computing.
Recent HCI research has explored active materials for programmable displays, yet accessibility remains a key challenge. While some programmable materials, such as thermochromic ink, are widely available, others—like electronic ink (EInk)—remain confined to industrial production. Despite its versatility, EInk is rarely used in HCI due to complex production processes that require expert knowledge in chemistry. We address this limitation by adapting existing microencapsulation techniques from other fields and identifying barriers to broader adoption. We present a simplified, safe, and more accessible method for producing EInk microcapsules. Through a series of analyses, we evaluate the viability of the resulting EInk and we evaluated the process with six participants unfamiliar with EInk fabrication and found it to be accessible and easy to follow. Our work represents a step toward democratizing EInk, bridging the gap between chemical engineering and practical application in HCI, and enabling broader integration of EInk into the design of diverse interactive devices.
