Green Rhythms is a symbiotic interactive installation that translates the circadian rhythm of plants into a tangible sensory experience shared between humans and plants. Grounded in more-than-human design and time-ecology frameworks, this artwork visualizes the differential physiological processes of photosynthesis and nocturnal gas exchange through the media of light and air. The environmental light and the carbon dioxide exhaled by audiences establish a cycle of metabolic exchange. This work reimagines interaction design as a medium for ecological coordination, which emphasizes decentralized temporality and the shared rhythm across species.
This paper presents Connected Prayerwheels, a pair of augmented cultural objects using gesture sensors, ambient lights, and a song sharing feature designed to enhance everyday cultural connectedness for Bhutanese families over distance. We conducted interviews and design discussions with participants across Bhutan and Australia to understand families' everyday routines, practices, and interests. We found many of their everyday routines were rooted in Buddhist cultural and religious traditions, which they were keen to share over distance. We therefore designed a prototype, the Connected Prayerwheel, and demonstrated it to participants to explore whether it would be fitting to use a cultural device in a connected form. Our findings highlight how cultural connectedness can be enriched through connected cultural artifacts. Families enjoyed sharing prayers, Bhutanese songs and hymns (Choeyangs), and the meaning of cultural symbols. This study contributes the design and discussion of connected cultural artifacts to studies of family connection over distance. CCS Concepts center dot Human-centered computing -> Haptic devices.
While generative AI has gained immense popularity, its application in specialized fields, particularly in music production, remains challenging. Little research has been done on how music producers can most intuitively communicate with generative AI to co-produce music. In this paper, we explored how to employ gestural and vocal interactions for an intuitive human-AI music co-production process. Through task analysis of computer-assisted music production, online surveys probing the key challenges in music production, and an in-lab gesture elicitation study with 14 music producers, we uncovered how music producers employ 9 types of gestures (e.g., finger snapping, body shaking, etc.) accompanied by 2 types of vocal expressions (i.e., beatboxing and humming) to interact with a music-generating AI system, communicating 4 music features during music production. This research advances understanding of embodied interactions between music producers and generative AI, while demonstrating how gesture elicitation data from open-ended user requests can be analyzed.
We present RoboTheater, a multi-robot storytelling platform, designed to explore and demonstrate the expressive capabilities of robots in stage execution. RoboTheater bridges computational narrative generation with embodied robotic execution, enabling robots to perform with varied dialogues and movements to create immersive stage narratives. The system uses large language models (LLMs) to generate structured scripts that are mapped into robots movements, speech, and visual projections, with all commands sent over a wireless network. A user study with 13 participants was conducted to evaluate the systems usability and gather feedback. Our findings indicate that the RoboTheater effectively conveys emotion and character through multimodal cues, sustaining audience engagement and immersion. The work expands multi-robot storytelling and paves the way for creative, narrative applications.CCS Concepts center dot Human-centered computing -> Human computer interaction (HCI).
The rapid pace of technological development over the past several decades has led to an exponential rise in electronic waste, or "e-waste." As devices become obsolete through cycles of innovation and planned obsolescence, the data once contained on these devices fades into obscurity. This project situates itself within ongoing discourses of material circularity and archival media, asking how obsolete technologies might experience renewal in new and creative forms. Specifically, this research explores the intersection of electronic waste, data sonification, and reuse through the design and development of Waste Speaker, a hand-held audio device built from discarded hard disk drives (HDDs). By translating latent data stored within these HDDs into generative sound compositions, Waste Speaker reconfigures the hard drive as both a medium and message of revival. In doing so, this project contributes to broader conversations around defunct media and sustainable design, positioning e-waste as a site of resurgence: no longer "waste," and instead a medium for sensory engagement and critical reflection.
Hybrid crafts blend traditional materials with embedded interactivity, yet everyday use of such artefacts remains underexplored. This paper contributes the study of lived experience with high-fidelity hybrid craft artefacts (made with interactive stained glass), following an initial study with 11 practitioners. We conducted three co-design sessions with nine end-users to create bespoke crafted artefacts deployed across three households for three weeks each, supported by user diaries and interviews. Our findings reveal how participants developed habitual patterns (placement, timing, relocation), emotional attachments, and disengagement needs, perceiving the artefacts as warm, expressive, and socially meaningful, distinguishing hybrid crafts from screen-based or smart devices. We offer design recommendations for leveraging artistic and pictorial qualities, supporting routine-based interaction and emotional connection, and designing for placement flexibility and spatial legibility. This work marks a shift from practitioner-only explorations of hybrid crafts toward user-centred in-the-wild understanding of how these artefacts function as lived-with technologies.
We demonstrate PufFab, an interactive design-to-fabrication pipeline that enables customizable edible shape transformation using Vietnamese rice paper. The pipeline combines a user interface with a modified 3D printer to allow users to customize and fabricate 2D patterns that transform into 3D structures when fried. Users can modify rice paper through two distinct manipulation techniques: moistening, which locally softens and flattens the materialresulting in reduced expansion and controlled shrinkage during frying-and cutting, which introduces appearance differences and structural discontinuities that cause the paper to curve or deform along edges. When fried, the rice paper morphs according to the pattern because of its unique material response to heat and hydration. Our preliminary work explores an accessible and culturally grounded approach to human-food interaction and demonstrates new possibilities for customizable computational gastronomy, playful edible interaction, and artistic food transformation. CCS Concepts center dot Human-centered computing -> Interactive systems and tools.
In creating an academic paper, many hours of work are spread across planning, conducting research, analyzing findings, designing figures, and writing up the results. Ultimately, the effort can be obscured behind the final publication. For our recent paper for IEEE VIS, "Stitching Meaning: Practices of Data Textiles Creators," we tracked every hour of work we did during its creation. Mirroring the topic of this paper, data textiles, where information is represented through fiber arts, we create a series of pieces using the data we collected on our time spent. Collectively, our 'paper pillows' encode three different views of our worktime in three textile mediums, representing our work as beautiful and practical objects. The act of making encouraged us to engage with our data from close and far perspectives that helped us to contextualize and reflect on our experiences, allowing us to live with and iteratively understand our data.
Past research explains how digital well-being tools tend to reduce stress to something measurable and optimizable, overlooking its embodied, emotional, and temporal nature. Drawing on previous research on rituals, which shows how repeated, intentional practices shape attention, create meaning, and confer a perception of continuity in everyday life, we explore an alternative approach to stress management. We propose a tangible interaction that encourages users to reflect on how they cope with stress, supporting preparation for overwhelming moments in their lives, rather than suppressing them. We highlight how physical materials can be utilized for emotional grounding and fostering continuity across daily life. This approach suggests a new direction for emotional well-being technology that honors the everyday rhythms of the body and opens space for longer-term use and other emotional experiences.
Negative experiences challenge the familiar ideals of ease, usability, and control that often shape interaction design practice. This work-in-progress investigates how such experiences can be intentionally engaged and mediated through materials in tangible interaction design, and how they might unfold in everyday contexts beyond artistic practice. Using a Research-through-Design approach, we combine first-person material explorations with a co-creation workshop to study how affective tensions such as discomfort, awkwardness, irritation, and even disgust or cringe arise through making. The study introduces three early-stage insights: Material Subjectivity, Materializing Negative Experiences in Practice, and The Positivity Appeal. Together, these insights show how materials can mediate and reframe negative experience as a generative design resource, offering designers new ways to approach negative experience in everyday interaction. CCS Concepts center dot Human-centered computing -> Interaction design theory, concepts and paradigms.
We present Ephemeral Breath, a self-contained, contactless bio-feedback system that invites brief, spontaneous moments of mindfulness during everyday interludes in semi-public spaces. Motivated by the challenge that many biofeedback systems require personal wearables, explicit training, or app initiation-thus limiting accessibility and long-term adoption-we explore an ambient alternative that preserves privacy while affording immediate, low-effort engagement. Our system uses a 60 GHzmmWave radar module to estimate respiratory phase and rate without physical contact, and integrates environmental sensing (illuminance, dominant color, and sound level) to adapt feedback to the surrounding context. Respiratory signals are transformed into synchronized audiovisual expressions: LED hue, brightness, and motion visualize inhalation and exhalation as a traveling wave, while a softly detuned, reverberant soundscape follows the breathing envelope. This study proposes an interactive design to support daily mindfulness through prototyping non-contact acoustic and visual biofeedback using mmWave, while also exploring the potential for sharing biofeedback in semi-public spaces. CCS Concepts center dot Human-centered computing -> Interaction design; Ubiquitous and mobile computing; Interaction design theory, concepts and paradigms.
Contained Attraction is an interactive kinetic sculpture that explores material agency through mediated human-material interaction. The sculpture uses capacitive sensing to sense human presence, and electromagnetic actuation to computationally choreograph the motion of magnetic fluid. Participants use their hands to influence the chaotic movement of dark, viscous magnetic fluid that resists deterministic control. The fluid's organic splitting, merging, and flowing create unpredictable behaviors that blur the boundaries between computational control and material autonomy. This interactivity foregrounds the tension between human intent and material response, and reinforces human tendencies to anthropomorphize organic behavior through the attribution of pronouns, relationships, and intentionality. This work contributes an exploration of the entangled relationships between humans and materials - where the sculpture as a medium embodies dynamics of agency and mediated interaction.
This study advances the design of timing-sensitive, full-body interactive experiences supporting both collaboration and individual agency in children's group play. We investigate how temporal structuring within an improved version of a full-body interactive system influences behavioral and perceived synchrony among children aged 9-10. Prior research has often focused on dyads, required human facilitation, or neglected relationship between perceived and actual synchrony. Addressing these gaps, we conducted a multi-user study comparing synchronous and asynchronous interaction modes using system-logged behavioral data and self-reports. Results demonstrate the synchronous condition fosters higher user performance, tighter temporal coordination, and stronger perceptions of shared movement. In contrast, the asynchronous mode effectively disrupts unintended coordination, showing how timing design shapes interaction dynamics. Notably, identical movements are perceived as less uniform without temporal alignment. These findings highlight the potential of embodied systems to scaffold or inhibit synchrony through intentional design, supporting richer social engagement and inclusive group play.
Families often reflect on shared experiences, but few tools support this process through tangible, collaborative interaction. WePrism explores this space by introducing a constructive data physicalization toolkit designed for domestic self-tracking. Using modular tokens in various shapes, sizes, and colors, WePrism allows family members to represent and share aspects of their everyday lives through co-created sculptural forms. Emphasizing open-ended, screenless interaction, the system invites users to define their own symbolic mappings, encouraging creativity, interpretation, and ongoing dialogue. To evaluate its use, we conducted a five-day in-situ study with a family of three. Findings highlight the toolkit's accessibility, ability to spark low-effort reflection, and potential to foster shared awareness. Participants appreciated its material presence but noted the need for clearer onboarding and more visible digital features. WePrism contributes to the growing field of tangible self-tracking by showing how physical artifacts can support inclusive, expressive, and reflective practices in home environments. CCS Concepts center dot Human-centered computing -> Collaborative and social computing devices; Visualization toolkits.
As hardware prototyping toolkits and accessible programming environments become more widespread, it is easier than ever to build functional prototypes of interactive devices. Yet moving beyond a prototype towards manufacturable devices still presents significant challenges. This evolution often requires specialised knowledge in circuit design, firmware development, and hardware testing, which can discourage less experienced creators. We present MakeDevice, a browser-based tool that helps users navigate this transition by leveraging the intuitive micro:bit, Jacdac and MakeCode ecosystem coupled with a process we call device flattening. Modular micro:bitplus-Jacdac hardware assemblies are progressively transformed, or flattened, into increasingly-integrated custom printed circuit board (PCB) designs. We demonstrate MakeDevice through two examples. MakeDevice lowers barriers to the creation and deployment of new tangible, embedded and embodied devices by providing an alternative to the traditional PCB design process.
The haptic properties of textile interfaces support eyes-free use well. However, user interfaces often require composing many controls, which complicates orientation by palpation. We investigated three composition concepts that combine textile components in different ways, using a sample smart home scenario: One uses the select-and-control flow of universal remotes, one resembles the users' environment, and one divides the interface into application sections. Of the latter, we also created a spacious variant to observe the effect of blank space between interface regions. We first explored how well users understand those composition concepts when first using the interface eyes-free. After familiarization, we measured input performance and preferences. We found task completion times much faster than expected from previous recognition studies. While performance was similar for most interfaces, the select-and-control flow was slower, but participants preferred it after familiarization. From our findings, we derive design recommendations for compound textile UIs and individual components.
We propose GadJets, an approach to remotely actuating passive materials and mechanisms using air jets for interactive and shape-changing tangible user interfaces (TUIs). Compared to previous HCI research in remote actuation, air jets offer relatively strong, distant, and safe actuation with simple control. Leveraging these advantages, we can selectively actuate multiple passive materials and mechanisms (or GadJet modules) using only one actuator. We defined a design space to outline the basic architecture and generalizable primitives for air-jet-based TUIs. Also, we exemplified our approach with diverse GadJets modules and proof-of-concept implementation of a computer-controlled air jet system. Finally, we developed a visualization system of the estimated air range based on pre-collected data to support user control. With the vision of deploying multiple passive GadJet modules in an environment fused with the actuated air jets, we demonstrated applications of interactive tabletop objects, an actuated workbench, and an actuated shelf.
This paper presents insights from a Research through Design investigation exploring the integration of tangible and embodied interaction (TEI) technologies into Montessori elementary mathematics education. Unlike traditional classrooms, Montessori prioritizes self-directed exploration, mixed-age groups, and hands-on engagement with physical manipulatives. While prior research shows TEI can support math learning in traditional classrooms, its use in Montessori settings is underexplored. This paper reports on the foundational phase of a Research through Design (RtD) study with ten Montessori teachers. Our process began with foundational inquiry, including semi-structured interviews, classroom observations, and visioning workshops with teachers, which guided the development of a proof-of-concept prototype. This prototype served as a central artifact for inquiry, enabling collaborative exploration and refinement during iterative sessions with each teacher. Through this design-led inquiry, our qualitative analysis identifies initial considerations for supporting individualized learning, collaboration, and physical experiences. CCS Concepts center dot Human-centered computing -> Human computer interaction (HCI); Interaction design.
Accelerometers built into devices, such as smartphones and smart-watches, are used for applications including activity tracking and health management. A method has been proposed that intentionally modifies sensor output by irradiating sound waves, and research exists that applies this technique to embed arbitrary bit sequences into capacitive accelerometers. This paper proposes a method for real-time analysis and identification of bit sequences embedded in acceleration data. The proposed method enables contactless control of smartphones through the use of sound wave irradiation. In the evaluation experiment, the success rate of correctly launching the appropriate application for 4 to 8-bit sound wave patterns was measured. While Devices 1 and 3 achieved 100% accuracy under all conditions, Device 2 showed a tendency for the success rate to decrease as the number of bits increased.
The more-than-human field has contributed numerous opportunities for interacting with nature, animals, plants, and microorganisms. However, few studies have examined water ecosystems. Current water-related work primarily treats water as a medium for human-centered activities, rarely positioning water as an interactive subject. Building upon prior research, we explore how to better integrate water more playfully into digital-physical interactions as an interactive subject. We designed and developed Water's Echo, an AI-powered public installation that enables human-water communication through a playful stone-throwing dialogue. We conducted a field study at a local pond, recruiting 15 residents to participate in Water's Echo-a playful conversational interaction. Our findings indicate that this playful dialogue approach raises participants' awareness and understanding of surrounding aquatic environments. This research provides insights for design researchers to establish engaging water ecology interactions across cultural communities, promoting a More-than-human perspective in re-examining human-nature relationships. CCS Concepts center dot Human-centered computing -> Natural language interfaces; Contextual design; Interface design prototyping; Empirical studies in HCI; Sound-based input / output.
