High-performance machine learning models generally suffer from a lack of transparency that calls into question the ability of humans to understand how they work. Through the study of CoMo, an interactive learning system that allows users to associate sounds and gestures, we show that a form of model understanding can emerge from user-model interactions. This interactive approach offers an alternative to explanation- and interpretation-based epistemologies. Although data- and computation-intensive deep learning systems are not well suited to this form of interactivity, shallow learning models offer underappreciated epistemological advantages in contexts where machine learning can be made interactive.

Hip-hop battles are central to the hip-hop dance community as improvisational and collective events shaped by complex interactions between dancers, DJs, MCs, judges, and audience. This paper explores how hip-hop professionals perceive and negotiate these interactions. We conducted eight interviews to examine the structure of battles, the fluidity of roles, and the coded forms of interaction inside the battle. Our findings reveal that battles rely on a shared yet constantly evolving framework, redefined through "concepts" that introduce new rules and experiences for participants. Coordination across roles is maintained through embodied and sonic cues rather than explicit communication, producing a system of "distributed cognition" that connects music, movement, and listening. We discuss (1) the fluidity of roles within battles, (2) the battle as an interactive system itself, and (3) implications for designing technologies that align with hip-hop's values of improvisation, expression, and participation. Our contributions are: (1) empirical insights into the dynamics of hip-hop battles, and (2) a conceptual articulation of design challenges and opportunities for technology in this context. This study is intentionally exploratory as it is the first stage of a broader research program on hip-hop and interactive technologies.

Polytempo music – where multiple, simultaneous tempi are performed – poses unique challenges for ensemble coordination, yet empirical research on its performance remains scarce. This exploratory study investigates how expert musicians execute different forms of polytempo, comparing temporal stratification (distant tempi) and phasing-like configurations (close tempi) to both a simple polyrhythm condition and a baseline condition (in which all musicians played at the same tempo). Six highly experienced performers completed 32 trials in which they played isochronous pulses at individually assigned tempi while hearing individualized click tracks that either continued or disappeared early in the trial. We analyzed temporal regularity, tempo change, and group-level success in maintaining prescribed tempo ratios. Across measures, polytempo configurations elicited greater difficulty than the polyrhythm condition, which did not clearly differ from unison tempi, suggesting that expert performers may spontaneously assimilate simple tempo ratios into a shared rhythmic framework. Performances at distant tempi produced significantly more tempo change and markedly lower success in reproducing target ratios than either close tempi or polyrhythms, indicating that temporal stratification is more challenging than phasing when performed without shared structural cues. Individual click tracks had limited influence on stability or coordination, though they did reduce tempo change in both polytempo conditions. These findings provide the first empirical comparison of various types of polytempo performance in an ecologically valid, multi-instrumental context, offering insight into the coordinative demands of contemporary polytempo practice, suggesting new considerations for composers and performers working with complex temporal structures, and shedding new light on the broader cognitive mechanisms involved in resisting the seemingly inescapable tendency to synchronize with one another.

Body movement sonification has gained increasing attention in rehabilitation and healthcare as a means to influence movement, and body perception. Prior research on the Footsteps Illusion demonstrated that real-time pitch manipulation of self-produced footstep sounds can alter perceived body weight and emotional responses. However, the systems used for that illusion rely on microphones that require controlled laboratory conditions, limiting their applicability in natural settings. This study introduces a novel, portable approach that employs prerecorded footstep sounds, played synchronously with walking movements and filtered to evoke sensations of lighter (high-frequency) or heavier (low-frequency) bodies. Participants walked along an indoor circuit while listening to these sounds; their responses were evaluated through self-reports and gait analysis. Results show that prerecorded, movement-triggered sounds can reproduce the perceptual and emotional effects observed in real-time sonification setups. These findings highlight the importance of auditory feedback in shaping body perception and emotional experience. The proposed system extends the Footsteps Illusion beyond the lab, opening possibilities for sound-based Body Transformation Experiences (BTEs) in diverse contexts, such as healthcare, rehabilitation, and everyday movement practices that support well-being.

We present CoMo, a web-based ecosystem developed for movement-sound interaction. After several years of using a first version in workshops, artistic projects, and rehabilitation applications, we conducted interviews and specific trials with experts from different disciplines to guide an upgrade of the system. In this paper, we report the general motivation and initial implementation of the application, as well as the results of the interviews, which helped specify different use cases and informed the design of the revised version. This open-source ecosystem should benefit the Movement and Computing community at large by enabling the rapid development of a wide range of prototypes for movement-sound interactions.

Over the last 30 years researchers in HCI, Cognitive Neuroscience, and Interaction Design have shown a growing interest in experiences that engage the moving and sensual body to alter one’s body perception. The way one’s body is perceived is highly plastic and can be altered through multisensory signals and feedback related to the body. The emerging developments in multisensory interfaces open opportunities to enrich, augment and transform body experiences in the real-world through the senses. This workshop focuses on the theories, approaches, methods, and tools to design multisensory technology that elicit and support Body Transformation Experiences, and on how to best design these for and from a first-person, lived experience. We will explore how to elicit and assess multisensory Body Transformation Experiences, and showcase concrete examples of supporting them with technology. Through technology presentations, panel sessions with experts, and multidisciplinary discussions, this workshop aims to: (i) bring together researchers creating Body Transformation through sensory technology with those studying experiential effects of sensory-body interactions; (ii) map current methods, opportunities, and challenges in designing Body Transformation Experiences; and (iii) envision a road map for this field with future directions by fostering a multidisciplinary community, building collaborations, and inspiring innovative directions for design and research.

Hip-hop dance battles are events where dancers improvise to an unfamiliar DJ’s mix. The dancers’ technicality in responding to the music, along with the collective dimensions of these encounters, represents a largely unexplored area of investigation in HCI. Using an autobiographical design approach grounded in the hip-hop practice of the first author, we developed five interaction scenarios on a hip-hop–specific movement vocabulary. Our interactive sonification enables both individual and collective improvisation through synchronized motion sensing and interactive sound loops. This approach allowed us to design three interconnected workshops to investigate how dancers use sonification in improvisation. Our findings show that sonification was perceived either as a validation of movement or as a medium for open-ended exploration. We contribute with: (1) the formalization of a movement vocabulary and the design of interactive scenarios for hip-hop movement sonification, (2) insights into how dancers experience sonification through exploration and improvisation, and (3) guidelines for the design of movement sonification in hip-hop contexts.

Inertial Measurement Units (IMUs), which embed several sensors (accelerometers, gyroscopes, magnetometers) are employed by musicians and performers to control sound, music, or lighting on stage. In particular, wireless IMU systems in the performing arts require particular attention due to strict requirements regarding streaming sample rate, latency, power consumption, and programmability. This article presents a review of systems developed in this context at IRCAM as well as in other laboratories and companies, highlighting specificities in terms of sensing, communication, performance, digital processing, and usage. Although basic IMUs are now widely integrated into IoT systems and smartphones, the availability of complete commercial wireless systems that meet the constraints of the performing arts remains limited. For this reason, a review of systems used in performing Arts provides exemplary use cases that may also be relevant to other applications.

Gait impairments following stroke are prevalent and persistent, often affecting walking ability, body perception, and quality of life. Movement sonification—mapping body motion to sound—has shown promise for modulating sensorimotor function. This multiple-case experimental study investigated the effects of movement sonification using metaphorical sound on gait, body perception, and emotional state in three individuals with chronic stroke. Using an ABAC design, participants walked with real-time auditory feedback delivered via an interactive walking-interface system (SoniBand) across four conditions: baseline, “Wind” sound, return to baseline, and “Mechanical” sound, presented in a counterbalanced order. Gait parameters (symmetry, step frequency, velocity, and acceleration) were assessed alongside self-reported body perception and emotional state. Exposure to the “Wind” sound condition led to consistent improvements in walking velocity and symmetry, as well as enhanced perceptions of body lightness, speed, flexibility, and physical capability. Accelerometer data indicated carry-over effects from the sound condition in gait symmetry and step regularity. These findings support the potential of metaphorical sound sonification to influence internal models of movement and facilitate sensorimotor adaptation in stroke survivors. This study underscores the malleability of body perceptions through auditory feedback and highlights the value of personalized, perceptually grounded interventions in neurorehabilitation.

Storytelling is an essential practice in preschool. Recent advances in digital tools for multimodal and collective interactions represent opportunities for new approaches in storytelling. Motivated by an embodied approach of storytelling, we developed CoMo•education, an interactive system where whole-body movements and sounds are in play. Moreover, this application enables storytelling as a collective and multimodal experience. Using gesture recognition with screenless smartphones, the teacher and pupils can use movements to play various soundscapes, which enliven the story. The aim of this article is to present the overall project, which has spanned more than 3 years of iterative developments. Importantly, it was developed through an iterative co-design process including designers, teachers, early-development experts and engineers. A user study was carried out in two kindergarten classes which allowed us to gather feedback from teachers and children, and formalize a series of guidelines for future works. A workshop was also dedicated to educators to evaluate how educators can appropriate the application to create interactive stories by themselves.

In recent years, significant advances have been made in deep learning models for audio generation, offering promising tools for musical creation. In this work, we investigate the use of deep audio generative models in interactive dance/music performance. We adopted a performance-led research design approach, establishing an art-research collaboration between a researcher/musician and a dancer. First, we describe our motion-sound interactive system integrating deep audio generative model and propose three methods for embodied exploration of deep latent spaces. Then, we detail the creative process for building the performance centered on the co-design of the system. Finally, we report feedback from the dancer’s interviews and discuss the results and perspectives. The code implementation is publicly available on our github1.

We present a retrospective autoethnography grounded in data-driven design. The first author collected her movement data and subjective experience of learning the dance repertoire of modern dance pioneer Isadora Duncan, which together were encoded into the design of a set of plaster artefacts physicalising her embodied dance learning progression. The artefacts reflect the first author’s bodily transformation, mirroring her transition from discomfort to ease, and changes in her expressive capabilities. Our method offers an alternative to documentation of embodied learning through design. Throughout our design process we leverage on the movement data, the field notes and the first author’s memory of her journey, all of which constitute entangled and complementary input into her experience of dance learning. We show that the data physicalisations provided a gateway into the intangible experience and allowed for a deep and reflexive understanding of our dataset.

In this paper, we present Simone, a distributed musical instrument for collective improvisation created using a research through design approach. The collective interaction with Simone is performed through different interaction scenarios specifying how data is exchanged between users on the local network. First, we present the design and implementation of Simone including the audio synthesis system and the different scenarios. Then, we present a study with groups of expert users, asking them to improvise collectively with Simone to observe their appropriation of the instrument and how they interact collectively. Qualitative data analysis shows that the process of appropriation is a complex phenomenon that depends on participants’ musical background and experience and is a preliminary condition for collaboration. Additionally, our results reveal participants’ perception of networked elements and the influence it had on their collective interaction. Finally, we discuss the design of collective interaction and distributed music instruments.

Wearables integrating movement sonification can support body-perception changes and related physical activity; yet, we lack design principles for such sonifications. Through two mixed-methods studies, we investigate sound pitch and movement direction interaction effects on self-perception during squat exercises. We measured effects on body perception, affective quality of the experience, and actual and perceived movement, and compared them with two control conditions: no-sound and vibrotactile feedback. Results show that regardless of movement direction, ascending pitch enhances several body feelings and overall experience quality, while descending pitch increases movement acceleration. These effects were moderated by exercise physical demand. Sound and vibrotactile feedback enhanced flexibility and strength feelings, respectively, and contributed to exercise completion in different ways. Sound was perceived as an internal-to-body force while vibrotactile feedback was perceived as an external-to-body force. Feedback effects were stronger in people with lower fitness levels. We discuss results in terms of malleability of body perceptions and highlight opportunities to support demanding physical activity through wearable devices.

Many technologies for promoting physical activity (PA) give limited importance to critical variables for engagement in PA, such as negative body perceptions. Here, we aim to address this gap by incorporating barriers and experienced body sensations into the design process for wearables and body-based devices thus expanding the design space for such technologies. We first report four co-design workshops with physically inactive participants (n=9); in these workshops, we explored tangible tools (i) to sensitize people to body sensations experienced when facing barriers to PA and (ii) to ideate how haptic and auditory feedback could transform body sensations to increase body-awareness and engagement in PA; results show several interactive sensorial patterns with potential to inform the design of body transformation wearables. These were validated through a follow-up workshop (n=13 participants) and reflections based on insights from a literature review. Our findings are significant for the design of ubiquitous technology to support and initiate PA in everyday contexts through a novel approach of transforming body perceptions/ sensations related to PA barriers. We contribute design inspirations and cards for identifying barriers to PA and to empower designers and researchers to integrate them early in the design process.