There is little evidence of the role that sound in interfaces may play in our tendency to anthropomorphize animated or static objects. This paper explores the influence of sound on perceptions of animacy, anthropomorphism, and related dimensions in user interfaces. Across two studies, a total of 533 participants interacted with simulated parking meter and vending machine interfaces under one of three sound conditions: no sound, interface sounds, or anthropomorphized sounds. Results indicate that while sound had no significant effect on anthropomorphism, it significantly enhanced perceptions of animacy and likeability, particularly when anthropomorphized sounds were used. These findings highlight the role of sound in fostering emotional engagement and lifelikeness in interfaces, with implications for designing more dynamic and engaging user experiences.
The Conduction Series is a collaborative live radio broadcast produced by sound and transmission artists across the Americas on Wave Farm'sWGXC 90.7-FM Radio for Open Ears in New York's Upper Hudson Valley. The collective comes together on the first Friday of every month at 3:02pm ET using a custom web audio platform. Emphasizing live interactivity and media archaeological methods, our work explores the possibilities of remote sonic collaboration at scale and across borders. This paper discusses the history of and motivation behind our project, and the mixed methods we employ that place the development of bespoke network audio software in conversation with ongoing creative practices in live performance.
Autonomous Sensory Meridian Response (ASMR) is a tingling sensation in the neck and spine often triggered by specific sounds. This paper reports a study on the impact of different cyclic patterns and spatial orientations-defined here as the perceived directionality and motion of sound sources in a three-dimensional auditory space-on inducing ASMR experiences. The results demonstrate that both the type of cyclic pattern and the spatial orientation significantly influence the intensity and nature of ASMR experiences. Furthermore, the research explores synthesizing ASMR-inducing sounds while preserving key audio characteristics from acoustically recorded ASMR content. Through survey data analysis and regression modeling, distinct patterns emerge regarding the relationship between personality traits and ASMR experience. The findings contribute to a deeper understanding of ASMR as a sensory phenomenon and provide insights into the potential applications of artificially generated ASMR stimuli. Additionally, the research sheds light on the role of spatiality in ASMR experiences and the synthesis of ASMR-inducing sounds for future studies and practical applications.
The layout of the Audio Mixing Interface (AMI) has remained consistent since it was first conceived over 50 years ago. Whilst usability studies have evaluated alternative AMI designs against this established paradigm, they often lack a holistic perspective. Recently, user experience approaches have been used in ethnographic studies to explore the way music producers use technology. This qualitative study explores novice users experience of mixing popular music. Data was collected via a survey, the user experience questionnaire, and semi-structured group interviews after completing a mixing task with a Digital Audio Workstation (DAW) and analogue mixing console. The analysis revealed that the participants favoured the analogue AMI over the DAW but there were numerous issues with both AMIs. Navigating the AMI, particularly with the DAW, was the biggest reported issue. Furthermore, the DAW's adaptation of the analogue mixing console does not fully translate, partly due to the lack of tactile control. Novice users enjoyed the additional visual feedback provided by the DAW, specifically for EQ. These findings highlight novice AMI users needs, which are only partly met by the DAWand analogue mixing console. Ideally novice users require a combination of the simplified and tactile experience of the analogue AMI combined with the data visualisation enhanced visual feedback of the DAW to provide a more learnable AMI.
This paper introduces Motiv, a dataset of expert saxophonist recordings illustrating parallel, similar, oblique, and contrary motions. These motions are variations of three phrases from Jesus VillaRojo's "Lamento," with controlled similarities. The dataset includes 116 audio samples recorded by four tenor saxophonists, each annotated with descriptions of motions, musical scores, and latent space vectors generated using the VocalSet RAVE model. Motiv enables the analysis of motion types and their geometric relationships in latent spaces. Our preliminary dataset analysis shows that parallel motions align closely with original phrases, while contrary motions exhibit the largest deviations, and oblique motions show mixed patterns. The dataset also highlights the impact of individual performer nuances. Motiv supports a variety of music information retrieval (MIR) tasks, including gesture-based recognition, performance analysis, and motion-driven retrieval. It also provides insights into the relationship between human motion and music, contributing to real-time music interaction and automated performance systems.
In this study, audio designs were created with the aim of eliciting specific emotional states within a virtual reality (VR) environment. The study successfully replicated prior findings on the relationship between music and emotions and further investigated this relationship in VR and by using custom-designed audio compositions that adhered to a defined set of musical formulae. Six musicians were recruited to compose background music and audio feedback designs for two VR environment listening tasks. These musicians followed a set of design constraints informed by research on emotions and music for learning. The resulting audio designs were integrated into a VR scene where participants rated their emotional responses after exposure using a Self-Assessment Manikin (SAM) and PANAS in VR surveys. The audio designs targeted specific affective states - positive, negative, activating and deactivating. Results from this study indicate that audio adhering to specific characteristics such as tempo, melody directionality, rhythm complexity, harmony and scale can effectively induce emotional responses even while in VR and by creating custom-made audio designs.
Human and animal locomotion establishes a tight connection between movement in space and rhythm. Previous studies have explored the connection between rhythmic regularity and locomotor speed. In human-computer interaction, it has been shown that it is possible to control the trajectories of objects in two-dimensional space by producing rhythmic cells. The two studies here reported have the dual aim to investigate how well humans can (i) interpret overlapping rhythms as indications of direction and speed of a dragged moving object, and (ii) perform overlapping rhythms as corresponding to the speed and direction of a visual object. We assessed the accuracy and precision of these rhythmic trajectories using multisensory translation tasks.
Digital musical instruments (DMIs) offer new possibilities for music creation, but also raise challenges around how performers, composers and audiences connect through sound. Unlike traditional instruments where gestures are visible and physically linked to sound production. DMIs often separate interaction from output, making musical intention harder to perceive. This paper explores the challenges of composing for DMIs, particularly how actions are mapped to sound and how such compositions might be notated. Through an ecological perspective, we examine the relationships between performer, instrument and audience - relationships that are central to musical meaning but are often overlooked in digital performance. Our study follows the development of the DMI Soundscape, a project designed to investigate these connections across the full process of creating and performing with a DMI. Preliminary findings highlight the importance of designing for expressive, embodied interaction and call for new approaches to composition that support clarity, connection, and creativity in digital music.
This work examines user preferences for the design of real-time audio feedback while writing, sketching, and underlining on a tablet. Participants were provided with a GUI which allowed them to select between different mappings between stylus location, pressure, and acceleration and sonic parameters such as waveform type, frequency, amplitude, filter cutoff frequency and resonance. Subsequently, they provided their preference and responded to a short survey. Audio feedback had a positive impact on user engagement, especially while sketching and was considered to be an easy and engaging task. Mapping pen location to frequency and pen acceleration to cutoff frequency or resonance were preferred often. The results show that sound design should be specific to the task characteristics. Insights can be used for future studies into sound design for tasks performed with a stylus on a tablet.
In this paper we discuss our latest iteration of RAMI (the Robotic Arm for Musicking Inclusively) as prototype digital score. We deploy RAMI as interactive technology for the purpose of improvisation ensemble work, documenting the narrative journey of a single elite musician-M-as they engage with its nascent qualities over three extended workshop sessions. This study builds in part on the musical togetherness model of human-human interaction [4], and necessarily draws adjacent concepts in robotics, human computer interaction (HCI), and digital score. Recording musicking interactions with RAMI, we elicit and report on M's thematic perceptions of instances of: familiarization; engagement strategy; decision making; connection, agency, and ensemble; assertions of schemas of RAMI; and moments of enjoying RAMI. Beyond the rich, narrative insight that these six themes offer, we critically assess these as variably (un)certain in nature; as particularly certain when asserting two schemas-RAMI as a dancer, and RAMI as a score, and as indicative of an underlying continuum of musician perception that recognises RAMI as an AI tool for musicking, on the one hand, and on the other hand, as an AI agent for embodied collaboration.
This paper presents a method to use t-SNE for training low-dimensional timbre maps obtained from musical recordings in a way that similar sound spectra are represented at similar location within the map. To use such maps for generating novel sounds and timbre trajectories, we use Kernel Regression Mapping for the inverse transformation from map space to timbre space and Griffin-Lim algorithm for phase reconstruction. With this mix of methods, we achieve a way to both visually explore timbral patterns and compress control data. As an application for navigable timbre maps we introduce a novel method - related to and inspired from Wave Space Sonification (WSS) - for the auditory exploration of patterns in multivariate time-series data by using the semantic sound map as the connecting representation. We demonstrate our approach by sonifying ECG data relating to cardiac pathologies.
This paper introduces Pletude, a game engine-based score merging music notation, gaming, and improvisation into a single artistic experience. Building on the evolution of graphic scores and digital notation, Pletude presents non-traditional musical notation in the form of a video game where performers use acoustic instruments to control an on-screen avatar. Players navigate a two-dimensional game space to collect points, avoid enemies, and interact with allies and power-ups. Its four-note control system, inspired by early arcade games like Pac-Man, and its visual design, rooted in modernist abstract art and visual music, fuse gaming and musical expression. This paper situates Pletude and other game engine-based scores within the broader history of music notation and its relationship to the visual arts, examining how game elements such as goals, rules, constraints, and conflict function as musical notation. It then delves into the design, mechanics, and technical implementation of Pletude, alongside its artistic influences. Finally, the paper discusses recent performances and player experiences, emphasizing Pletude's accessibility and ability to foster improvisation and collaboration among musicians of all skill levels. Pletude offers a paradigm for interactive and participatory music-making, challenging traditional notions of composition and performance while opening new possibilities for interdisciplinary artistic practice.
In physical therapy and training it is crucial that exercises are performed correctly while performance is improved. The sonification of movements has potential in this context, especially for unsupervised exercises at home. Our exploratory design research study focused on sound design for skater jumps. Two sound stimuli were created by expert sound designers and evaluated with 17 participants: a noise-based sound with edited and layered ice skating recordings and a complex tonal design based on recorded string instruments. The evaluation assessed jumping force, user experience, movement potential and sonic associations. The noise-based stimulus significantly increased the average maximum force during jumps compared to a metronome sound. It was well received because of its movement potential and its relation to movement. In contrast, the tonal stimulus had no effect on jumping force, but was useful for movement guidance and was positively received by most participants. The study shows that custom-designed complex sounds can improve both movement execution and user experience in skater jump exercises and that causal certainty is not necessary for effective sound design in this context. This insight opens up a wide scope for sound designers to create novel, original designs beyond conventional approaches.
This paper explores the music-making capabilities of a swarm intelligence type of algorithm known as self-assembly in an interactive context using Extended Reality (XR) technologies. We describe the modifications made to a fully autonomous version of this algorithm, which we proposed in a previous work, allowing us to adapt it for user-interactive music. Moreover, we present the design of an XR system that supports this adaptation, modelled as a human-swarm interactive music system, which is implemented in the Meta Quest 3 headset. An auto-ethnographic study was conducted to discover the affordances of the system in a music improvisation session. The study, supported by empirical measurements collected during the session, enables a comparison between the interactive version and the original autonomous offline version, providing valuable insights into how a user can influence the swarm's behaviour. The results are used to discuss the music performance possibilities and future directions for this type of interactive music system.
Authoring 3D audio in immersive environments has great potential to design spatial paths as they will be perceived by listeners. Especially drawing such paths for audio directly in space gives direct access to expressive trajectories. But editing those trajectories can be tedious if now simplified, where any simplification will reduce expression again. Thereby, we propose a workflow for using spatial audio paths as interface to interact with the trajectories. For that, we introduce our fitting algorithm that runs on common standalone XR HMD's to simplify trajectories for interaction, while preserving the original data for playback. With that a discussion on spatial expression is given.
Enacted across the disciplines of sound art and HCI, this paper explores physical behaviours associated with sound installation listening and their mediation by social factors. It presents an ethnomethodological study conducted during the public exhibition of an ultrasonic installation, Being With The Waves. Inaudible to the naked ear, the artwork is heard via custom headphones and, as an experience, manifests differently for individual listeners according to their spatial orientation to speakers, body position, movement, and perceptual behaviour. Arguably, the installation is antisocial by design, foregrounding private rather than collective experience. However, up to six listeners may experience it together, creating a social dimension that inevitably mediates behaviour. An interaction analysis of video observations identifies important ways in which the physical behaviour of listeners appears to be mediated by the presence (or absence) of other people in the exhibition space. The study's findings indicate that the local social dynamic strongly impacts how listeners move and use their bodies, which affects the discovery and exploration of spatial and interactive effects. Critiquing the study from a feminist new materialist perspective, methodological changes are considered that might connect social behaviours with the material design of the installation and foreground situated knowledge.
We introduce RhyGlyph, a radial glyph-based visualization system for the representation of rhythmic interactions. Two rhythm tracing techniques -Dot and Meter- spatially map rhythm relationships among three core drum set instruments: kick, snare, and hi-hat. These interactions are depicted within three 120-degree wedges of the radial glyph. Starting from the inter-onset intervals (IOIs) of the individual sounding objects, three spacetime trajectories are computed and projected on thewedges. After providing background on glyphs and rhythm visualization techniques, we describe the design of RhyGlyph and present the functionalities of the prototype being demonstrated. The research and development of RhyGlyph is continuing along some directions, which are briefly outlined.
This paper presents the design of an original musical interface for a one-off performance in conjunction with a live football match. The instrument was built using a repurposed Gametrak string controller, transforming it into a haptic device that embraced limitations as a generative design space for musical expression. The paper will focus on the design of the interface and its use in rehearsals and performance, presenting it in context of other Gametrak-based work. I will argue that the hardware functions as a passive haptic device, and that the constraints provide opportunities for performance. I will also suggest that a growing body of Gametrak-based work warrants further investigation.
The impact of Artificial Intelligence is felt on every stage of contemporary musicking and is shaping our interaction with sound. Deep learning Generative AI (GenAI) systems for high-quality music generation rely on extremely large musical datasets for training. As a result, AI models tend to be trained on dominant mainstream musical genres, such as Western classical music, where large datasets are more readily available. In addition, the reliance on extremely powerful computing resources for deep learning creates barriers to use and negatively impacts our environment. This paper reports on contemporary concerns and interests of musicians, researchers, and music industry stakeholders in the responsible use of GenAI models for music and audio. Through analysis of focus group discussions and exemplar case studies of the use of GenAI in music making at a hybrid workshop of 148 participants, we offer insights into current discourses about the use of GenAI beyond dominant musical styles and suggest ways forward to increase creative agency in music making beyond the mainstream. Our findings highlight the value of small datasets of music for GenAI, the suitability of AI models for working with small datasets of music, and pose questions around what constitutes a 'small' dataset of music.
