High-performance machine learning models generally suffer from a lack of transparency that calls into question the ability of humans to understand how they work. Through the study of CoMo, an interactive learning system that allows users to associate sounds and gestures, we show that a form of model understanding can emerge from user-model interactions. This interactive approach offers an alternative to explanation- and interpretation-based epistemologies. Although data- and computation-intensive deep learning systems are not well suited to this form of interactivity, shallow learning models offer underappreciated epistemological advantages in contexts where machine learning can be made interactive.

We present CoMo, a web-based ecosystem developed for movement-sound interaction. After several years of using a first version in workshops, artistic projects, and rehabilitation applications, we conducted interviews and specific trials with experts from different disciplines to guide an upgrade of the system. In this paper, we report the general motivation and initial implementation of the application, as well as the results of the interviews, which helped specify different use cases and informed the design of the revised version. This open-source ecosystem should benefit the Movement and Computing community at large by enabling the rapid development of a wide range of prototypes for movement-sound interactions.

Hip-hop dance battles are events where dancers improvise to an unfamiliar DJ’s mix. The dancers’ technicality in responding to the music, along with the collective dimensions of these encounters, represents a largely unexplored area of investigation in HCI. Using an autobiographical design approach grounded in the hip-hop practice of the first author, we developed five interaction scenarios on a hip-hop–specific movement vocabulary. Our interactive sonification enables both individual and collective improvisation through synchronized motion sensing and interactive sound loops. This approach allowed us to design three interconnected workshops to investigate how dancers use sonification in improvisation. Our findings show that sonification was perceived either as a validation of movement or as a medium for open-ended exploration. We contribute with: (1) the formalization of a movement vocabulary and the design of interactive scenarios for hip-hop movement sonification, (2) insights into how dancers experience sonification through exploration and improvisation, and (3) guidelines for the design of movement sonification in hip-hop contexts.

In this paper, we propose to report and reflect on a long- term collaboration between researchers and a musician from the improvisation scene, to design and implement a Web- based distributed system - composed of 8 Raspberry Pi nano-computers equipped with speakers and batteries con- nected together through a local WiFi network - dedicated to co-located settings and live performance. The paper first presents the different applications developed during this col- laboration. Then we propose to reflect on some of the as- pects of the collaboration that appear salient during the project and that we think have important implications both for design and for the successful implementation of research- creation methodologies. More precisely, we propose to un- fold the different socio-economical questions and constraints that we noticed during the collaboration. We examine the learning and appropriation process and analyse it in the framework of the Instrumental Genesis Theory. Finally, we reflect on how artistic practice and design considerations in- teract in the creation of a Digital Music Instrument (DMI) as a co-evolutionary process.

Storytelling is an essential practice in preschool. Recent advances in digital tools for multimodal and collective interactions represent opportunities for new approaches in storytelling. Motivated by an embodied approach of storytelling, we developed CoMo•education, an interactive system where whole-body movements and sounds are in play. Moreover, this application enables storytelling as a collective and multimodal experience. Using gesture recognition with screenless smartphones, the teacher and pupils can use movements to play various soundscapes, which enliven the story. The aim of this article is to present the overall project, which has spanned more than 3 years of iterative developments. Importantly, it was developed through an iterative co-design process including designers, teachers, early-development experts and engineers. A user study was carried out in two kindergarten classes which allowed us to gather feedback from teachers and children, and formalize a series of guidelines for future works. A workshop was also dedicated to educators to evaluate how educators can appropriate the application to create interactive stories by themselves.

Agents engaged in creative joint actions might need to find a balance between the demands of doing something collectively, by adopting congruent and interacting behaviors, and the goal of delivering a creative output, which can eventually benefit from disagreements and autonomous behaviors. Here, we investigate this idea in the context of collective free improvisation-a paradigmatic example of group creativity in which musicians aim at creating music that is as complex and unprecedented as possible without relying on predefined plans or individual roles. Controlling for both the familiarity between the musicians and their physical copresence, duos of improvisers were asked to freely improvise together and to individually annotate their performances with a digital interface, indicating at each time whether they were playing with, against, or without their partner. At an individual level, we found that musicians largely intended to converge with their coimproviser, making only occasional use of noncooperative or noninteractive modes such as playing against or playing without. By contrast, at the group level, musicians tended to combine their relational intents in such a way as to create interactional dissensus. We also demonstrate that copresence and familiarity act as interactional smoothers: They increase the agents' overall level of relational plasticity and allow for the exploration of less cooperative behaviors. Overall, our findings suggest that relational intents might function as a primary resource for creative joint actions.

In this paper, we present Simone, a distributed musical instrument for collective improvisation created using a research through design approach. The collective interaction with Simone is performed through different interaction scenarios specifying how data is exchanged between users on the local network. First, we present the design and implementation of Simone including the audio synthesis system and the different scenarios. Then, we present a study with groups of expert users, asking them to improvise collectively with Simone to observe their appropriation of the instrument and how they interact collectively. Qualitative data analysis shows that the process of appropriation is a complex phenomenon that depends on participants’ musical background and experience and is a preliminary condition for collaboration. Additionally, our results reveal participants’ perception of networked elements and the influence it had on their collective interaction. Finally, we discuss the design of collective interaction and distributed music instruments.

In this paper, the authors present two related libraries, web-audio-api-rs and nodeweb-audio-api, that provide a solution for using the Web Audio API outside the Web browsers.The first project is a low-level implementation of the Web Audio API written in the Rust language, and the second provides bindings of the core Rust library for the Node.jsplatform.The authors' approach here is to consider Web standards and specifications as tools for defining standardized APIs across different environments and languages, which they believe could benefit the audio community in a more general manner.Although such a proposition presents some portability limitations due to the differences between languages, the authors think it nevertheless opens up new possibilities in sharing documentation, resources, and components across a wide range of environments, platforms, and users.The paper first describes the general design and implementation of the authors' libraries.Then, it presents some benchmarks of these libraries against state-of-the-art implementation from Web browsers, and the performance improvements that have been made over the last year.Finally, it discusses the current known limitations of these libraries and proposes some directions for future work.The two projects are open-source, reasonably feature-complete, and ready to use in production applications.

This short paper reports on recent advances in the development of an open-source prototyping platform for the creation of distributed and embedded musical systems based on Web technologies. The proposed architecture is based on user-grade hardware and open-source software and aims at fostering rapid-prototyping and experimentation in the context of colocated distributed systems for musical research, creation and performance. After a short review of related works, the paper describes the general design and the different building blocks of the system. Then, it exposes a first characterization of the proposed design, and concludes with the description of a prototype that highlights different features of the platform.

Nous présentons Simone [simɔn], un système Web pour l'improvisation collective en réseau basé sur des entrées audio (microphone). Inspirés par plusieurs exemples de systèmes de performance musicale en réseau, notre objectif était de concevoir un instrument distribué basé sur 3 concepts : l’improvisation collective, l’utilisation de la voix dans le processus de synthèse sonore, et l’échange d’information par le réseau. Simone est développé avec des technologies web et est utilisable avec un équipement matériel léger, portable et adapté à plusieurs situations. Cet article présente en premier lieu le design du système ainsi que de son implémentation technique. En second lieu, nous présentons des premiers exemples d’utilisation de Simone via un atelier réalisé avec 4 participants ayant improvisé avec le système, et via une étude au plus long cours avec un artiste improvisateur ayant utilisé le système dans différents contextes pendant plusieurs mois. Simone est disponible au téléchargement sous licence BSD-3-Clause à l’adresse suivante : https://github.com/ircam-ismm/simone

In this article, we discuss some of our research with Local Area Networks (LAN) in the context of sound installations or musical performances. Our systems, built on top of Web technologies, enable novel possibilities of collective and collaborative interaction, in particular by simplifying public access to the artwork by presenting the work through the web browser of their smartphone/tablet. Additionally, such a technical framework can be extended with so-called nano-computers, microprocessors and sensors. The infrastructure is completely agnostic as to how many clients are attached, or how they connect, which means that if the work is available in a public space, groups of friends, or even informally organised flash mobs, may engage with the work and perform the contents of the work at any time, and if available over the Internet, at any place. More than the technical details, the specific artistic directions or the supposed autonomy of the agents of our systems, this article focuses on how such ‘networks of devices’ interleave with the ‘network of humans’ composed of the people visiting the installation or participating in the concert. Indeed, we postulate that an important point in understanding and describing such proposals is to consider the relation between these two networks, the way they co-exist and entangle themselves through perception and action. To exemplify these ideas, we present a number of case studies, sound installations and concert works, very different in scope and artistic goal, and examine how this interaction is materialised from several standpoints.

[...]there are many forms of networks: from biological, mycorrhizal networks that connect plants in mutualistic partnerships to personal community or business networks to electronic and digital networks. Many artists and researchers working in sound have explored the concept of ‘the network’ and developed a wide range of musical practices such as tele-presence performances, telematics artwork, network jamming, streaming or multiuser interactive environments. An international conference on soundscape at Banff, Alberta, in 1993, to mark Schafer’s 60th birthday, established the World Forum for Acoustic Ecology (WFAE), the pre-eminent international body, seeking to foster a ‘desirable acoustic community where sound connects all of its inhabitants’.

Does being part of an orchestra from an early age have an impact on cognitive and emotional capacities? Researchers from the University of Geneva (Unige), Switzerland, the University of Genoa (Unige-IT), and the Institute of Research and Coordination Acoustic/Music (IRCAM) in Paris investigated this question in the context of the Demos orchestras organized by the Philharmonie de Paris-Cité. This two-year study evaluated the development and evolution of the cognitive, emotional, and motor skills of 255 children engaged in two Demos orchestras, aged 7 to 13 years living in education priority zones (ZEPs) in France. The results showed that after two years of playing music as part of a group, the working memory, cognitive flexibility, and emotion understanding skills of the children improved at a higher rate than the published measurements on standard tests. Important improvements were also shown in movement synchronization.

This paper presents [°poly°], a generative and interactive music installation based on networked, tangible and semi-autonomous modules. After a presentation of the research objectives and inspirations, the paper describes the hardware and software implementation, and finally reports on the main compositional aspects. A video presentation is available at https: //youtu.be/6azZ5_TcmkE.

This paper presents soundworks, a framework dedicated to prototyping and developing distributed multimedia applications using Web technologies. Since its first release in 2015, the framework has been used in numerous artistic and research projects such as concerts, installations, workshops, teaching, and experimental setups. We first present how this diversity of contexts and objectives permitted us to identify a set of patterns able to support recurring needs of expert users in exploratory tasks. We then detail new developments that have been achieved to provide better support to these patterns. More particularly, we describe the novel distributed state management system dedicated to simplifying the implementation of remote control and monitoring interfaces and the plug-in system implemented to improve the extensibility of the framework and foster composition of dedicated functionalities. We believe that these new developments can provide a solid ground for further research and artistic practices in the area of distributed music systems. The soundworks framework is open source and released under BSD-3-Clause license.