A 12 lead electrocardiogram (ECG) is the standard diagnostic method for the detection of an acute coronary syndrome, as it is also used in emergency medical services. A novel sonification method can convert an important part of the ECG signal into an acoustic signal: The ST segment sonification is particularly useful for the detection of transient ST elevations in patients with suspicion of acute coronary syndrome. A quick and accurate detection of transient ECG changes of the ST segment is prerequisite for proper treatment, thus having immediate therapeutic consequences. As part of an emergency training program, a cohort of n = 44 medical students was recruited to participate in a two-part study. Some of them, namely n=32 of the total 44 subjects, participated in a second part of the study, an RCT, which we report on here. The diagnostic accuracy recently estimated in a classification study involving all 44 subjects with regard to acoustically presented ECG sequences of varying degrees of severity of ST-elevation myocardial infarction forms the background for the RCT described here. The n=32 subjects who participated in the RCT were randomly assigned in two-person teams to either an intervention ( n=8 teams of two) or a control ( n=8 teams of two) arm, respectively, whereby all teams, except for one dropout due to a technical failure in the intervention arm, went through an emergency simulation where they had to detect an emerging ST elevation myocardial infarction. The intervention group was endowed with a sonification-assisted equipment whereas the control group used standard visual-based ECG diagnosis only. An adjusted multivariable regression yielded a statistically significant reduction for the intervention group of the delay time from starting a first ECG to the correct diagnosis by 163 seconds ( p = 0.002 ) corresponding to 56% of the average delay time in the control group. A subgroup analysis within the intervention arm revealed a notable impact of the attitude toward sonification on delay time between the second ECG and diagnosis. Specifically, increasing disagreement with statement “I conceived the sound of the sonification as pleasant” counterintuitively reduced the delay, whereas an increasing disagreement with “sonification was helpful in making the diagnosis” increased the delay. The sonification of ECG proved to be significantly superior as an accompanying diagnostic measure in emergency medical services in cases of suspected acute coronary syndrome in a simulated emergency scenario in terms of a proof-of-concept. The established dependence on individual attitudes towards sonification serves to further optimize sonification aesthetics and implementation with a focus on greater alertness and the reduction of stress-induced destraction or alarm fatigue.

This paper presents a method to use t-SNE for training low-dimensional timbre maps obtained from musical recordings in a way that similar sound spectra are represented at similar location within the map. To use such maps for generating novel sounds and timbre trajectories, we use Kernel Regression Mapping for the inverse transformation from map space to timbre space and Griffin-Lim algorithm for phase reconstruction. With this mix of methods, we achieve a way to both visually explore timbral patterns and compress control data. As an application for navigable timbre maps we introduce a novel method - related to and inspired from Wave Space Sonification (WSS) - for the auditory exploration of patterns in multivariate time-series data by using the semantic sound map as the connecting representation. We demonstrate our approach by sonifying ECG data relating to cardiac pathologies.

We introduce a novel technique for the sonification/auditory representation of a 12-lead electrocardiogram (ECG), the standard diagnostic method for the detection of ST elevation myocardial infarction (STEMI). Our approach to ST elevation sonification conveys the detailed variation of the ST segment to enable differentiated, correct interpretation and severity without consulting a visual display. We present a variety of novel sonification designs and discuss their benefits and limitations. As part of an emergency training program, a cohort of 44 medical students (5th academic year) participated in a classification study in which the diagnostic accuracy of the participants was determined with regard to audibly presented ECG sequences of different STEMI severity levels. Regarding the classification of sonified ECG sequences, the discrimination of isoelectricity (IE, the healthy class) from all other (STEMI) classes combined yielded a perfect classification of all 660 classification instances (sensitivity = specificity = 1). With respect to the individual classification of all five classes (IE, inferior/anterior, and moderate/severe STEMI), an overall accuracy of 0.82 (0.79, 0.85) and an intraclass coefficient of κ=0.77 were estimated.

This paper introduces sonecules, a flexible, extensible, end-user friendly and open-source Python sonification toolkit to bring ’sonification to the masses’. The package comes with a basic set of what we define as sonecules which are sonification designs tailored for a given class of data, a selected internal logic for sonification and offering a set of functions to interact with data and sonification controls. This is a design-once-use-many approach as each sonecule can be reused on similarly structured data. The primary goal of sonecules is to enable novice users to rapidly get their data audible – by scaffolding their first steps into auditory display. All sonecules offer a description for the user as well as controls which can be adjusted easily and interactively to the selected data. Users are supported to get started as fast as possible using different sonification designs and they can even mix and match sonecules to create more complex aggregated sonecules. Advanced users are enabled to extend/modify any sonification design and thereby create new sonecules. The sonecules Python package is provided as open-source software, which enables others to contribute their own sonification designs as a sonecule – thus it seeds a growing/growable library of well-documented and easy-to-reuse sonifications designs. Sonecules is implemented in Python using mesonic as the sonification framework, which provides the path to rendering-platform agnostic sonifications.

Facial behavior occupies a central role in social interaction. Its auditory representation is useful for various applications such as for supporting the visually impaired, for actors to train emotional expression, and for supporting annotation of multi-modal behavioral corpora. In this paper we present a prototype system for interactive sonification of facial behavior that works both in real-time mode, using a webcam, and offline mode, analyzing a video file. The system is based on python and Jupyter notebooks, and relies on the python module sc3nb for sonification-related functionalities. Facial feature extraction is realized using OpenFace 2.0. Designing the system led to the development of a framework of reusable components to develop interactive sonification applications, called Panson, which can be used to easily design and adapt sonifications for different use cases. We present the main concepts behind the facial behavior sonification system and the Panson framework. Furthermore, we introduce and discuss novel sonifications developed using Panson, and demonstrate them with a set of sonified videos. The sonifications and Panson are Open Source reproducible research available on GitHub.

Introduction: ST segment elevation myocardial infarction is a common reason for out-of-hospital cardiac arrest in adult patients. The surveillance of the ST segment in the electrocardiogram is limited to visual presentation. However, the ST segment can change during the course of treatment. If ST elevation is present immediate coronary revascularization is needed, therefore detecting ST elevation changes the treatment fundamentally. Sonification of the ST segment is a new method which enables the emergency team to detect intermediate changes of the ST segment. Material and methods: We have chosen two sonification designs which were introduced to two groups, medical students and computer science students. Twenty-one participants took part in the study. The sonification was designed for evaluation of the ST segment. The user was supposed to become empowered to distinguish between no, medium-low, medium-high or extreme ST elevation by listening to the sonification. The two groups were asked to evaluate the sounds for possible ST elevation as well as for aesthetics and usability. In a second study twenty-five medical students were taking part in a medical scenario in which sonification was played during a simulated case. The patient was suffering from a myocardial infarction, ST elevation was transient and sonification sounds were changing appropriately. The students were supposed to detect these changes and act accordingly by modifying the treatment. Results: Both groups were able to classify ST segment elevation by listening to the sonification samples. The higher the ST segment, the better was the detection rate overall. In all of the three categories (pleasantness, informativeness and long-term listening) the Water Ambience sonification was rated higher compared to the Polarity sonification. Moreover, in the two groups that took part in the study, we found a significant difference when comparing classification performance using both sonification designs. For the group of medical students as t(20) = 4.31, p = 3.44 x 10(-4), p < 0.01 and for the computer science students as t(19) = 3.40, p = 9.39 x 10(-6), p < 0.01. In the simulated medical scenario participants indicated that 96% detected the ST elevation. 60% stated that sonification played a role whereas for 32% it did not play a role for the detection of ST elevation. Conclusions: Sonification has the potential to play an important role as a new supporting tool for the surveillance of the ST segment during the care of patients with suspicion of myocardial infarction. It can be helpful to differentiate between ST segment elevation myocardial infarction and non-ST segment myocardial infarction especially if ST elevation is transient. Furthermore, sonification is viewed as pleasant to listen to and might not contribute to alarm fatigue. (C) 2022 Elsevier Inc. All rights reserved.

Auditory feedback from everyday interactions can be augmented to project digital information in the physical world. For that purpose, auditory augmentation modulates irrelevant aspects of already existing sounds while at the same time preserving relevant ones. A strategy for maintaining a certain level of plausibility is to metaphorically modulate the physical object itself. By mapping information to physical parameters instead of arbitrary sound parameters, it is assumed that even untrained users can draw on prior knowledge. Here we present AltAR/table, a hard- and software platform for plausible auditory augmentation of flat surfaces. It renders accurate augmentations of rectangular plates by capturing the structure-borne sound, feeding it through a physical sound model, and playing it back through the same object in real time. The implementation solves basic problems of equalization, active feedback control, spatialization, hand tracking, and low-latency signal processing. AltAR/table provides the technical foundations of object-centered auditory augmentations, for embedding sonifications into everyday objects such as tables, walls, or floors.

This paper introduces sc3nb, a Python package for audio coding and interactive control of the SuperCollider programming environment. sc3nb supports Jupyter notebooks, enables flexible means for sound and music computing such as sound synthesis and analysis and is particularly tailored for sonification. We present the main concepts and interfaces and illustrate how to use sc3nb at hand of selected code examples for basic sonification approaches, such as audification and parameter-mapping sonification. Finally, we introduce TimedQueues which enable coordinated audiovisual displays, e.g. to synchronize matplotlib data and sc3nb-based sound rendition. sc3nb enables interactive sound applications right in the center of the pandas/numpy/scipy data science ecosystem. The open source package is hosted at GitHub and available via the Python Package Index PyPI.

Intra-hemispheric interference has been often observed when body parts with neighboring representations within the same hemisphere are stimulated. However, patterns of interference in early and late somatosensory processing stages due to the stimulation of different body parts have not been explored. Here, we explore functional similarities and differences between attention modulation of the somatosensory N140 and P300 elicited at the fingers vs. cheeks. In an active oddball paradigm, 22 participants received vibrotactile intensity deviant stimulation either ipsilateral (within-hemisphere) or contralateral (between-hemisphere) at the fingers or cheeks. The ipsilateral deviant always covered a larger area of skin than the contralateral deviant. Overall, both N140 and P300 amplitudes were higher following stimulation at the cheek and N140 topographies differed between fingers and cheek stimulation. For the N140, results showed higher deviant ERP amplitudes following contralateral than ipsilateral stimulation, regardless of the stimulated body part. N140 peak latency differed between stimulated body parts with shorter latencies for the stimulation at the fingers. Regarding P300 amplitudes, contralateral deviant stimulation at the fingers replicated the N140 pattern, showing higher responses and shorter latencies than ipsilateral stimulation at the fingers. For the stimulation at the cheeks, ipsilateral deviants elicited higher P300 amplitudes and longer latencies than contralateral ones. These findings indicate that at the fingers ipsilateral deviant stimulation leads to intra-hemispheric interference, with significantly smaller ERP amplitudes than in contralateral stimulation, both at early and late processing stages. By contrast, at the cheeks, intra-hemispheric interference is selective for early processing stages. Therefore, the mechanisms of intra-hemispheric processing differ from inter-hemispheric ones and the pattern of intra-hemispheric interference in early and late processing stages is body-part specific.

This paper presents the design and evaluation of four sonification methods to support monitoring and diagnosis in Electrocardiography (ECG). In particular we focus on an ECG abnormality called ST-elevation which is an important indicator of a myocardial infarction. Since myocardial infarction represents a life-threatening condition it is of essential value to detect an ST-elevation as early as possible. As part of the evaluated sound designs, we propose two novel sonifications: (i) Polarity sonification , a continuous parameter-mapping sonification using a formant synthesizer and (ii) Stethoscope sonification , a combination of the ECG signal and a stethoscope recording. The other two designs, (iii) the water ambience sonification and the (iv) morph sonification , were presented in our previous work about ECG sonification (Aldana Blanco AL, Steffen G, Thomas H (2016) In: Proceedings of Interactive Sonification Workshop (ISon). Bielefeld, Germany). The study evaluates three components across the proposed sonifications (1) detection performance, meaning if participants are able to detect a transition from healthy to unhealthy states, (2) classification accuracy, that evaluates if participants can accurately classify the severity of the pathology, and (3) aesthetics and usability (pleasantness, informativeness and long-term listening). The study results show that the polarity design had the highest accuracy rates in the detection task whereas the stethoscope sonification obtained the better score in the classification assignment. Concerning aesthetics, the water ambience sonification was regarded as the most pleasant. Furthermore, we found a significant difference between sound/music experts and non-experts in terms of the error rates obtained in the detection task using the morph sonification and also in the classification task using the stethoscope sonification . Overall, the group of experts obtained lower error rates than the group of non-experts, which means that further training could improve accuracy rates and, particularly for designs that rely mainly on pitch variations, additional training is needed in the non-experts group.

Despite 2-factor authentication and other modern approaches, authentication by password is still the most commonly used method on the Internet. Unfortunately, as analyses show, many users still choose weak and easy-to-guess passwords. To alleviate the significant effects of this problem, systems often employ textual or graphical feedback to make the user aware of this problem, which often falls short on engaging the user and achieving the intended user reaction, i.e., choosing a stronger password. In this paper, we introduce auditory feedback as a complementary method to remedy this problem, using the advantages of sound as an affective medium. We investigate the conceptual space of creating usable auditory feedback on password strength, including functional and non-functional requirements, influences and design constraints. We present web-based implementations of four sonification designs for evaluating different characteristics of the conceptual space and define a research roadmap for optimization, evaluation and applications.

Every day, we rely on the information that is encoded in the auditory feedback of our physical interactions. With the goal to perceptually enhance those sound characteristics that are relevant to us — especially within professional practices such as percussion and auscultation — we introduce the method of real-time Auditory Contrast Enhancement (ACE). It is derived from algorithms for speech enhancement as well as from the remarkable sound processing mechanisms of our ears. ACE is achieved by individual sharpening of spectral and temporal structures contained in a sound while maintaining its natural gestalt. With regard to the targeted real-time applications, the proposed method is designed for low latency. As the discussed examples illustrate, it is able to significantly enhance spectral and temporal contrast.

We introduce Auditory Contrast Enhancement (ACE) as a technique to enhance sounds at hand of a given collection of sound or sonification examples that belong to different classes, such as sounds of machines with and without a certain malfunction, or medical data sonifications for different pathologies/conditions. A frequent use case in inductive data mining is the discovery of patterns in which such groups can be discerned, to guide subsequent paths for modelling and feature extraction. ACE provides researchers with a set of methods to render focussed auditory perspectives that accentuate inter-group differences and in turn also enhance the intra-group similarity, i.e. it warps sounds so that our human built-in metrics for assessing differences between sounds is better aligned to systematic differences between sounds belonging to different classes. We unfold and detail the concept along three different lines: temporal, spectral and spectrotemporal auditory contrast enhancement and we demonstrate their performance at hand of given sound and sonification collections.

This paper introduces systematic research on how uni- and bimanual tapping tasks can profit from interactive sonification of hand-surface interactions to support coordination. To that end, we developed a new experimental platform featuring a web app that allows multi-touch tablet-based interaction for sonification experiments. We present and discuss two experiments to test tap- interval-to-tone-frequency-mappings, to assess their ability to support rhythmic, resp. polyrhythmic tapping. The results, although negative with regards to our initial hypothesis, provide guidance for subsequent experiments to better understand how sonification can best be used to support coordination tasks, particularly in motor control contexts that involve discrete, goal- directed movements.