We explore the practical advantages of using the Julia programming language for music and audio data processing tasks. Julia is a high-level dynamic programming language for technical computing characterized by its familiar and concise syntax and high performance. This paper reviews the features of Julia that are useful for audio signal processing, and introduces JuliaAudio and MusicProcessing.jl, which provide a set of Julia packages for basic I/O and transformations of audio data as well as various feature extraction methods for music information retrieval tasks. We quantitatively evaluate the package in terms of its performance relative to existing audio feature extraction libraries. We argue that using Julia for music and audio processing brings a number of benefits to programmers and researchers, including its high performance in numerical computations, the ease of development coming from Julia's conciseness and versatility, and its scalability for distributed computing.
Content-based Music Informatics includes tasks that involve estimating the pitched content of music, such as the main melody or the bass line. To date, the field lacks a good machine representation that models the human perception of pitch, with each task using specific, tailored representations. This paper proposes factoring pitch estimation problems into two stages, where the output of the first stage for all tasks is a multipitch contour representation. Further, we propose the adoption of pitch contours as a unit of pitch organization. We give a review of the existing work on contour extraction and characterization and present experiments that demonstrate the discriminability of pitch contours.
Most of recent studies on vocal detection in audio recordings focus either on the development of new features or on classification methods. The impact of training and test data is largely neglected, leading to weaknesses in the design of databases which do not cover differences of vocal techniques across music genres. In this paper we compare approaches for singing voice detection on individual genres. For those methods, which gave the best performance, we further investigate the impact of disjoint distribution of training and test tracks with regard to their genres. In particular, the tracks of electronic genres, which are barely contained in public databases for vocal recognition, contribute to a better classification performance identifying vocals in tracks of other genres.
A Matlab/GNU Octave toolbox for phase (signal) reconstruction from the short-time Fourier transform (STFT) magnitude is presented. The toolbox provides an implementation of various, conceptually different algorithms ranging from the well known Griffin-Lim algorithm and its derivatives to the very recent ones. The list includes real-time capable algorithms which are also implemented in real-time audio demos running directly in Matlab and GNU Octave. The toolbox is well-documented, open-source and it is available under the GPL3 license. In this paper, we give an overview of the algorithms contained in the toolbox and discuss their properties.
In this paper, a system for automatic transcription of multiple-instrument polyphonic music is proposed, which supports tracking multiple concurrent notes using linear dynamical systems (LDS). The system is based on a spectrogram factorisation model which extends probabilistic latent component analysis (PLCA), and supports the detection of multiple pitches, instrument contributions, and pitch deviations. In order to jointly track multiple concurrent pitches, the use of LDS as prior to the PLCA model is proposed. LDS parameters are learned in a training stage using score-informed transcriptions; for LDS inference, online and offline variants are evaluated. The MAPS piano music dataset and the Bach10 multi-instrument dataset are used for note tracking experiments, with the latter dataset also being evaluated with respect to instrument assignment performance. Results show that the proposed LDS-based method can successfully track multiple concurrent notes, leading to an improvement of over 3% in terms of note-based F-measure for both datasets over benchmark note tracking approaches.
This work presents a system that is able to provide fine-grained time-dependent context while listening to recorded music. By utilizing acoustic fingerprinting techniques the system recognizes which music is playing in the environment and also determines an exact playback position. This makes it possible to provide context at exactly the right time. The design of the system can be used to augment listening experiences with lyrics, scores, tablature or even music videos. To test the concept, a prototype has been built that is able to give feedback that coincides with the beat of music playing in the users environment. The system is evaluated with respect to timing and is able to respond to beats within 16 ms on average.
In this paper, we propose a new representation as input of a Convolutional Neural Network with the goal of estimating music structure boundaries. For this task, previous works used a network performing the late-fusion of a Mel-scaled log-magnitude spectrogram and a self-similarity-lag-matrix. We propose here to use the squaresub-matrices centered on the main diagonals of several self-similarity-matrices, each one representing a different audio descriptors. We propose to combine them using the depth of the input layer. We show that this representation improves the results over the use of the self-similarity-lag-matrix. We also show that using the depth of the input layer provide a convenient way for early fusion of audio representations.
To improve the search functionality of online sound effect libraries, timbral information could be extracted using perceptual models, and added as metadata, allowing users to filter results by timbral characteristics. This paper identifies the timbral attributes that end-users commonly search for, to indicate the attributes that might usefully be modelled for automatic metadata generation. A literature review revealed 1187 descriptors that were subsequently reduced to a hierarchy of 145 timbral attributes. This hierarchy covered the timbral characteristics of source types and modifiers including musical instruments, speech, environmental sounds, and sound recording and reproduction systems. A part-manual, part-automated comparison between the hierarchy and a freesound.org search history indicated that the timbral attributes hardness, depth, and brightness occur in searches most frequently.
While dynamics is an important characteristic in music performance, it has been rarely researched in automatic music transcription. We propose a method to estimate individual note intensities from a piano recording given pre-aligned score data of the recording. To this end, we use non-negative matrix factorization in a score-informed framework, where the basis vectors and weights are constrained to estimate harmonic note spectra and corresponding intensities, respectively. We examine various choices in the learning process including the use of synthesized note scale for initialization, type of spectrum, and temporal constraint. We evaluate our method using Saarland Music Data (SMD) and estimate note intensities in MIDI velocity. The results show that the proposed method makes an improvement compared to previous work.
Harmonic-percussive separation is a technique that splits music recordings into harmonic and percussive components-it can be used as a preprocessing step to facilitate further tasks like key detection (harmonic component) or drum transcription (percussive component). We propose a cascaded harmonic-residual-percussive (HRP) procedure yielding a mid-level feature to analyze musical phenomena like percussive event density, timbral changes, and homogeneous structural segments. We first outline the steps to compute cascaded HRP features (CHRP) and then illustrate their capabilities by means of three examples: to visualize percussive and noise-like properties of snare-drum playing techniques, to examine changes between harmonic and percussive timbres in electronic music, and to identify homogeneous, purely percussive passages in funk and soul recordings (also known as breaks).
In this paper we describe an online system for broadcast content, which can detect sound scene changes with high accuracy. The system is unsupervised and does not require prior information on the segment classes. A scene change probability score is computed for each frame of the signal using a hybrid approach combining a model-based (Gaussian Mixture Model) with a distance-based (Hotelling's T-2-Statistic) segmentation method. The mixture model parameters are adapted online using the previous frames of the signal. Experiments on real recordings show that we can achie more than 85% correct segment change detection with only 16% false detections.
Key detection in electronic dance music is important for producers and DJ's who want to mix their tracks harmonically or organise their music collection by tonal content. In this paper, we present an algorithm that improves the performance of an existing method by introducing a system of multiple profiles, addressing difficult minor tracks as well as possibly amodal ones. After the explanation of our method, we use three independent datasets of electronic dance music to evaluate its performance, comparing it to other academic algorithms and commercially available solutions.
This work presents a spectrogram factorisation method applied to automatic music transcription of a cappella performances with multiple singers. A variable-Q transform representation of the audio spectrogram is factorised with the help of a 6-dimensional sparse dictionary which contains spectral templates of vowel vocalizations. A post-processing step is proposed to remove false positive pitch detections through a binary classifier, where overtone-based features are used as input into this step. Preliminary experiments have shown promising multi-pitch detection results when applied to audio recordings of Bach Chorales and Barbershop music. Comparisons made with alternative methods have shown that our approach increases the number of true positive pitch detections while the post-processing step keeps the number of false positives lower than those measured in comparative approaches.
In live multitrack recordings, each voice is usually captured by dedicated close microphones. Unfortunately, it is also captured in practice by other microphones intended for other sources, leading to so-called "interferences". Reducing this interference is desirable because it opens new perspectives for the engineering of live recordings. Hence, it has been the topic of recent research in audio processing. In this paper, we show how a Gaussian probabilistic framework may be set up for obtaining good isolation of the target sources. Doing so, we extend several state-of-the art methods by fixing some heuristic parts of their algorithms. As we show in a perceptual evaluation on real-world multitrack live recordings, the resulting principled techniques yield improved quality.
This paper presents a new method for Long Short-Term Memory Recurrent Neural Network (LSTM) based speech overlap detection. To this end, speech overlap data is created artificially by mixing large amounts of speech utterances. Our elaborate training strategies and presented network structures demonstrate performance surpassing the considered state-of-the-art overlap detectors. Thereby we target the full ternary task of non-speech, speech, and overlap detection. Furthermore, speakers' gender is recognised, as the first successful combination of this kind within one model.
In Automatic Music Transcription, onset information is useful for correcting timing issues of multi-pitch estimation processes and obtain note-level representations of audio signals. Although this idea has been often used in transcription systems, it is still unclear to which degree its use is beneficial. We address this question by studying the influence of the accuracy of onset information in piano music transcription. Results indicate that note tracking results improve when the onset information provided is accurately estimated and properly included with the correct strategy. Additionally, results depict an important accuracy gap in note tracking when considering ground-truth onset information compared to using an automatic onset estimation algorithm, showing the need for more accurate onset detection methods for music transcription systems.
A digital audio workstation (DAW) is a PC or Macintosh equipped with sound cards and software for editing and processing digital audio. A properly outfitted computer can now function like a sophisticated recording studio, using plug-ins and special software to create recording-studio quality sound on the desktop, and then allowing users to edit and process that sound to perfection. DAWs are not only revolutionizing the way music is recorded, but allow new forms of electronic music, created entirely in the computer, to be written and recorded on the desktop. The Digital Audio Workstation Studio, by renowned digital music expert Colby Lieder, will be the first book to teach the complete DAW process, from outfitting your computer with the necessary hardware and software, recording with the proper microphones, using DAW power to create perfectly modulated mixes, to finally mastering down into a completely professional finished product. With this book, a computer, and a little imagination, any musician can create utterly professional music CDs, DVDs, or streamed audio. Table of contents Chapter 1: Computers, Sound, and Digital Audio Chapter 2: The Development of the Modern DAW Chapter 3: Sound Files: Formats, Storage,and Transport Chapter 4: Editing and Mixing Chapter 5: Plug-Ins Chapter 6: A Closer Look at the DAW Studio Chapter 7: Surround Mixing and Monitoring Chapter 8: Recording and the DAW Chapter 9: Expanding the Palette Chapter 10:Mastering and Distribution
This paper explores the problem of automatically detecting electric guitar solos in rock music. A baseline study using standard spectral and temporal audio features in conjunction with an SVM classifier is carried out. To improve detection rates, custom features based on predominant pitch and structural segmentation of songs are designed and investigated. The evaluation of different feature combinations suggests that the combination of all features followed by a post-processing step results in the best accuracy. A macro-accuracy of 78.6% with a solo detection precision of 63.3% is observed for the best feature combination. This publication is accompanied by release of an annotated dataset of electric guitar solos to encourage future research in this area.
We propose a machine learning approach based on hinge-loss Markov random fields to solve the problem of applying reverb automatically to a multitrack session. With the objective of obtaining perceptually meaningful results, a set of Probabilistic Soft Logic (PSL) rules has been defined based on best practices recommended by experts. These rules have been weighted according to the level of confidence associated with the mentioned practices based on existent evidence. The resulting model has been used to extract parameters for a series of reverb units applied over the different tracks to obtain a reverberated mix of the session.
In this paper, we focus on transcribing walking bass lines, which provide clues for revealing the actual played chords in jazz recordings. Our transcription method is based on a deep neural network (DNN) that learns a mapping from a mixture spectrogram to a salience representation that emphasizes the bass line. Furthermore, using beat positions, we apply a late-fusion approach to obtain beat-wise pitch estimates of the bass line. First, our results show that this DNN-based transcription approach outperforms state-of-the-art transcription methods for the given task. Second, we found that an augmentation of the training set using pitch shifting improves the model performance. Finally, we present a semi-supervised learning approach where additional training data is generated from predictions on unlabeled datasets.
