For concertgoers, musical interpretation is the most important factor in determining whether or not we enjoy a classical performance. Every performance includes mistakes---intonation issues, a lost note, an unpleasant sound---but these are all easily forgotten (or unnoticed) when a performer engages her audience, imbuing a piece with novel emotional content beyond the vague instructions inscribed on the printed page. While music teachers use imagery or heuristic guidelines to motivate interpretive decisions, combining these vague instructions to create a convincing performance remains the domain of the performer, subject to the whims of the moment, technical fluency, and taste. In this research, we use data from the CHARM Mazurka Project---forty-six professional recordings of Chopin's Mazurka Op. 63 No. 3 by consumate artists---with the goal of elucidating musically interpretable performance decisions. Using information on the inter-onset intervals of the note attacks in the recordings, we apply functional data analysis techniques enriched with prior information gained from music theory to discover relevant features and perform hierarchical clustering. The resulting clusters suggest methods for informing music instruction, discovering listening preferences, and analyzing performances.

Background: Several screening tools are available for use in a clinical setting to predict injury. However, there is a lack of evidence regarding the accuracy of these tools to predict soccer-specific injuries. Objectives: The purpose of this systematic literature review was to determine the psychometric properties or accuracy of screening tools for common soccer injuries. Methods: A systematic review of diagnostic test accuracy was undertaken based on the Joanna Briggs Institute (JBI) procedure for conducting systematic reviews. Databases such as SPORT Discus, Cinahl, Medline, Science Direct, PubMed and grey literature were searched in order to access suitable studies. Results: A total of 10 studies were included for the analysis – three were analysed quantitatively whilst the remaining seven were analysed qualitatively. The screening tools were of high reliability, sensitivity and specificity (calculated as intraclass correlation coefficient [ICC] (0.68 95% confidence interval [CI]: 0.52–0.84 and 0.64 95% CI: 0.61–0.66, respectively). Conclusion: The screening tools assessed for the prediction of common soccer injuries that emerged from this systematic review include the Functional Movement Screening (FMS™), the Landing Error Scoring System (LESS), the Tuck Jump Assessment, the Soccer Injury Movement Screening (SIMS) and the conventional hamstrings to quadriceps ratio; all with good evidence of predicting common soccer injuries. These tools were of high sensitivity and specificity thus reliable for soccer screening. Clinical implications: The validity of these tools is acceptable and therefore the authors recommend that these tools be included in an injury prevention programme for soccer players.

We present a first step towards developing an interactive piano tutoring system that can observe a student playing the piano and give feedback about hand movements and musical accuracy. In particular, we have two primary aims: 1) to determine which notes on a piano are being played at any moment in time, 2) to identify which finger is pressing each note. We introduce a novel two-stream convolutional neural network that takes video and audio inputs together for detecting pressed notes and finger presses. We formulate our two problems in terms of multi-task learning and extend a state-of-the-art object detection model to incorporate both audio and visual features. In addition, we introduce a novel finger identification solution based on pressed piano note information. We experimentally confirm that our approach is able to detect pressed piano keys and the piano player's fingers with a high accuracy.

In a common music practice scenario a player works with a musical score, but may jump arbitrarily from one passage to another in order to drill on difficult technical challenges or pursue some other agenda requiring non-linear movement through the score. In this work we treat the associated score alignment problem in which we seek to align a known symbolic score to audio of the musician’s practice session, identifying all “do-overs” and jumps. The result of this effort facilitates a quantitative view of a practice session, allowing feedback on coverage, tempo, tuning, rhythm, and other aspects of practice. If computationally feasible we would prefer a globally optimal dynamic programming search strategy; however, we find such schemes only barely computationally feasible in the cases we investigate. Therefore, we develop a computationally efficient off-line algorithm suitable for practical application. We present examples analyzing unsupervised and unscripted practice sessions on clarinet, piano and viola, providing numerical evaluation of our score-alignment results on handlabeled ground-truth audio data, as well as more subjective and easy-to-interpret visualizations of the results.

A Python framework, “Pydactyl,” supporting the rapid development of piano fingering models, is described and demonstrated. Leveraging the popular music21 toolkit, this object-oriented framework standardizes problem decomposition, data formats, fingering segment combination, and evaluation methods, while supporting automated and manual phrase segmentation. Reference implementations of three published models are included.

We consider the task of mapping the performance of a musical excerpt on one instrument to another. Our focus is on excitation continuous instruments, where pitch, amplitude, spectrum, and time envelope are controlled continuously by the player. The synthesized instrument should follow the target instrument's expressive gestures as much as possible, while also following its natural characteristics. We develop an objective function that balances distance of the synthesis to the target and smoothness in the spectral domain. An experiment mapping violin to bassoon playing by concatenating together short excerpts of audio from a database of solo bassoon recordings serves as an illustration.

We present the problem of music renotation, in which the results of optical music recognition are rendered in image format, while changing various parameters of the notation, such as the size of the display rectangle or transposition. We cast the problem as one of quadratic programming. We construct parameterizations of each composite symbol expressing the degrees of freedom in its rendering, and relate all the symbols through a connected graph. Some of the edges in this graph become terms in the quadratic cost function expressing a desire for spacing similar to that in the original document. Some of the edges express hard linear constraints between symbols expressing relations, such as alignments, that must be preserved in the renotated version. The remaining edges represent linear inequality constraints, used to resolve overlapping symbols. The optimization is solved through generic techniques. We demonstrate renotation on several examples of piano music.

We present our ongoing work in optical music recognition in which we seek to transform printed music notation images into symbolic representations, suitable for playback, analysis, and rendering. While music notation contains a small core of symbols and primitives composed in a rule-bound way, there are a great many common exceptions to these rules, as well as a heavy tail of rarer symbols. Since our goal is to create symbolic representations with accuracy near that of published music scores, we doubt the feasibility of fully-automatic recognition, opting instead for a human-guided approach. We define a simple communication channel between the user and recognition engine, in which the user imposes pixel-level or model-level constraints.

We propose a human-in-the-loop scheme for optical music recognition. Starting from the results of our recognition engine, we pose the problem as one of constrained optimization, in which the human can specify various pixel labels, while our recognition engine seeks an optimal explanation subject to the humansupplied constraints. In this way we enable an interactive approach with a uniform communication channel from human to machine where both iterate their roles until the desired end is achieved. Pixel constraints may be added to various stages, including staff finding, system identification, and measure recognition. Results on a test show significant speed up when compared to purely human-driven correction. Introduction Optical Music Recognition (OMR) holds potential to transform score images into symbolic music libraries, thus enabling search, categorization, and retrieval by symbolic content, as we now take for granted with text. Such symbolic libraries would serve as the foundation for the emerging field of computational musicology, and provide data for a wide variety of fusions between music, computer science, and statistics. Equally exciting are applications such as the digital music stand, and systems that support practice and learning through objective analysis of rhythm and pitch. In spite of this promise, progress in OMR has been slow; even the best systems, both commercial and academic, leave much to be desired[7]. In many cases the effort needed to correct OMR output may be more than that of entering the music data from scratch[8]. In such cases OMR systems fail to make any meaningful contribution at all. The reason for these disappointing results is simply that OMR is hard. Bainbridge [17] discusses some the challenges of OMR that impede its development. One central problem is that music notation contains a large variety of somewhat-rare musical symbols and conventions [4], such as articulations, bowings, tremolos, fingerings, accents, harmonics, stops, repeat marks, 1st and 2nd endings, dal segno and da capo markings, trills, mordants, turns, breath marks, etc. While one can easily build recognizers that accommodate these somewhat-unusual symbols and special notational cases, the false positive detections that result often outweigh the additional correct detections they produce. Under some circumstances, some not-so-rare symbols fall into this better-not-to-recognize category, such as augmentation dots, double sharps, and partial beams. Another issue arises from the difficulty in describing the high-level structure of music notation. Objects such as chords, beamed groups, and clef-key-signatures, are highly structured and lend themselves naturally to grammatical representation, however, the overall organization of symbols within a measure is far less constrained. The OMR literature contains several efforts to formulate a unified grammar for music notation [10, 11]. These approaches represent grammars of primitive symbols (beams, flags, note heads, stems, etc.) and begin by assuming a collection of segmented primitives. While our grammars have significant overlap with these approaches, one of our primary uses for the grammar is the segmentation of the symbols into primitives — we do not believe it is realistic to identify the primitives without understanding the larger structures that contain them. Kopec [12] describes a compelling Markov Source Model for music recognition that simultaneously segments and recognizes. However, the approach addresses a small subset of music notation and does not generalize in any obvious way. In particular, our primary focus is on the International Music Score Library Project (IMSLP), while Kopec’s model covers a small minority of the examples encountered there. Other difficulties stem from the kinds of image degradation encountered, including poor or variable contrast, skew and warping of an image caused when the document is not aligned or flat in the scanner bed, hand-written marks, damage to pages, as well as other sources. Some recent research has been dedicated to the improvement of fully automated OMR systems in post-process fashion, or other ways that leave the core recognition engine intact. These efforts either create systems that adapt automatically [16, 24], add musically meaningful constraints for recognition [1, 5], or combine multiple recognizers to achieve better accuracy [9, 7]. However, OMR research is still a long way from our shared goal of creating large scale symbolic music databases. Hankinson et al. [15] created a prototype system for distributed large-scale OMR, which converts a collection of Gregorian chant scores into symbolic files to facilitate their in situ content-based retrieval, though the approach still requires a large amount of careful proofreading and correction. In light of these many obstacles and our collective past history, it seems unwise to bet on fully automated OMR systems that will produce high-quality results with any consistency. Instead we favor casting the problem as an interactive one, thus putting the human in the computational loop. In this case the essential challenge becomes one of minimizing the user’s effort, putting as much burden as possible on the computer, (but no more). There are many creative ways to integrate a person into the recognition pipeline, allowing her to correct, give hints, or direct the computation. This work constitutes an effort in this direction. Our first attempt to bring the human into OMR pipeline built a user interface allowing the correction of individual primitives: stem, beam, note head, single flag, sharp, augmentation dot, etc. Thus the user’s task was simply to cover the image ink by adding and deleting appropriate primitives. A benefit of this approach is that it presents the user with a clearly-defined task that doesn’t require knowledge of the system’s inner workings. There are, however, several weaknesses to this approach: the human tagging process is laborious; it fails to provide important syntactic relations between primitives; it requires the person to precisely register the primitive with the image; and it allows the person to create uninterpretable configurations of primitives (say a stem with no note head) creating havoc further down the OMR pipeline. Our aim here is to improve on all these weaknesses while still presenting a simple task to the user. Our current approach first presents the user with the original recognition results, obtained through fully automatic means. The user may then label any individual pixel according to the recognition task at hand. For instance, during system recognition the user may label a pixel as white space or bar line, while during measure recognition we use a richer collection of labels including, closed/half/whole note head, stem, ledger line, beam, sharp, single flag, etc. The system then re-recognizes subject to the user-imposed constraint. Since our recognizers embed highly restrictive assumptions on the primitives they assemble, a single correction often fixes a number of problems at once. Human and machine then iterate the process of providing and synthesizing human-supplied constraints into recognized results. This approach leaves the registration problem — the precise location of primitives — in the hands of the machine, where we believe it belongs. Furthermore, since our system can only recognize meaningful configurations of symbols, we avoid the problem of trying to assemble human-tagged composite symbols that may not make sense. While the resulting process may still be laborious, our results indicate that the human burden can be reduced considerably by employing this strategy. Furthermore, there are many other ways of introducing human-specified constraints into the recognition process, thus the current effort constitutes an initial exploration of a longer-term goal. Interactive OMR Various authors, such as Rebelo [13], suggest that interactive OMR system could be a realistic solution to the problem, though the central challenge of fusing the human and machine contributions still remains open. Human-in-the-loop computation has received considerable attention recently [23]. It has been applied to a wide variety of areas, such as retrieval systems [19], object classification [20], character recognition [18], document indexing [25], image labeling [22] and fined-grained visual categorization [21]. Romero [26] proposed a Hidden Markov Model (HMM) for computer-assisted text transcription, in which the user-imposed prefix is used to constrain both the sequence decoding and language priors. The potential of all these different applications is summarized in von Ahn’s statement [18]: “Human processing power can be harnessed to solve problems that computer cannot yet solve.” There have already been several OMR systems taking into account human-in-the-loop computation. For instance, Fujinaga [4] proposed an adaptive system that could incrementally improve its symbol classifiers based on human feedback. Church [6] implemented an interface accepting user feedback to guide misrecognized measures toward similar correct measures found elsewhere in the score. Our system uses human feedback in an entirely different manner — as a means of constraining the recognition process in a user-specified manner, thus leveraging the user’s input in the heart of the system. It is worth noting that our approach constitutes a generic framework that poses human-in-theloop recognition as constrained optimization, applicable beyond the specific confines of OMR. Human-Directed Recognition As motivation consider the example given in Figure 1. Suppose our recognition misses the upper note head of the chord (Figure 1b). Then suppose the user labels a single pixel that belongs to the missing note head as solid head (Figure 1c). When the system re-recognizes subject to this constraint, the note head, its associated ledger line, accidental and stem portion may all b

We propose a human-driven Optical Music Recognition (OMR) system that creates symbolic music data from common Western notation scores. Despite decades of development, OMR still remains largely unsolved as state-ofthe-art automatic systems are unable to give reliable and useful results on a wide range of documents. For this reason our system, Ceres, combines human input and machine recognition to efficiently generate high-quality symbolic data. We propose a scheme for human-in-the-loop recognition allowing the user to constrain the recognition in two ways. The human actions allow the user to impose either a pixel labeling or model constraint, while the system rerecognizes subject to these constraints. We present evaluation based on different users’ log data using both Ceres and Sibelius software to produce the same music documents. We conclude that our system shows promise for transcribing complicated music scores with high accuracy.

We describe the music renotation problem, in which one transforms a collection of recognized music notation primitives (e.g. note heads, stems, beams, flags, clefs, accidentals, etc.) into a different notation format, such as transposing the notation or displaying it in a rectangle or arbitrary size. We represent a limited degree of image understanding through a graph that connects pairs of symbols sharing layout constraints that must be respected during renotation. The layout problem is then formulated as the optimization of a convex objective function expressed as a sum of penalty terms, one for each edge in the graph. We demonstrate results by generating transposed parts from a recognized full score.

We present a method for recognizing and interpreting the text labels for the instruments in an orchestra score, thereby associating staves with instruments. This task is one of many necessary in optical music recognition. Our approach treats the score system as the basic unit of processing. A graph structure describes the possible orderings of instruments in the system. Each instrument may apply to several staves, may be represented with several possible text strings, and may appear at several possible positions relative to the staves. We find the optimal labeling of staves using a globally optimal dynamic programming approach that embeds simple template-based optical character recognition within the overall recognition scheme. When given an entire score, we simultaneously optimize on the text labeling for each system, as well as the character template models, thus adapting to the font at hand. Our implementation alternately optimizes over the text label identification and re-estimates the character templates. Experiments are presented on 10 different scores showing a significant improvement due to adaptation.

We present a system that interprets the notated rhythm obtained from optical music recognition (OMR). Our approach represents the notes and rests in a system measure as the vertices of a graph. We connect the graph by adding voice edges and coincidence edges between pairs of vertices, while the rhythmic interpretation follows simply from the connected graph. The graph identification problem is cast as an optimization where each potential edge is scored according to its plausibility. We seek the optimally scoring graph where the score is represented as a sum of edge scores. Experiments were performed on about 60 score pages showing that our system can handle difficult rhythmic situations including multiple voices, voices that merge and split, voices spanning two staves, and missing tuplets.

A system is presented for optical recognition of music scores. The system processes a document page in three main phases. First it performs a hierarchical decomposition of the page, identifying systems, staves and measures. The second phase, which forms the heart of the system, interprets each measure found in the previous phase as a collection of non-overlapping symbols including both primitive symbols (clefs, rests, etc.) with fixed templates, and composite symbols (chords, beamed groups, etc.) constructed through grammatical composition of primitives (note heads, ledger lines, beams, etc.). This phase proceeds by first building separate top-down recognizers for the symbols of interest. Then, it resolves the inevitable overlap between the recognized symbols by exploring the possible assignment of overlapping regions, seeking globally optimal and grammatically consistent explanations. The third phase interprets the recognized symbols in terms of pitch and rhythm, focusing on the main challenge of rhythm. We present results that compare our system to the leading commercial OMR system using MIDI ground truth for piano music.

The academic discipline focusing on the processing and organization of digital music information, commonly known as Music Information Retrieval (MIR), has multidisciplinary roots and interests. Thus, MIR technologies have the potential to have impact across disciplinary boundaries and to enhance the handling of music information in many different user communities. However, in practice, many MIR research agenda items appear to have a hard time leaving the lab in order to be widely adopted by their intended audiences. On one hand, this is because the MIR field still is relatively young, and technologies therefore need to mature. On the other hand, there may be deeper, more fundamental challenges with regard to the user audience. In this contribution, we discuss MIR technology adoption issues that were experienced with professional music stakeholders in audio mixing, performance, musicology and sales industry. Many of these stakeholders have mindsets and priorities that differ considerably from those of most MIR academics, influencing their reception of new MIR technology. We mention the major observed differences and their backgrounds, and argue that these are essential to be taken into account to allow for truly successful cross-disciplinary collaboration and technology adoption in MIR.

A model is presented for representing and generating piano performance. The model has far fewer parameters than the number of notes. This model explicitly addresses one of the fundamental characteristic of music performance that different areas in a performance have very different kinds of objectives or strategies that are employed. A graphical model is introduced to represent the evolution of the discrete strategies and tempo and dynamic progression. We design interactive procedures that allow users to modify the model intuitively. An algorithm is described to estimate parameters from partial performances that represent the skeleton of the music. Experiments are presented on the two-piano version of Rhapsody in Blue by George Gershwin.

A method is presented for the rhythmic parsing problem: Given a sequence of observed musical note onset times, we estimate the corresponding notated rhythm and tempo process. A graphical model is developed that represents the simultaneous evolution of tempo and rhythm and relates these hidden quantities to observations. The rhythm variables are discrete and the tempo and observation variables are continuous. We show how to compute the globally most likely configuration of the tempo and rhythm variables given an observation of note onset times. Preliminary experiments are presented on a small data set. A generalization to arbitrary conditional Gaussian distributions is outlined.

An approach of parsing piano music interpretation is presented. We focus mainly on quantifying expressive timing activities. A small number of different expressive timing behaviors (constant, slowing down, speeding up, accent) are defined in order to explain the tempo discretely. Given a MIDI performance of a piano music, we simultaneously estimate both discrete variables that corresponds to the behaviors and continuous variables that describe tempo. A graphical model is introduced to represent the evolution of the discrete behaviors and tempo progression. We demonstrate a computational method that acquires the approximate most likely configuration of the discrete behaviors and the hidden continuous variable tempo. This configuration represent a “smoothed” version of the performance which greatly reduces parametrization while retaining most of its musicality. Experiments are presented on several MIDI piano music performed on a digital piano. An user study is performed to evaluate our method.

We address the pruning or filtering problem, encountered in exact value iteration in POMDPs and elsewhere, in which a collection of linear functions is reduced to the minimal subset retaining the same maximal surface. We introduce the Skyline algorithm, which traces the graph corresponding to the maximal surface. The algorithm has both a complete and an iterative version, which we present, along with the classical Lark's algorithm, in terms of the basic dictionary-based simplex iteration from linear programming. We discuss computational complexity results, and present comparative experiments on both randomly-generated and well-known POMDP benchmarks.