Multi-instrument automatic music transcription (MIAMT) requires learning representations from limited labelled data. We present a controlled comparison of two self-supervised paradigms: masked spectrogram reconstruction with a masked autoencoder (MAE) and predictive learning with a joint embedding predictive architecture (JEPA). We use identical Transformer encoders, evaluating both transcription performance and latent space properties: geometric organisation, retrieval quality and representational capacity. On transcription, the two paradigms are broadly comparable, with task-specific advantages: JEPA performs better for monophonic wind instruments, MAE for dense polyphonic ensembles. Both substantially exceed the prior SSL MIAMT baseline, and on URMP both also exceed a recent state-of-the-art supervised system. In latent space, JEPA achieves better geometric organisation while MAE preserves higher representational capacity. Furthermore, MAE converges substantially faster under supervision. Initialising JEPA from MAE-pretrained weights consistently improves transcription, demonstrating that the two objectives are complementary rather than competing. Three key findings emerge. First, no single latent metric reliably predicts transcription performance. Second, MAE provides a richer initialisation for predictive pretraining than random. Third, reconstruction and prediction objectives reveal opposite capacity trajectories under supervision, a behaviour that persists regardless of initialisation.

In this paper, we study the task of automatic music upmixing, wherein a system predicts spatial mixing parameters from a multi-stem recording. Different from existing methods that rely on task-specific music encoders, we approach this task via audio language model (ALM) post-training, leveraging rich representations from existing ALMs, which encode both music semantics and mixing knowledge. Specifically, we propose a post-training recipe that first employs rejection sampling SFT, followed by reinforcement learning (RL) with verifiable rewards (RLVR) via GRPO. We propose Sphere (Spatial Heuristic Rewards), a deterministic reward suite inspired by music mixing conventions, to guide our post-training. It consists of 6 perceptually-motivated sub-rewards and encourages the output mix to be centered, balanced and spacious. More broadly, our results suggest that expert domain knowledge can be encoded as verifiable rewards and distilled into language models, without task-specific architectures.

Automatic Music Transcription (AMT) models have achieved a high level of success in polyphonic transcription of various instruments. Velocity, typically a measure of note intensity, is less commonly predicted in these models due to the absence of velocity labels in available datasets and lack of a proper definition for instruments other than piano. We present a methodology and model for velocity prediction in Automatic Guitar Transcription (AGT) which uses virtual instruments to generate synthetic training data with velocity labels. We first pretrain a model on this synthetic data. These weights are then transferred to a different model and trained on real guitar audio, allowing the model to retain the working velocity prediction while also achieving high performance and generalisability from the real training data. The velocity prediction is shown to outperform a baseline model which does not use the pretrained velocity weights, when evaluated on synthetic data. In addition, using the pretrained velocity weights offers a small improvement in note transcription, though the magnitude of this improvement is limited and not always significant depending on the testing data. Overall the model achieves results comparable to the state of the art in guitar transcription, while also successfully predicting velocity.

Monophonic audio-to-score transcription remains a challenging task, especially for improvised jazz solos. Prior work commonly follows a modular pattern: combining audio-to-performance-MIDI transcription, state-of-the-art beat tracking, and a post-processing algorithm that assigns metrical durations to notes (note values). This approach is prone to error accumulation at every step. Motivated by how expert transcribers work, we propose an end-to-end approach that inverts the logical order of subtasks: first, we model rhythm perception by transcribing the rhythmic structure of the melody at the beat level; then, we estimate the pitch for each note within the predicted rhythmic structures. Technically, our model introduces a rhythm-aware transformer architecture with perceiver-style cross-attention that jointly aligns frame-level acoustic features with beat-synchronous rhythmic embeddings. The proposed solution achieves state-of-the-art results on a challenging corpus of alto saxophone solos by Charlie Parker (the "Omnibook").

Humans recognize a musical passage even when it is shifted in time or transposed in pitch, indicating a notion of equivariance in the representation space. Our analysis, however, shows that standard music transformers map such time-shifted or pitch-transposed inputs onto uncorrelated representations: these models become progressively less equivariant as they scale in size or train longer. This suggests that in standard music transformers, additional model capacity is allocated to memorizing absolute patterns rather than capturing shared musical structures. In this paper, we propose the Equivariant Music Transformer (EMT), which enforces equivariance through self-distillation by jointly optimizing a next-token-prediction and an auxiliary equivariance regularization loss. We find that the additional equivariance loss acts as a beneficial regularizer, simultaneously improving next-token prediction and producing equivariant latent representations. Through both objective and subjective evaluations, EMT demonstrates superior equivariance and generative capability compared to data augmentation, feature engineering, and state-of-the-art (SOTA) baselines. More broadly, our findings reveal that standard language modeling methods alone do not capture music's translational symmetries, and dedicated inductive biases are required to produce better music representations. The code, weights and demos are available online.

While music generation models have evolved to handle complex multimodal inputs mixing text, lyrics, and reference audio, evaluation mechanisms have lagged behind, remaining fragmented and narrowly focused. In this paper, we bridge this critical gap by establishing a comprehensive ecosystem for Compositional Music Instruction (CMI) reward modeling, where the generated music may be conditioned on text descriptions, lyrics, and/or audio prompts. We first introduce CMIPref-Pseudo, a large-scale preference dataset comprising 110k pseudo-labeled samples, and CMI-Pref, a high-quality, human-annotated corpus tailored for fine-grained alignment tasks. To unify the evaluation landscape, we propose CMIRewardBench, a unified benchmark that evaluates music reward models on heterogeneous samples across musicality, text–music alignment, and compositional instruction alignment. Leveraging these resources, we develop CMI reward models (CMI-RMs), a parameter-efficient reward model family capable of processing heterogeneous inputs. We evaluate their correlation with human judgments scores on Music Arena and CMI-Pref test set, as well as preference agreement on Music Arena and CMI-Pref. Additional analyses examine performance variation across factors such as annotators, annotation timing and confidence, music generation models, and audio length. Experiments demonstrate that CMI-RM not only correlates strongly with human judgments, but also enables effective inference-time scaling via topk filtering. Our work provides the necessary data, benchmarks, and models to advance aligned music generation.

This paper presents a comprehensive documentation of RenCon 2025, the revival of the expressive performance rendering competition which took place at ISMIR 2025 in Daejeon, Korea. The competition attracted 9 entries from international research groups, representing diverse approaches to expressive piano performance rendering. The two-phase assessment structure comprised a preliminary online evaluation and live real-time rendering at the conference. We analyze the competition format, participant demographics, system performance, and lessons learned for future iterations. The results demonstrate significant advances in expressive rendering capabilities while highlighting remaining challenges in achieving human-level musical expression.

We present MAJEPPA, a self-supervised framework to learn piano performance representations that span the full skill spectrum, from beginner practice sessions to virtuoso concert recordings. We curate the MAJEPPA dataset, comprising 4,000 annotated recordings across six expertise levels and six recording contexts. We adapt a single pre-trained MIDI autoregressive model with a joint objective: next-token prediction learns score-conditioned performance generation at various skill levels, while InfoNCE and supervised contrastive losses align abstract score and performance representations in a joint embedding space. The proposed model both generates and understands performances in a unified framework. By introducing the EVPMR benchmark, a suite of downstream tasks spanning quality assessment, competition ranking, mistake and technique classification, we evaluate the learnt representations, demonstrating progress towards a real-world model for the piano performance space.

Moonbeam is a transformer-based foundation model for symbolic music, pretrained on a large and diverse collection of MIDI data totaling 81.6K hours of music and 18 billion tokens. Moonbeam incorporates music-domain inductive biases by capturing both absolute and relative musical attributes through the introduction of a novel domain-knowledge-inspired tokenization method and Multidimensional Relative Attention (MRA), which captures relative music information without additional trainable parameters. Leveraging the pretrained Moonbeam, we propose 2 finetuning architectures with full anticipatory capabilities, targeting 2 categories of downstream tasks: symbolic music understanding and conditional music generation (including music infilling). Our model outperforms other large-scale pretrained music models in most cases in terms of accuracy and F1 score across 3 downstream music classification tasks on 4 datasets. Moreover, our finetuned conditional music generation model outperforms a strong transformer baseline with a REMI-like tokenizer. We open-source the code, pretrained model, and generated samples on Github.

Expressive music performance rendering involves interpreting symbolic scores with variations in timing, dynamics, articulation, and instrument-specific techniques, resulting in performances that capture musical can emotional intent. We introduce RenderBox, a unified framework for text-and-score controlled audio performance generation across multiple instruments, applying coarse-level controls through natural language descriptions and granular-level controls using music scores. Based on a diffusion transformer architecture and cross-attention joint conditioning, we propose a curriculum-based paradigm that trains from plain synthesis to expressive performance, gradually incorporating controllable factors such as speed, mistakes, and style diversity. RenderBox achieves high performance compared to baseline models across key metrics such as FAD and CLAP, and also tempo and pitch accuracy under different prompting tasks. Subjective evaluation further demonstrates that RenderBox is able to generate controllable expressive performances that sound natural and musically engaging, aligning well with prompts and intent.

This study introduces RUMAA, a transformer-based framework for music performance analysis that unifies score-to-performance alignment, score-informed transcription, and mistake detection in a near end-to-end manner. Unlike prior methods addressing these tasks separately, RUMAA integrates them using pre-trained score and audio encoders and a novel tri-stream decoder capturing task interdependencies through proxy tasks. It aligns human-readable MusicXML scores with repeat symbols to full-length performance audio, overcoming traditional MIDI-based methods that rely on manually unfolded score-MIDI data with pre-specified repeat structures. RUMAA matches state-of-the-art alignment methods on non-repeated scores and outperforms them on scores with repeats in a public piano music dataset, while also delivering promising transcription and mistake detection results.

Recent advances in text-to-music editing, which employ text queries to modify music (e.g. by changing its style or adjusting instrumental components), present unique challenges and opportunities for AI-assisted music creation. Previous approaches in this domain have been constrained by the necessity to train specific editing models from scratch, which is both resource-intensive and inefficient; other research uses large language models to predict edited music, resulting in imprecise audio reconstruction. To Combine the strengths and address these limitations, we introduce Instruct-MusicGen, a novel approach that finetunes a pretrained MusicGen model to efficiently follow editing instructions such as adding, removing, or separating stems. Our approach involves a modification of the original MusicGen architecture by incorporating a text fusion module and an audio fusion module, which allow the model to process instruction texts and audio inputs concurrently and yield the desired edited music. Remarkably, Instruct-MusicGen only introduces 8

Jazz improvisation has many similarities to spoken language, and it might be expected that large language models would be effective tools for information retrieval and generative applications applied to it. There are, however, important practical differences. The success of modeling natural language has, in part, been due to the availability of vast corpora of symbolic text. By comparison, collections of transcribed jazz are orders of magnitude smaller. For this reason, neural architectures are unlikely to be as effective for music as they have been for text without the support of additional information. For applications with limited data, various strategies have been shown to be helpful, one of which is the injection of domain knowledge. The objective of this paper is to analyze the relationship between melody and harmony as a method for extracting jazz-specific domain knowledge. To that end, we describe an automated system for identifying and tagging harmonic features of jazz melody, and apply it to a corpus of 325 transcribed, bebop-style solos with over 300,000 notes. A unique aspect of our work is that the tags are based on terminology used by jazz musicians, and this allows us to directly analyze the statistical characteristics of improvisational devices taught by educators and found in instructional books. Our analysis confirms the expressiveness of harmonic tagging, and identifies a convergence of vocabulary used across the thirteen musicians represented in our data. The results show that harmonic tags capture useful domain knowledge and should be beneficial in improving the effectiveness and accuracy of deep learning architectures applied to jazz applications.

This paper presents an unsupervised method for learning disentangled representations of monophonic music signals into three factors: global timbral, local pitch, and local variational features. While existing methods have achieved this for short isolated notes using random perturbation, they fail for sounds with pitch transitions or singing voices, causing leakage of the three characteristics into mismatched latent features. To address this, we introduce a new framework called re-entry training, which applies the network for three-factor disentanglement twice in series with shared weights. Re-entry training refines the characteristics extracted by the encoders and increases data variety, effectively performing implicit data augmentation. This serial model can be reinterpreted as a unified large variational autoenco der, offering an alternative probabilistic formulation for unsupervised training. Our experiments demonstrate that re-entry training results in a more focused extraction of sound characteristics, thereby enhancing the three-factor disentanglement for various monophonic music signals.

Automatic Drum Transcription (ADT) remains a challenging task in MIR but recent advances allow accurate transcription of drum kits with up 5 classes - kick, snare, hi-hats, toms and cymbals - via the ADTOF package. In addition, several drum kit stem separation models in the open source community support separation for more than 6 stem classes, including distinct crash and ride cymbals. In this work we explore the benefits of combining these tools to improve the realism of drum transcriptions. We describe a simple post-processing step which expands the transcription output from five to seven classes and furthermore, we are able to estimate MIDI velocity values based on the separated stems. Our solution achieves strong performance when assessed against a baseline of 8-class drum transcription and produces realistic MIDI transcriptions suitable for MIR or music production tasks.

We demonstrate that the Joint-Embedding Predictive Architecture is effective for learning representations suitable for Music Information Retrieval tasks. Specifically, we explore its application to multi-instrument automatic music transcription, focusing on multi-pitch estimation and instrument recognition. We evaluate the learned representations across multiple settings: (1) finetuning a pretrained JEPA model with transcription supervision, (2) end-to-end training with transcription supervision, (3) training an instrument-aware transcriber on frozen JEPA embeddings and (4) training an instrument-agnostic transcriber on frozen JEPA embeddings. To assess the structure of the learned representations, we compute Calinski-Harabasz clustering scores with respect to pitch index, pitch class, instrument, and octave. We find that the representations learned by JEPA and its modified version (2), primarily capture instrument identity and pitch height information, rather than pitch class distinctions. Despite this, our results demonstrate promising transcription performance and highlight the potential of non-generative self-supervised learning for multi-instrument music transcription. Code and model configurations are available on GitHub.1

Representation learning aims to find a minimum set of features that is sufficient to solve downstream tasks. This is particularly challenging in Music Information Retrieval (MIR), as different tasks may focus on different aspects of the audio input. Existing methods involving music generation, masked modelling and contrastive learning rely on pre-training large neural networks to build a single representation. While generally effective, computational costs for training are high and it is difficult to adapt the representation to different aspects of music. Instead, we investigate general-purpose representations using parameter-efficient models, aiming for a flexible model that can be trained on a single GPU within two days. This paper introduces Coord-VAE, a variational autoencoder (VAE) with a coordinate network as its decoder. By conditioning the likelihood on coordinate distributions, our representation requires 300 times less parameters than state-of-the-art models while surpassing a vanilla VAE baseline in most downstream tasks, demonstrating the potential of our approach for efficient representation learning.

Research in music understanding has extensively explored composition-level attributes such as key, genre, and instrumentation through advanced representations, leading to cross-modal applications using large language models. However, aspects of musical performance such as stylistic expression and technique remain underexplored, along with the potential of using large language models to enhance educational outcomes with customized feedback. To bridge this gap, we introduce LLaQo, a Large Language Query-based music coach that leverages audio language modeling to provide detailed and formative assessments of music performances. We also introduce instruction-tuned query-response datasets that cover a variety of performance dimensions from pitch accuracy to articulation, as well as contextual performance understanding (such as difficulty and performance techniques). Utilizing AudioMAE encoder and Vicuna-7b LLM backend, our model achieved state-of-the-art (SOTA) results in predicting teachers' performance ratings, as well as in identifying piece difficulty and playing techniques. Textual responses from LLaQo was moreover rated significantly higher compared to other baseline models in a user study using audio-text matching. Our proposed model can thus provide informative answers to open-ended questions related to musical performance from audio data.

Guitar tablatures enrich the structure of traditional music notation by assigning each note to a string and fret of a guitar in a particular tuning, indicating precisely where to play the note on the instrument. The problem of generating tablature from a symbolic music representation involves inferring this string and fret assignment per note across an entire composition or performance. On the guitar, multiple string-fret assignments are possible for most pitches, which leads to a large combinatorial space that prevents exhaustive search approaches. Most modern methods use constraint-based dynamic programming to minimize some cost function (e.g. hand position movement). In this work, we introduce a novel deep learning solution to symbolic guitar tablature estimation. We train an encoder-decoder Transformer model in a masked language modeling paradigm to assign notes to strings. The model is first pre-trained on DadaGP, a dataset of over 25K tablatures, and then fine-tuned on a curated set of professionally transcribed guitar performances. Given the subjective nature of assessing tablature quality, we conduct a user study amongst guitarists, wherein we ask participants to rate the playability of multiple versions of tablature for the same four-bar excerpt. The results indicate our system significantly outperforms competing algorithms.

Algorithms for automatic piano transcription have improved dramatically in recent years due to new datasets and modeling techniques. Recent developments have focused primarily on adapting new neural network architectures, such as the Transformer and Perceiver, in order to yield more accurate systems. In this work, we study transcription systems from the perspective of their training data. By measuring their performance on out-of-distribution annotated piano data, we show how these models can severely overfit to acoustic properties of the training data. We create a new set of audio for the MAESTRO dataset, captured automatically in a professional studio recording environment via Yamaha Disklavier playback. Using various data augmentation techniques when training with the original and re-performed versions of the MAESTRO dataset, we achieve state-of-the-art note-onset accuracy of 88.4 F1-score on the MAPS dataset, without seeing any of its training data. We subsequently analyze these data augmentation techniques in a series of ablation studies to better understand their influence on the resulting models.