Automatic Drum Transcription (ADT) is commonly formulated as a direct mapping from a music mixture to symbolic drum events. While effective for transcription, this formulation discards the acoustic stems that are useful for editing, remixing, and production. We revisit an alternative separate-and-detect formulation, where a drum source separation front end first produces five editable drum stems, and a fixed onset detector then converts each stem into symbolic events. The separator is built on a five-stem latent diffusion model that jointly generates kick, snare, toms, hi-hats, and cymbals in a compact VAE latent space. We further study two training-only auxiliary branches–an onset branch (OB) and a timbre branch (TB)–which shape the separator during learning but are discarded at inference. Trained on synthetic drum multitracks and evaluated on MDB Drums and ENST-Drums, the proposed pipeline consistently improves over a strong U-Net-based drum separation baseline in overall transcription F1. It also outperforms a representative end-to-end ADT system on kick and snare F1 under our evaluation protocol, while additionally providing separated audio stems. The ablation results show that OB gives the most stable transcription gains, whereas TB changes the trade-off between reconstruction, acoustic stem quality, and onset detection. These results suggest that generative drum demixing can serve not only as a source separation model, but also as a practical front end for interpretable drum transcription.

Taiwanese opera (Kua-á-hì), a major form of local theatrical tradition, underwent extensive television adaptation notably by pioneers like Iûnn Lē-hua. These videos, while potentially valuable for in-depth studies of Taiwanese opera, often have low quality and require substantial manual effort during data preparation. To streamline this process, we developed an interactive system for real-time OCR correction and a two-step approach integrating OCR-driven segmentation with Speech and Music Activity Detection (SMAD) to efficiently identify vocal segments from archival episodes with high precision. The resulting dataset, consisting of vocal segments and corresponding lyrics, can potentially supports various MIR tasks such as lyrics identification and tune retrieval. Code is available at https://github.com/z-huang/ocr-subtitle-editor .

The limited availability of large, high-quality annotated datasets in music information retrieval (MIR) remains a key challenge for advancing automatic music transcription (AMT). To address this limitation, we introduce a fully self-supervised framework for note-level AMT that integrates multi-pitch estimation (MPE) and onset detection without labeled data. Our method extends the reconstruction/transcription paradigm by incorporating spectral flux and local group delay features. Experiments show consistent gains over prior self-supervised baselines in MPE and promising results in note-level transcription. Although note tracking still trails supervised systems, analysis highlights the role of reconstruction losses in capturing instrument-sensitive pitch/onset information. This work advances self-supervised AMT methods that generalize across diverse musical domains with limited labeled resources.

While automatic music transcription is well-established in music information retrieval, most models are limited to transcribing pitch and timing information from audio, and thus omit crucial expressive and instrument-specific nuances. One example is playing technique on the violin, which affords its distinct palette of timbres for maximal emotional impact. Here, we propose VioPTT (Violin Playing Technique-aware Transcription), a lightweight, end-to-end model that directly transcribes violin playing technique in addition to pitch onset and offset. Furthermore, we release MOSA-VPT, a novel, high-quality synthetic violin playing technique dataset to circumvent the need for manually labeled annotations. Leveraging this dataset, our model demonstrated strong generalization to real-world note-level violin technique recordings in addition to achieving state-of-the-art transcription performance. To our knowledge, VioPTT is the first to jointly combine violin transcription and playing technique prediction within a unified framework.

Electronic synthesizer sounds are controlled by parameter settings that yield complex timbral characteristics and ADSR envelopes, making synthesizer-style audio transfer particularly challenging. Recent approaches to timbre transfer often rely on spectral objectives or implicit style matching, offering limited control over envelope shaping. Moreover, public synthesizer datasets rarely provide diverse coverage of timbres and ADSR envelopes. To address these gaps, we present SynthCloner, a factorized codec model that disentangles audio into three attributes: ADSR envelope, timbre, and content. This separation enables expressive audio transfer with independent control over these attributes. Additionally, we introduce SynthCAT, a new synthesizer dataset with a task-specific rendering pipeline covering 250 timbres, 120 ADSR envelopes, and 100 MIDI sequences. Experiments show that SynthCloner outperforms baselines on both objective and subjective metrics, while enabling independent attribute control. The code, model checkpoint, and audio examples are available at https://buffett0323.github.io/synthcloner/.

Motion capture (MOCAP)-free music for performance generation using deep generative models is emerging as a promising solution for the next-generation animation industry. This technology allows users to create dynamic musical performance animations without relying on MOCAP. However, implementing MOCAP-free content-to-performance generation systems presents significant challenges. First, integrating various standalone models into a cohesive system is essential, as each model governs distinct aspects of the avatar's behavior. For example, a facial expression generation module influences the avatar's facial expressions, whereas a fingering generation module determines hand positions. Second, most applications focus on humanonly performance generation, such as virtual vocalists and virtual dancers, without considering interactions with other objects, such as instruments, referred to as human-instrument performance generation. To our knowledge, comprehensive human-instrument content-to-performance generation systems are still rare. In this paper, we present a complete content-to-performance generation system, demonstrating its capabilities through a web application that enables users to create animations from their music content. This system incorporates four modules to generate parameters for controlling avatars: 1) a facial expression module, 2) a fingering generation module, 3) a body movement generation module, and 4) a video shot generation module. Additionally, we integrate an expressive music synthesis module to generate expressive audio from symbolic music data. Quantitative evaluations confirm the effectiveness of the four modules, whereas a user study provides qualitative insights into the system's performance. The web application is available on our website (https://virtual-musician.iis.sinica.edu.tw:8800).

Understanding the hierarchical structure of music remains a longstanding challenge in music information retrieval. Unlike typical AI tasks, hierarchical music structure analysis is inherently a context-dependent hierarchical clustering problem, requiring the identification of structures from individual motif occurrences to phrases and larger sections. In this paper, we propose a data-driven framework that adapts a recently proposed pretrained model to jointly address motif discovery and music segment labeling in the symbolic domain, paving the way toward hierarchical music structure analysis. Our method improves motif discovery by learning motif-level embeddings through sequential modules, including motif note identification, motif boundary detection, motif embedding, and motif clustering. These learned modules are then reused for music segment similarity measurement. Results demonstrate that our approach outperforms baseline methods and effectively identifies exact and inexact motif repetitions across musical segments. Also, the proposed motif-level embeddings outperform the note-level embeddings in improving music structure analysis. This study suggests a promising direction for data-efficient symbolic music understanding using pretrained models.

A fundamental challenge in neuroscience is to decode mental states from brain activity. While functional magnetic resonance imaging (fMRI) offers a non-invasive approach to capture brain-wide neural dynamics with high spatial precision, decoding from fMRI data – particularly from task-evoked activity – remains challenging due to its high dimensionality, low signal-to-noise ratio, and limited within-subject data. Here, we leverage recent advances in computer vision and propose STDA-SwiFT, a transformer-based model that learns transferable representations from large-scale fMRI datasets via spatial-temporal divided attention and self-supervised contrastive learning. Using pretrained voxel-wise representations from 995 subjects in the Human Connectome Project (HCP), we show that our model substantially improves downstream decoding performance of task-evoked activity across multiple sensory and cognitive domains, even with minimal data preprocessing. We demonstrate performance gains from larger receptor fields afforded by our memory-efficient attention mechanism, as well as the impact of functional relevance in pretraining data when fine-tuning on small samples. Our work showcases transfer learning as a viable approach to harness large-scale datasets to overcome challenges in decoding brain activity from fMRI data.

Music Information Retrieval (MIR) encompasses a broad range of computational techniques for analyzing and understanding musical content, with recent deep learning advances driving substantial improvements. Building upon these advances, this paper explores how large language models (LLMs) can serve as an integrative bridge to connect and integrate information from multiple MIR tools, with a focus on enhancing automatic chord recognition performance. We present a novel approach that positions text-based LLMs as intelligent coordinators that process and integrate outputs from diverse state-of-the-art MIR tools-including music source separation, key detection, chord recognition, and beat tracking. Our method converts audio-derived musical information into textual representations, enabling LLMs to perform reasoning and correction specifically for chord recognition tasks. We design a 5-stage chain-of-thought framework that allows GPT-4o to systematically analyze, compare, and refine chord recognition results by leveraging music-theoretical knowledge to integrate information across different MIR components. Experimental evaluation on three datasets demonstrates consistent improvements across multiple evaluation metrics, with overall accuracy gains of 1-2.77

Music boundary detection in music structure analysis has traditionally relied on objective metrics that may not align with human perception. This paper introduces a user study driven evaluation pipeline combining three stage evaluation: objective performance, subjective listener ratings and meta evaluation of metric validity. Audio is first converted into normalized log-mel spectrograms augmented with SpecAugment. Lightweight deep neural network (DNN) algorithms: convolutional neural network (CNN), bidirectional long short-term memory (BiLSTM), and convolutional recurrent neural network (CRNN) are trained to produce boundary-confidence curves, from which discrete timestamps are extracted by empirically tuned thresholds. Objective evaluation is measured by precision, recall, $\mathbf{F 1}$-score and Information Gain. Subjective evaluation is computed by mean opinion scores (MOS) collected from ten music experts using our web-based graphical user interface. Meta evaluation uses Pearson correlation to reveal which metric best detects listener judgments. On the SALAMI dataset, CNN with SpecAugment achieves precision of $\mathbf{0. 5 0}$ and $\mathbf{F 1}$-score of $\mathbf{0. 4 2}$ at a $\mathbf{0. 5}$-second tolerance, and Information Gain shows the strongest listener alignment with a correlation of $\mathbf{0. 6 0}$.

This study investigates the use of music language models (LMs) in singing MIDI transcription, the task of estimating the pitch, onset time, and offset time of each note in the vocal part from a musical audio signal. While recent studies have investigated acoustic models that predict pitch frame by frame using deep neural networks (DNNs), transcription errors remain due to large pitch fluctuations and ambiguous note boundaries in singing. To address this issue, we formulate Markov- and DNN-based LMs that estimate pitch probabilities at the note level, and integrate them with a DNN-based acoustic model using two methods: generative modeling and the sequential transducer. Experimental results show that both integration methods significantly improve transcription accuracy over a baseline acoustic model. Moreover, different strengths and characteristics of the compared LMs and integration methods are discussed.

We propose a pre-trained BERT-like model for symbolic music understanding that achieves competitive performance across a wide range of downstream tasks. To achieve this target, we design two novel pre-training objectives, namely token correction and pianoroll prediction. First, we sample a portion of note tokens and corrupt them with a limited amount of noise, and then train the model to denoise the corrupted tokens; second, we also train the model to predict bar-level and local pianoroll-derived representations from the corrupted note tokens. We argue that these objectives guide the model to better learn specific musical knowledge such as pitch intervals. For evaluation, we propose a benchmark that incorporates 12 downstream tasks ranging from chord estimation to symbolic genre classification. Results confirm the effectiveness of the proposed pre-training objectives on downstream tasks.

In the performance of Western art music, musicians apply various strategies to manipulate the sound performed and communicate their musical interpretations through these subtle acoustic variations. It is a common practice for musicians to use typical conventions to express each compositional style (e.g., Baroque, Classical, or Romantic compositions). However, such stylistic expressive conventions have yet to be fully discussed in previous research. In this initial foray, we systematically compare the expressive strategies for different piano compositions. A series of piano performances are recorded in a controlled experimental setting (3 compositions × 8 pianists × 3 repeated trials = 72 recordings), and expressive acoustic elements are derived using Music Information Retrieval techniques. In our analysis, we reveal that expressive manners in music performance exhibit stable and systematic features corresponding to each music composition, and those stylistic trends serve as empirical observations for typical performance conventions in different music styles.

Motif discovery in polyphonic symbolic music data is an important yet challenging task in music processing. In this paper, we propose a novel motif-discovery method created by combining the traditional rule-based repeated pattern discovery algorithms with a machine learning–based model that performs the task of motif note identification, i.e., identifying whether or not a note belongs to a motif. More specifically, the motif note identification model extracts motif notes for subsequent repeated pattern discovery. Removing non-motif notes can reduce the unwanted outputs in repeated pattern discovery and thereby improve performance. With a limited amount of training data, motif note identification can be implemented by fine-tuning a pre-trained model for symbolic music using pseudo-labels. The results demonstrate the feasibility of applying data-driven methods to assist the motif-discovery task, specifically on the occurrence and three-layer metrics, under the situation that labeled training data of the motif and repeated pattern are scarce.

Automatic music transcription (AMT) has achieved remarkable progress for instruments such as the piano, largely due to the availability of large-scale, high-quality datasets. In contrast, violin AMT remains underexplored due to limited annotated data. A common approach is to fine-tune pretrained models for other downstream tasks, but the effectiveness of such transfer remains unclear in the presence of timbral and articulatory differences. In this work, we investigate whether training from scratch on a medium-scale violin dataset can match the performance of fine-tuned piano-pretrained models. We adopt a piano transcription architecture without modification and train it on the MOSA dataset, which contains about 30 hours of aligned violin recordings. Our experiments on URMP and Bach10 show that models trained from scratch achieved competitive or even superior performance compared to fine-tuned counterparts. These findings suggest that strong violin AMT is possible without relying on pretrained piano representations, highlighting the importance of instrument-specific data collection and augmentation strategies.

Orchestral texture, which reflects the interaction of instruments in ensemble music, is central to understanding multitrack music; however, symbolic texture classification has not been widely studied. This study addresses the task of assigning textural roles to individual track-bar units (i.e., each bar of each track) in symphonic movements. We evaluate a range of features and models, including random forest, convolutional neural networks, hybrid approaches, and pre-trained models, on the Orchestration and S3 datasets. The results show that duration-related statistical features are the most informative, while identifying melodic roles remains challenging. Overall, our findings suggest the potential of symbolic texture classification, and highlight key challenges in melody extraction for multi-track music and ensuring labeling consistency across datasets.

Recent advances in AI-based music generation have focused heavily on text-conditioned models, with less attention given to reference-based generation such as song adaptation. To support this line of research, we introduce LargeSHS, a large-scale dataset derived from SecondHandSongs, containing over 1.7 million metadata entries and approximately 900k publicly accessible audio links. Unlike existing datasets, LargeSHS includes structured adaptation relationships between musical works, enabling the construction of adaptation trees and performance clusters that represent cover song families. We provide comprehensive statistics and comparisons with existing datasets, highlighting the unique scale and richness of LargeSHS. This dataset paves the way for new research in cover song generation, reference-based music generation, and adaptation-aware MIR tasks.

The paper discusses the classification of four music styles, Serialism, Impressionism, Neoclassicism, and Nationalism, of early-twentieth-century music using interpretable rule learning techniques. Three interpretable rule learning techniques are considered: decision tree, minimum description length (MDL) rule list, and rule set (the skope-rule algorithm). The features of the classifiers are fundamental musical elements based on pitch and interval distributions. Objective evaluation based on the F1 score and subjective evaluation using user study is conducted to understand the result of our classifiers from the musicians' point of view. The results show that a rule set is preferred as the algorithm attained the highest scores for objective and subjective evaluations. The rule set can also generate rules which support music theory and provide new insights regarding the musical characteristics of early twentieth-century music.

Expressive music synthesis (EMS) for violin performance is a challenging task due to the disagreement among music performers in the interpretation of expressive musical terms (EMTs), scarcity of labeled recordings, and limited generalization ability of the synthesis model. These challenges create trade-offs between model effectiveness, diversity of generated results, and controllability of the synthesis system, making it essential to conduct a comparative study on EMS model design. This paper explores two violin EMS approaches. The end-to-end approach is a modification of a state-of-the-art text-to-speech generator. The parameter-controlled approach is based on a simple parameter sampling process that can render note lengths and other parameters compatible with MIDI-DDSP. We study these two approaches (in total, three model variants) through objective and subjective experiments and discuss several key issues of EMS based on the results.

The selection of violin fingerings is influenced by factors such as musical context, skill level, and personal taste. Current deep-learningbased models, relying solely on symbolic data, are able to generate playable fingerings but struggle to capture the personal nuances of musical performance, which only lie in the audio data. To address this limitation, we introduce a novel model that incorporates both audio and symbolic data, allowing users to upload music scores and their corresponding violinist recordings to obtain personalized fingerings related to the audio data. To simulate such a real-world application scenario, we also collect a new dataset from online audios. The experiment results demonstrate the superiority of our proposed method over previous symbolic-based methods, even in the situations involving multiple instruments in audio.