For automated speech recognition (ASR) systems, code-switched speech in which speakers alternate between two or more languages in a single utterance presents substantial difficulties, especially when it comes to Tamil-English languages. By creating a robust code-switched corpus and a parameter-efficient ASR system specifically suited for Tamil-English multilingual settings, this work seeks to address the difficulties. We employ the Whisper ASR model, which has been optimized by parameter-efficient adaptation methods like Quantized LoRA (QLoRA) and Low-Rank Adaptation (LoRA). These techniques allow huge pre-trained models to be scaled and resource-efficiently customized to the distinct linguistic and phonetic features of Tamil-English code-switched communication. Additionally, we suggest optimization techniques to raise the recognition accuracy of Tamil-English code-mixed combi-nations. With a Word Error Rate (WER) of 32.4
With the rapid advancement of computer technology, its application in basketball game analysis has become a prominent research focus. Sound events in basketball games encapsulate critical game-related information. However, research on sound event classification in basketball games remains in its early stages. This research systematically analyzes the sound events in basketball games, and constructs an open-source dataset of sound events in basketball games, which is characterized by larger scale and more diverse categories. Furthermore, a deep learning-based sound event classification method integrating convolutional neural network with the transformer architecture is proposed. The proposed method employs a time–frequency feature aggregation approach to effectively capture both local time–frequency characteristics and global dependencies within audio signals, enabling precise sound event classification. Experimental results demonstrate that the proposed classification method significantly outperforms traditional methods across multiple evaluation metrics, offering a novel and effective solution for sound event analysis in basketball games. The proposed sound events dataset in basketball games will be available at https://github.com/holhouse/Sound-Event-Dataset-in-Basketball-Game.
This study explores content-based clustering using features from the Free Music Archive (FMA) dataset. A diverse feature selection pipeline—including mRMR, SHAP, LightGBM importance, Pearson correlation, and Chi-square tests—was used to refine the dataset, resulting in a reduced set of key features. Multiple clustering methods, including K-Means, Birch, Self-Organizing Maps (SOM), Gaussian Mixture Models (GMM), and Fuzzy-CMeans were evaluated across k values from 7 to 40. To assess clustering quality, we applied internal validation metrics and class imbalance measures. Additionally, we introduce an unsupervised centroid-variation analysis to assess feature contributions, providing a direct measure of how audio descriptors differentiate cluster structure. Our findings emphasize the critical role of feature selection in shaping clustering outcomes. By combining internal validation and model interpretation tools that assess feature contributions, we provide evaluation of clustering quality and feature relevance. Our results show that simple clustering models such as K-Means often outperform more complex methods when applied to well-selected audio features, and segment-level features derived from Laplacian segmentation impacts the variance in the cluster structure.
Head-worn devices such as smartglasses and headsets are the predominant form factor for augmented reality and telepresence applications, where real-world environments are augmented with virtual sound sources. For these sources to appear perceptually convincing, the acoustics of their virtual environment must closely match the room acoustics of the physical space. Estimating room impulse responses (RIRs) in this setting is challenging because practical scenarios require small arrays that continuously move with the user’s head. This study presents a method for the blind identification of RIRs from speech signals captured with a moving head-worn microphone array and the subsequent rendering of virtual sound sources based on perceptually relevant acoustic parameter estimates. A motion-aware signal model that estimates spatial RIRs as sound field coefficients and incorporates position tracking data is compared against an omnidirectional model and a baseline. Numerical results show that the motion-aware model provides the most accurate acoustic parameter estimates when used in an informed setting with the true reference signal. In the blind setting, however, its advantage largely diminishes. In a listening experiment, renderings based on the omnidirectional model are rated as most similar to the reference condition and are most often associated with the correct room significantly above chance. The findings highlight the practical relevance of the proposed framework, with the omnidirectional model offering robust and perceptually convincing performance, while the motion-aware model remains promising for parameter estimation.
Analysing spatial information within recorded music is increasingly important for audio research given the explosion of available datasets, but relevant techniques are spread over diverse fields. To address this, a systematic literature review (SLR) was conducted to ascertain all existing methods for describing the spatial features of stereophonic music. The Scopus, Web Of Science (WOS), and Semantic Scholar (SS) databases were searched on 26th February 2026, and articles were deemed eligible if the method they described included any extraction of spatial information from a music audio signal. Links and similarities were drawn between studies to identify the core methods of describing stereo image. One hundred seventy-eight articles were included, with 29 of those deemed “very relevant”. The fundamental methods of extracting spatial information are detailed and a framework for creating a custom stereo image profile (SIP) is drawn up, followed by suggestions for future research. The review was limited by the substantial volume of literature to be assessed by few researchers, although this wide breadth served to mitigate bias by gathering at least one example of all existing SIP methods. Inconsistent application of terminology within the existing literature presented a significant challenge. The work was funded in part by The Knowledge Foundation (KKS).
While automatic speech recognition (ASR) systems have achieved impressive performance under clean conditions, their reliability in acoustically challenging environments remains an open concern—particularly in multilingual settings. This work presents a comparative evaluation of several modern ASR models, including Whisper variants, QuartzNet, and Conformer-based architectures, under a range of controlled synthetic transformations and real-world environmental noises. Using the Common Voice 17.0 dataset in English, Italian, and German, we assess recognition robustness under additive white noise, pitch shifts, time-stretching, and ecologically valid background recordings (office, cafe, traffic) from the DEMAND dataset. Word error rate (WER) is computed across a spectrum of signal-to-noise ratios, with confidence intervals derived via bootstrap resampling to estimate variability. Unlike many studies that evaluate complete speech pipelines (enhancement front-ends followed by ASR or task-specific fine-tuning), we deliberately focus on off-the-shelf pretrained models without additional front-end processing or adaptation. This design isolates the intrinsic robustness of the ASR architectures themselves and provides a clean baseline against which future enhancement or fine-tuning strategies can be compared. To support the interpretation of extreme-noise regimes, we additionally incorporate a perceptually motivated glimpse proportion analysis, which quantifies the amount of locally audible speech under different noise types and signal-to-noise ratios. This auxiliary analysis is used to contextualize recognition failures in terms of acoustic masking rather than model performance alone. Finally, we include a limited supervised fine-tuning study on English speech for a subset of models, not as a primary contribution, but to illustrate how standard adaptation shifts robustness trends relative to the inference-only baseline. Our analysis highlights model- and language-specific response patterns to distortion, with larger models generally exhibiting greater robustness, yet still susceptible to temporal and spectral changes. Notably, models showed higher stability on Italian and German, which we hypothesize may be due to more regular phoneme-to-grapheme mappings in these languages. The findings provide actionable insights into failure modes under distortion, informing the design of more robust ASR systems for deployment in diverse auditory scenarios.
This paper introduces the Hi-Audio online platform, an open-source tool designed to support musicians and researchers in the field of Music Information Retrieval (MIR). The platform enables the recording, uploading, and sharing of multitrack musical compositions, aiming to build an open-access audio database to advance research in music technology. Uploaded audio files are automatically analyzed upon synchronization with the server, leveraging signal processing techniques and machine learning models to generate rich metadata. The platform facilitates remote and asynchronous collaboration via a web-based interface accessible at hiaudio.fr. Furthermore, a novel built-in method for accurate and robust round-trip latency estimation in the browser is proposed and integrated into the platform, demonstrating its applicability in real-world distributed recording scenarios. Finally, an initial user evaluation with musicians was conducted to assess usability and practical relevance under realistic usage conditions. The evaluation combined task-based performance analysis with standardized usability and workload measures. The results indicate high task completion rates for core recording functions and show that the platform can be used effectively by musicians with minimal prior training.
Music emotion recognition (MER) is a critical task in the field of music information retrieval. However, most MER research relies solely on single-scale music spectrograms and fails to consider the complementary effects of spectrograms at different scales. Meanwhile, fully extracting emotion-related information from spectrograms remains a major challenge in MER. In this paper, we propose a hybrid attention model based on multi-resolution spectrograms, named MSMHA. The MSMHA model takes multi-scale Mel-spectrograms as inputs, and each input is fed into a well-designed hybrid attention network. The designed attention network successively includes a low-level feature extraction module, a local feature extraction module based on window attention, a channel attention-based long skip connection module, a high-level feature extraction module, and a branch classifier. After being processed by the hybrid attention network, each branch can fully extract emotion-related semantic features from a spectrogram of the specific resolution and output an emotion-classification probability. Finally, a decision-level weighted fusion strategy is applied to the multi-branch outputs to generate the final classification results. The experimental results on the PMEmo dataset demonstrate that our model is both promising and effective, achieving classification accuracies of 90.9
Although the roles of the torso and head-above-torso orientation (HATO) in human spatial hearing have been thoroughly studied in psychoacoustic research, binaural sound source localization (SSL) models generally treat head and torso as a rigid unit, not reflecting natural, independent head-torso postures or their dynamics during rotation. This study is the first to systematically assess HATO in deep-learning-based binaural SSL for both static scenes and under dynamic rotations. Convolutional recurrent networks are trained with and without varying HATO as well as with and without explicit head-orientation input (scalar angle or quaternion) across three configurations: joint head-and-torso rotation with fixed HATO, torso rotation below a fixed head, and head rotation above a fixed torso. Performance is measured via a set of complementary localization metrics including the mean great circle distance, coordinate-isolated angular components and the quadrant error rate on simulated and measured data, with analyses with respect to signal-to-noise ratio, sound-incidence direction, and head-rotation velocity. In static scenarios, matched single-HATO training reduces mean great circle distance and quadrant error rate by about 24
Data augmentation is crucial for automatic speech recognition, especially in low-resource settings. This paper introduces a time-domain augmentation method, FadeOutIn, designed to better capture the dynamics of spontaneous speech and remain compatible with classical techniques. Experimental results show that combining FadeOutIn with SpecAugment yields a relative improvement of 4.24
Music Emotion Recognition (MER) is a computational field of affective computing and audio signal processing. Although previous attempts used traditional machine learning algorithms, e.g., Support Vector Machines and k-Nearest Neighbors, for emotion classification in music, these methods are often challenged by the intricate temporal and spectral nature of sound signals, thereby constraining classification performance. To address this, the work proposes a new model, the Lion-Optimized CNN-BiGRU for Emotion Recognition (LOCBER), leveraging the strengths of Convolutional Neural Networks and Bidirectional Gated Recurrent Units for effective feature extraction and temporal sequence modeling. For additional performance optimization, LOCBER is optimized using the Lion Swarm Optimization (LSO) algorithm, which adjusts model parameters to minimize loss and achieve better accuracy and convergence. This study aims to improve music emotion recognition by integrating a CNN–BiGRU model with Lion Swarm Optimization, achieving 97.7
At present, in the field of deep learning speech signal enhancement, encoder-decoder structures are introduced to suppress noise and restore speech. Models with good performance often have large parameters, which is very unfriendly to edge computing chips. In this study, we propose channel grouped iterative temporal frequency convolution convolutional recurrent network with only 15.8 K parameters, which can be easily deployed on headphones. In the encoder-decoder structure, an improved four-layer block iterative temporal frequency convolution module is used. Deep convolutional networks often have some redundancy, which can be effectively reduced by channel grouped processing. In order to make full use of all channel information, the method of channel shift iterative processing is applied, so that all channel information is processed after multi-layer time-frequency convolution module. In the time-frequency convolution module, sub-band feature extraction and multi-scale dilated convolution are used to enhance the frequency domain perception ability, and RNN network is introduced to enhance the time domain modeling ability. Experimental results on the VCTK and DEMAND dataset show that our model with extremely low parameter surpasses conventional methods reaches or even exceed multiple evaluation metrics. Specifically, it achieves a PESQ score of 2.70 using GRU and 2.75 using CFC with 8.22 dB SISNR, reflecting improved speech quality. The algorithm is deployed on the edge computing chip with only 0.1TOPS computing power used in headphones, which can process audio signals with 33 ms delay. Through the joint processing of left and right channels and adaptive training methods, better performance has been achieved.
Domain shifts, such as changes in language, noise types, or recording environments, can significantly degrade the performance of speech enhancement systems. Most current research focuses on domain adaptation methods. While existing research often assumes that domain shifts have already been identified, this study addresses the crucial challenge of detecting them automatically. We propose a novel domain-shift detection method that monitors prediction uncertainty in a speech quality assessment network. Our method includes both supervised and unsupervised approaches. The supervised method requires access to clean speech, whereas the unsupervised method does not. Experimental results across various domain-shift scenarios demonstrate that our method effectively identifies domain mismatches, enabling timely adaptation to improve performance in real-world speech enhancement systems. We use 100 s of noisy speech to detect actionable domain shifts, typically achieving an SI-SDR improvement of 3–6 dB after adaptation. To the best of our knowledge, this is the first method for domain-shift detection in speech enhancement, offering both supervised and unsupervised variants with broad applicability to real-world systems.
Acoustic event detection faces great challenges in recognizing event categories and locating event temporal boundaries due to the scarcity of high-quality labeled data and the limitation of label granularity. To conquer these constraints, a novel data augmentation method, LTSRR, and a two-stage method, PCL-AED, are proposed in this study. LTSRR is developed to generate training samples with rich spectral characteristics, which significantly expands the diversity of the training data. Subsequently, PCL-AED puts up a segment-level progressive event classification strategy for alleviating the impact of uneven sample distribution on audio tagging, and a temporal contrast learning mechanism is designed to enhance the sensitivity of the model to event boundaries for acoustic event detection. Experiments on DCASE2018 and DCASE2021 Task 4 datasets show that the LTSRR and PCL-AED proposed in this paper exhibit competitive performance, providing a new stroke to the semi-supervised acoustic event detection task.
Despite advances in cochlear implant (CI) technology, music enjoyment remains a significant challenge for most CI users. To facilitate music listening in CI-mediated hearing, music processing algorithms have been proposed, which reduce the perceived complexity of music pieces. As these algorithms rely on a multitude of signal processing parameters, finding suitable and possibly genre-specific settings can be a tedious process. Therefore, in this study, we propose and evaluate an interactive optimization scheme for parametric music processing algorithms. To minimize the burden on CI users, the process was divided into two phases: first, normal-hearing (NH) participants evaluated a wide range of random parameter combinations using a vocoder-based CI simulation. Then, CI users participated in shorter, focused optimization sessions to refine these parameters. For comparison, we also conducted parallel listening sessions with NH listeners using the vocoder simulation. We tested the approach using a parametric music remixing algorithm that had previously been validated with a hand-crafted parameter setting found through expert knowledge. Starting from completely randomized initial settings, the method was able to find optimized parameter values through interactive user feedback. Both groups, CI listeners and NH listeners using a CI simulation, significantly preferred the optimized remixes over unprocessed music, while for CI listeners, the preference for the optimized settings was on par with the hand-crafted setting. NH listeners using the CI simulation even significantly preferred their optimized remixes. The results of this study demonstrate the utility of the proposed optimization scheme and pave the way for time-efficient, interactive tuning of future music processing algorithms.
In speech and audio coding, quantization noise, also known as coding noise, is often modeled as additive white noise. This noise can cause noticeable distortions, particularly in frequency regions where human hearing is most sensitive. Noise shaping, a key technique in speech and audio coding, addresses this issue by redistributing the noise spectrum to less perceptible frequency regions, leveraging auditory masking effects to enhance perceived audio quality. This paper introduces a novel approach to perceptually shape the spectrum of coding noise in speech and audio codecs. The proposed approach is based on the well-known principle of pre- and post-processing with adaptive linear filters, but features a new method for estimating filter coefficients. Typically, these coefficients are estimated either from the current input to the pre-processor (forward adaptation) or from the past synthesized signal (backward adaptation). In contrast, the proposed approach estimates the pre-processing filter coefficients directly from the current input to the pre-processor, while the post-processing filter coefficients are estimated from the current input to the post-processor. This estimation approach eliminates the need for transmitting information and introduces no lag between the frame used for estimation and the frame being filtered. Because the adaptation is achieved without any increase in bitrate, the method is referred to as “zero-bit.” To function correctly, some constraints must be imposed on calculating the pre-processor filter coefficients. An implementation of this new approach, termed “constrained adaptation,” is described. Subjective evaluation results demonstrate that constrained adaptation performs at least as well as forward or backward adaptation in shaping coding noise, with no cost in bit rate or temporal lag.
Estimating acoustic context parameters is essential for characterizing acoustic environments, thereby enhancing immersive perception in spatial audio creation and improving speech enhancement and dereverberation algorithms. In this paper, we propose a unified deep learning based framework that estimates various acoustic contexts, including frequency-dependent reverberation time ( T_30 ), direct-to-reverberant ratio, clarity ( C_50 ), room geometry, and sound source orientation from first-order Ambisonics (FOA) speech recordings. Our framework employs a novel feature, termed the Spectro-Spatial Covariance Vector (SSCV), which efficiently represents the temporal, spectral, and spatial information of FOA signals. This feature can be effectively utilized by several deep neural networks as back-ends. Experimental results demonstrate that the proposed framework, which incorporates spatial information derived from FOA recordings, significantly outperforms existing methods based solely on spectral information from single-channel audio, achieving more than a 50
Recent advancements in deep learning have paved the way for novel approaches to the problem of Packet Loss Concealment (PLC) in networked music performance systems. However, deep neural networks may have large inference times and, therefore, violate the strict temporal requirements of PLC methods for such systems. A promising avenue in this space lies in the exploration of the loss function used to train the network. Indeed, loss functions have a direct impact on the latent representation learned by the model during the training process without any additional cost at inference time. In this paper, we present the Tilt Loss, a perceptual loss function, i.e., a loss function that allows the model trained with it to have performances that correlate with the human evaluation. The proposed method was able to outperform the current state-of-the-art in PLC methods according to human evaluation, albeit the model exhibited unsatisfactory performance with unpitched instruments. Furthermore, our study pinpoints the need for novel objective metrics specifically tailored for the PLC case.
We propose a hybrid real- and complex-valued neural network (HNN) architecture, designed to combine the computational efficiency of real-valued processing with the ability to effectively handle complex-valued data. We illustrate the limitations of using a real-valued neural network (RVNN) for inherently complex-valued problems by showing how it learns to perform complex-valued convolution; learning twice as many weights as necessary. To create the HNN, we use building blocks containing both real- and complex-valued paths, where information between domains is exchanged through domain conversion functions. We also introduce novel complex-valued activation functions, with better generalisation and parameterisation efficiency. HNN-specific architecture search techniques are described to navigate the larger solution space. Experiments with the AudioMNIST dataset demonstrate that the HNN reduces cross-entropy loss and consumes fewer parameters compared to an RVNN for all cases considered. Further experiments for audio denoising also show performance gains using HNNs with a reduced model complexity when compared to its real- or complex-valued counterparts. Such results highlight the potential for the use of partially complex-valued processing in neural networks and applications of HNNs in many signal processing domains.
