Automatic speech recognition (ASR) systems typically require large volumes of high-quality transcribed speech. In low-resource settings, where accurate verbatim transcripts are scarce, weakly supervised (WS) sources provide a practical alternative. In this work, we use the audio, subtitle texts, and subtitle timestamps from 16 television programs broadcast by the Vlaamse Radio- en Televisieomroeporganisatie (VRT, Flemish Radio and Television Broadcasting Organisation) as WS resources for ASR training. We examine the value of subtitles by comparing them with pseudo transcripts generated by existing ASR models, which are commonly used in WS ASR. Since TV subtitles are often imperfectly aligned with speech, we first propose a subtitle refinement pipeline that uses Whisper-based pseudo transcriptions, forced alignment, and subtitle-specific processing to produce better-aligned training pairs. We then introduce a more flexible end-to-end alternative based on adversarial training, which models text predictions at the distribution level and reduces the reliance on precise text–speech alignment. Experiments with different ASR backbones show that training on refined subtitles substantially improves recognition performance over raw subtitles and direct pseudo-transcript supervision. The adversarial approach achieves competitive overall performance while offering a simpler training framework for more weakly aligned settings.
For automated speech recognition (ASR) systems, code-switched speech in which speakers alternate between two or more languages in a single utterance presents substantial difficulties, especially when it comes to Tamil-English languages. By creating a robust code-switched corpus and a parameter-efficient ASR system specifically suited for Tamil-English multilingual settings, this work seeks to address the difficulties. We employ the Whisper ASR model, which has been optimized by parameter-efficient adaptation methods like Quantized LoRA (QLoRA) and Low-Rank Adaptation (LoRA). These techniques allow huge pre-trained models to be scaled and resource-efficiently customized to the distinct linguistic and phonetic features of Tamil-English code-switched communication. Additionally, we suggest optimization techniques to raise the recognition accuracy of Tamil-English code-mixed combi-nations. With a Word Error Rate (WER) of 32.4
With the rapid advancement of computer technology, its application in basketball game analysis has become a prominent research focus. Sound events in basketball games encapsulate critical game-related information. However, research on sound event classification in basketball games remains in its early stages. This research systematically analyzes the sound events in basketball games, and constructs an open-source dataset of sound events in basketball games, which is characterized by larger scale and more diverse categories. Furthermore, a deep learning-based sound event classification method integrating convolutional neural network with the transformer architecture is proposed. The proposed method employs a time–frequency feature aggregation approach to effectively capture both local time–frequency characteristics and global dependencies within audio signals, enabling precise sound event classification. Experimental results demonstrate that the proposed classification method significantly outperforms traditional methods across multiple evaluation metrics, offering a novel and effective solution for sound event analysis in basketball games. The proposed sound events dataset in basketball games will be available at https://github.com/holhouse/Sound-Event-Dataset-in-Basketball-Game.
This study explores content-based clustering using features from the Free Music Archive (FMA) dataset. A diverse feature selection pipeline—including mRMR, SHAP, LightGBM importance, Pearson correlation, and Chi-square tests—was used to refine the dataset, resulting in a reduced set of key features. Multiple clustering methods, including K-Means, Birch, Self-Organizing Maps (SOM), Gaussian Mixture Models (GMM), and Fuzzy-CMeans were evaluated across k values from 7 to 40. To assess clustering quality, we applied internal validation metrics and class imbalance measures. Additionally, we introduce an unsupervised centroid-variation analysis to assess feature contributions, providing a direct measure of how audio descriptors differentiate cluster structure. Our findings emphasize the critical role of feature selection in shaping clustering outcomes. By combining internal validation and model interpretation tools that assess feature contributions, we provide evaluation of clustering quality and feature relevance. Our results show that simple clustering models such as K-Means often outperform more complex methods when applied to well-selected audio features, and segment-level features derived from Laplacian segmentation impacts the variance in the cluster structure.
Head-worn devices such as smartglasses and headsets are the predominant form factor for augmented reality and telepresence applications, where real-world environments are augmented with virtual sound sources. For these sources to appear perceptually convincing, the acoustics of their virtual environment must closely match the room acoustics of the physical space. Estimating room impulse responses (RIRs) in this setting is challenging because practical scenarios require small arrays that continuously move with the user’s head. This study presents a method for the blind identification of RIRs from speech signals captured with a moving head-worn microphone array and the subsequent rendering of virtual sound sources based on perceptually relevant acoustic parameter estimates. A motion-aware signal model that estimates spatial RIRs as sound field coefficients and incorporates position tracking data is compared against an omnidirectional model and a baseline. Numerical results show that the motion-aware model provides the most accurate acoustic parameter estimates when used in an informed setting with the true reference signal. In the blind setting, however, its advantage largely diminishes. In a listening experiment, renderings based on the omnidirectional model are rated as most similar to the reference condition and are most often associated with the correct room significantly above chance. The findings highlight the practical relevance of the proposed framework, with the omnidirectional model offering robust and perceptually convincing performance, while the motion-aware model remains promising for parameter estimation.
