The growing use of synthetic speech highlights the need to understand its differences from natural speech. Synthetic speech provides potential advantages in data augmentation, including privacy, security, and ethical data sourcing. However, the naturalness of synthetic speech is both a limitation and a risk, with deepfakes facilitating misinformation and fraud. These challenges underscore the need for improved methods to evaluate synthetic speech quality and deepfake detection. This research investigates whether latent representations from self-supervised models can identify and quantify differences between natural and synthetic speech regarding phoneme type, stress, manner, and roundedness. Our objective is to determine if models learn all phoneme categories equally or if certain groups present greater challenges, revealing limitations in synthetic speech. We pre-trained two wav2vec 2.0 models using matched natural and synthetic speech datasets. We mapped the learned codeword dictionaries to labeled test data with phoneme-level annotations and analysed the distribution and diversity of these latent representations across different phonemic categories. Our findings indicate a general lack of phonetic diversity in synthetic speech, with stress and manner showing the largest disparities. Vowels and diphthongs consistently exhibit reduced diversity. Identifying and quantifying differences in latent representations can be applied to enable enhanced synthetic speech generation, improve classification accuracy, and help to develop robust quality measurement metrics for synthetic speech.
Many recent Text-to-Speech (TTS) models employing zero-shot voice cloning techniques are capable of reproducing the emotional tone present in the reference speech. However, they frequently lack mechanisms for fine-grained emotion control due to the scarcity of labeled resources. The current work proposes a method for adjusting F5-TTS [1] in order to achieve fine-grained emotion control by fine-tuning under low-resource conditions, while aiming to keep the original model’s performance. To enforce the newly added condition, the classifier-free guidance (CFG) mechanism is modified by adding a supplementary term that amplifies the condition’s contribution, at the expense of increased inference time. We show that the proposed method is able to generate natural and intelligible speech with the same voice as the reference audio, but with changed emotion according to the label condition. Moreover, this method enables control over emotion intensity, despite the model not being explicitly trained for this task. Extensive automated evaluations and subjective tests with 20 human participants demonstrate that our method maintains intelligibility, naturalness and allows both emotion class and intensity control.Implementation code and demo samples are available at https://github.com/RaduBolbo/F5-TTS-Emotional-CFG
The rapid increase of computer-generated (CG) fake reviews on digital platforms poses a significant threat to consumer trust and market integrity. This research explores the effectiveness of established machine learning (ML) models in distinguishing CG reviews from original (OR), human-authored reviews, by using a public dataset of 40,432 labeled reviews. Comprehensive text preprocessing, including tokenization, stemming, and stop-word removal, is performed to standardize the data. Exploratory text analyses, namely word cloud visualization and sentiment scoring, reveal distinct lexical and emotional patterns between CG and OR reviews, with CG content displaying more constrained vocabulary and sentiment range. Multiple ML algorithms, including decision trees (DT), random forests (RF), and artificial neural networks (ANN), are systematically implemented and rigorously compared, with a primary focus on maximizing the recall metric for the CG class, in order to minimize the risk of undetected CG reviews. Results indicate that ANN models employing rectified linear unit (ReLU) activation achieve the highest recall, substantially reducing false negatives (FN) and enhancing the reliability of CG reviews detection. Feature analysis further highlights the importance of both lexical and sentiment-based attributes in differentiating review types. These findings underscore the practical effectiveness and computational efficiency of tailored ML approaches in safeguarding online review authenticity, supporting real-time deployment with low operational cost.
BLEU, ROUGE, and METEOR are widely used metrics for evaluating code summaries generated by large language models (LLMs), but their correlation with human judgment remains uncertain. In this study, we analyze whether a combination of these metrics can better approximate human evaluation scores. Using multilayer perceptrons (MLPs), Gaussian Process Regression (GPR) and Kolmogorov–Arnold Networks (KANs), we model the relationship between aggregated metric scores and human ratings. While individual metrics show weak alignment with human preferences particularly regarding semantic adequacy and contextual relevance, our results suggest that learned combinations offer modest improvements. Importantly, the interpretability of KANs provides insight into how these metrics interact. Rather than proposing a definitive new metric, this work offers a step toward understanding the limitations of current metrics and motivates the development of more human aligned evaluation methods for code summarization.
Echo cancellation is one of the most popular applications of adaptive filtering algorithms. In this framework, the algorithms have to be equipped with fast convergence/tracking features while should also be robust to different background perturbations. In terms of the convergence criteria, the decomposition-based recursive least-squares (RLS) algorithm represents a very appealing choice. It exploits an impulse response decomposition that relies on low-rank approximations and combines the estimates provided by two shorter adaptive filters using the nearest Kronecker product (NKP). In this paper, we develop a regularized version of the RLS-NKP algorithm with improved robustness features. The regularization components incorporate specific terms related to the background perturbations and model uncertainties, which are evaluated in a simple yet practical manner. Simulation results obtained in the context of network and acoustic echo cancellation support the performance gain.
Do pretrained speech models genuinely understand speech patterns, or do they simply learn to classify? We investigate this fundamental question by analyzing 10 state-of-the-art speech models across diverse tasks. We tested models on six speech characteristics (gender, accent, age, emotion, words, speaker identity) using both classification and clustering approaches. While models achieve impressive classification accuracy (up to $\mathbf{1 0 0 \%}$ for gender, $\mathbf{9 4 . 9 \%}$ for words), they show poor clustering performance when attempting to discover identical patterns without supervision. Most critically, we find systematic negative correlations between these capabilities—models better at accent classification are actually worse at discovering accent patterns through clustering ($\mathrm{r}=-0.904$). Our analysis suggests that current speech models learn equilateral decision boundaries rather than orthogonal pattern representations, due to training objectives promoting uniform dimensional alignment, lacking orthogonality in multiple dimensions. These findings expose a critical evaluation crisis in self-supervised learning: current benchmarks may systematically overestimate model capabilities, as high classification scores do not guarantee the pattern discovery capabilities that form SSL’s core value proposition.
This paper introduces an expanded and refined version of RoMEMEs, a multimodal corpus of Romanian memes collected from social media. Recognizing the limitations of the initial release, which contained 462 manually classified memes, this work details the methodology employed for a significant expansion of the corpus and briefly addresses refinements to the annotation guidelines to account for special cases identified in prior annotations. The expanded RoMEMEs corpus aims to enhance the resources available for training and evaluating advanced natural language processing and multimodal analysis models tailored for the specific characteristics of Romanian language memes. This paper outlines the data collection and annotation processes, describes the key features of the resulting dataset, and discusses the challenges encountered during its creation. The significantly enlarged and curated corpus, along with the updated annotation guidelines, is made publicly available to the research community to facilitate further investigation into the linguistic and cultural nuances of Romanian internet memes.
The paper presents a cognitive perspective on information structure (IS) of utterances. The cognitive approach of IS is based on the hypothesis that the neural activity which structures the auditory objects before they are converted into speech constituents modulates the F0 frequency of the related speech output. Section II presents the set of functional categories of the cognitive IS model, including the category of nuclear elements and the rules for their identification within F0 contours.In section III, the F0 contours of two English utterances are presented by showing how their cognitive/prosodic structure supports the syntactic and semantic functional elements. Two other utterances of the same sentences were synthesized by using Google Translate and their F0 contours are also presented in the paper by assigning them to two possible cognitive descriptions. The aim of the paper is to conclude in what extent the synthesized intonations without cognitive modelling support the syntactic and semantic functions of constituents, and to show how cognitive descriptions can drive speech synthesis to adapt the intonation to the message of input sentences.
In this work, we present the first version of a novel Hungarian–English code-switching dataset derived from VoxPopuli. The dataset includes two subsets – a randomly aligned set and a partially synthesized, context-preserving set – each specifically designed to support code-switching research. We evaluated three different ASR models on this dataset. Zero-shot results indicated that among the examined models, Canary 180M Flash was inherently the most capable of handling codeswitched speech. However, the best fine-tuned performance was achieved with the Whisper-large-v3-turbo model, attaining a multilingual WER of $8.07 \%$ and a CER of $3.74 \%$ on the test set. This represents a relative improvement of $89.84 \%$ in WER and $94.37 \%$ in CER compared to the zero-shot results. With the new dataset, we were able to improve the monolingual WER results for both languages, as well.
In the field of speech signal processing, both the duration and quality of audio recordings significantly influence system performance and response time. This work presents a comprehensive evaluation of a speaker recognition system developed using the SpeechBrain toolkit and deployed within a containerized application.Six experimental scenarios were designed to reflect real-world usage conditions and explore various dimensions of system performance. The evaluation includes a comparison of two voice activity detection (VAD) models — SpeechBrain and Silero — in multi-speaker noisy environments using the VoxConverse dataset; a speaker re-identification test measuring Equal Error Rate (EER) across multiple segmentation configurations; and a set of scalability tests analyzing the impact of CPU threading, number of enrolled users (up to 15,000), and test audio duration (up to 24 hours) on processing time. Additionally, system performance was measured under full hardware utilization using all available CPU cores and GPUs. Results offer valuable insights into the efficiency, robustness, and practical deployment considerations of open-source speaker recognition solutions.
Pruning, as a method of reducing model size and improving performance, has gained increasing traction in recent years. Models such as BERT have been shown to be over-parametrized: multiple attention heads encode the same patterns. Additionally, models trained on general data might underperform on domain-specific tasks, such as recipe interpretation. In this work, we explore the effects of score-based attention head pruning in multilingual transformer models. We conduct our experiments on a dataset comprised of six Indo-European languages with unequal representation across languages, on three tasks of varying difficulty. We grant each attention head a score, based on its contribution to overall model performance, then we evaluate the impact of successive pruning based on said score. Our findings suggest that substantial pruning (up to $80 \%$) can be performed without major performance loss when applied post-finetuning. We show that easier tasks show slower performance degradation as the percentage of pruned heads increases. We also report consistent reductions in inference time. Contrary to our expectations, low-resource languages did not suffer a significantly faster performance degradation when pruning.
Audio splicing is the manipulation of a recording by inserting, deleting, or rearranging segments to alter its content or context with the goal of misrepresenting the original message, fabricating evidence, or creating a misleading narrative. Audio splicing detection has become increasingly vital in the realms of security, digital forensics, and fact-checking, as manipulated audio can be used to spread misinformation or falsify evidence. This paper presents a novel approach to detecting audio splicing by leveraging a self-supervised front end, wav2vec2, in combination with a simple logistic regression classifier. Unlike conventional methods that require handcrafted features or complex models, our method utilizes the rich, context-aware representations produced by wav2vec2 to train a linear classifier that effectively distinguishes between authentic and spliced audio segments. We demonstrate that even with a basic classifier, the wav2vec2 representations capture sufficient discriminative information to achieve good performance on our benchmark dataset.
The study of the environment is one of the permanent objectives of research and innovation. Although numerous monitoring techniques are available, a constant interest in recent times is the use of acoustic methods. There are a multitude of recently published studies that focus on echo-acoustics. Many of these studies are actually partial monitoring and have as their objective the identification of the behavior of certain species of birds or animals. A quasi-complete monitoring of the area and the species that populate it is much more complicated. From the point of view of signal processing techniques, there are solutions that are sufficiently precise for a classification of the received audio signals, but sometimes of high computational complexity. If the monitoring is done through a sensor network, one of the problems that arise is the arrangement of the sensors in a given physical area. It has been previously established that a longitudinal arrangement of the sensors is more efficient than a mesh or grid arrangement, but that the reduction obtained depends on the shape of the monitored area and is not always significant. In this paper we explore the topic of acoustic sensor distribution in a protected area. General considerations are followed by the description of a specific case: a protected area in the vicinity of a lake in Transylvania.
This paper explores an innovative approach for numeral correction in Romanian text, addressing the challenges posed by spelling errors in the input data. The accurate transcription and identification of diverse numeral entities, encompassing both compound numerical expressions and standardized special codes, present significant challenges within natural language processing, particularly for morphologically rich languages such as Romanian. This paper introduces a robust hybrid system engineered to address these complexities by combining advanced artificial intelligence techniques, specifically Named Entity Recognition (NER) and Large Language Models (LLMs), with meticulously crafted rule-based methods. The proposed system architecture is detailed, emphasizing its novel pipeline designed for processing numeral entities in Romanian text. Key contributions include the integration of a Romanian-specific word-to-number conversion module, augmented by an LLM-based fallback mechanism for enhanced accuracy with complex expressions, and a comprehensive suite of validators for numerous Romanian and international standard codes, including but not limited to IBAN, ISBN, VIN and the Romanian personal identification number. The system demonstrates effectiveness in handling a wide array of numeral entities and is designed for practical, offline deployment. This work underscores the efficacy of hybrid approaches in tackling natural language processing tasks, offering a practical solution for numeral entity processing in specialized linguistic contexts.
Automatic Speech Recognition (ASR) for low-resource languages such as Arabic remains a challenging task due to the scarcity of annotated data and linguistic diversity. This paper explores the development of Arabic ASR models using the Mozilla Common Voice 11.0 Arabic dataset. We train models from scratch and investigate cross-lingual transfer learning using pretrained Conformer models trained on high-resource languages, specifically English and French. Through a series of experiments involving language and domain adaptation, we evaluate the performance gains from leveraging pretrained models versus training from scratch. Our findings demonstrate that transfer learning significantly improves model performance, especially in scenarios with limited Arabic training data. Word Error Rate (WER) comparisons reveal that models initialized with English pretrained weights yield better generalization and lower WERs compared to both scratch-trained and French-initialized models, highlighting the importance of source language selection in cross-lingual ASR. This work contributes to the understanding of efficient transfer strategies for Arabic ASR and underscores the potential of multilingual pretraining for low-resource language modeling.
We present an expressive German Text-to-Speech (TTS) system built on a modified VALL-E neural codec language model. Our approach introduces a style-conditioning mechanism to enable emotional and prosodic control during speech synthesis. Using the Thorsten German Emotional TTS dataset, we preprocess and augment 2,400 utterances across 8 emotions. Our model integrates emotion tokens and style embeddings to guide expressive generation without explicit supervision. Preliminary results suggest that this method can produce prosodically varied and natural-sounding German speech, demonstrating its potential in low-resource emotional TTS settings.
The development of automatic speech recognition (ASR) for low-resource languages is severely hindered by the lack of sufficient training data. While traditional data augmentation methods have long been employed to alleviate this issue, recent advances in speech synthesis technologies have shown promising potential for further boosting ASR performance. However, a key research gap remains regarding the impact of different textual sources used for speech synthesis and how synthetic and real speech should be optimally combined during ASR training. This study addresses this gap by systematically investigating (1) what mixing strategy between real and synthetic speech yields the best recognition results for low-resource languages and (2) how synthetic speech generated from different text sources affects downstream ASR performance. Our results show that a 1:1 ratio of real to synthetic speech achieves strong performance across both in-domain and out-of-domain test sets, suggesting it as an optimal training configuration for balancing data quality and quantity. Furthermore, our results also show that text translated from a high-resource language can indeed improve the ASR performance for low-resource languages to some extent. However, using native spoken utterances as the text source for TTS leads to consistently better performance in downstream ASR tasks.
Audio deepfakes are a growing challenge for the general public, as well as for journalists and fact-checkers. The latter need reliable tools to verify the authenticity of their sources, while at the same time keeping their information private. Commercial deepfake detection solutions rely on cloud-based processing, which raises privacy concerns. To solve this problem, we propose an on-device audio deepfake detection model. We show that a truncated self-supervised backbone with a simple logistic classifier is both very fast and often more accurate than existing solutions. Our solution outperforms the baseline AASIST by 10
A robotic model is being developed by the authors to facilitate Speech Language Pathologists for demonstration of correct articulation of speech sounds. This paper, as a part of the larger study, investigates the lip movement during the production of selected consonants by the speakers of Dravidian language Kannada. Thirty Kannada speaking adults were instructed to produce nine selected consonants. Lip movements were traced from still images at two defined points during articulation. Upper lip vertical movement, lower lip vertical movement, left corner horizontal movement and right corner horizontal movement were computed at these defined points. Computed values from the still images showed good reliability, when compared with the measurements taken with digital Vernier caliper. Lower values of coefficient of variation of the computed mean values indicate their suitability to be represented as normal values.
Speaker recognition-based biometric systems are used frequently worldwide because of their qualities and simplicity regarding the hardware setup. The increased usage of such systems has also driven the rise of different speech spoofing methods which can introduce errors for the liveness detection subsystem. This paper focuses on the detection of replay attacks (RA) and of identity spoofing methods using text-to-speech (TTS) or voice conversion (VC). We propose a hybrid MLPCNN neural network-based classification system for discerning between legitimate users’ spontaneous speech and spoofed speech, according to the three types of attacks mentioned. Two datasets were leveraged for developing the system: ASVspoof 2019 (used for training, validation, and evaluation) and ASVspoof 2021 (only for additional evaluation). Several detection scenarios are considered, involving 6 different classification subtasks. Measuring performance using the equal error rate (EER), depending on the subtask, the best proposed model reaches superior or comparable results to other state-of-the-art systems.
