We introduce ParCzech4Speech 1.0, a processed version of the ParCzech 4.0 corpus, targeted at speech modeling tasks with the largest variant containing 2,695 h. We combined the sound recordings of the Czech parliamentary speeches with the official transcripts. The recordings were processed with WhisperX and Wav2Vec 2.0 to extract automated audio-text alignment. Our processing pipeline improves upon the ParCzech 3.0 speech recognition version by extracting more data with higher alignment reliability. The dataset is offered in three flexible variants: (1) sentence-segmented for automatic speech recognition and speech synthesis tasks with clean boundaries, (2) unsegmented preserving original utterance flow across sentences, and (3) a raw-alignment for further custom refinement for other possible tasks. All variants maintain the original metadata and are released under a permissive CC-BY license. The dataset is available in the LINDAT repository, with the sentencesegmented and unsegmented variants additionally available on Hugging Face.
Teachers are increasingly using prompted LLMs to generate exam questions, and students can use generated questions for self-assessment. When generating questions from a given educational text-rather than relying solely on the LLM's internal knowledge-handling long textual content, such as a textbook spanning hundreds of pages, presents a challenge. In this paper, we experiment with three knowledge representation approaches tailored for educational question generation using LLMs. As a novel contribution among these alternatives, we adapt the atomic fact decomposition method from fact-checking research to the educational domain. We manually evaluate the generated questions based on various criteria. Our empirical results indicate that a list of atomic facts provides a better foundation for question generation than long plain text and that LLM-based question generation from Knowledge Graph triplets outperforms rule-based question generation from Knowledge Graphs.
We investigate what self-supervised speech recognition models (S3Ms) learn about speaker groups (SGs). We examine several states of S3Ms: pretrained, finetuned on speaker identification (SID), as well as finetuned automatic speech recognition (ASR) using fairness enhancing algorithms. We find that S3Ms encode information about several speaker group categories (SGCs), including their gender, age, dialect, ethnicity, and whether they are a native speaker. We find that finetuning for SID amplifies certain SGCs, namely those whose variance is more phonetic in nature. Meanwhile, finetuning for ASR discards phonetically variant speaker group information (SGI) but retains semantically variant SGI. We find that ASR algorithms designed for fairness improvement change to what extent SGI is encoded in S3Ms; however, this is primarily true for phonetically variant SGCs, and less true for semantically variant SGCs. We discuss how SGI is encoded by each layer, and identify subdimensions of embeddings responsible for encoding different SGCs. Finally, we discuss how our findings could be beneficial in designing fairer ASR algorithms.
Large language models (LLMs) have demonstrated remarkable performance across a wide range of NLP tasks. However, their effectiveness in discourse parsing remains underexplored where the existing LLM-based attempts fall significantly short of the performance achieved by the encoder-based models. In this study, we propose a Chain-of-Thought (CoT) prompting approach for the task of implicit discourse relation recognition (IDRR), leveraging the concept of abstract objects. We show that guiding the model to identify abstract objects within the arguments of the discourse relation systematically enhances the classification performance across both Level-1 and Level-2 senses, in both monolingual and multilingual settings. Through experiments on three monolingual and one multilingual corpora, covering seven languages and annotated according to PDTB 3.0, we demonstrate that our CoT-style prompting approach achieves significant improvements over previous LLM-based methods.
Dimensional affect prediction from speech has traditionally relied on acoustic features to estimate continuous affect representations (e.g., arousal, valence) at each time step. However, affect evolves dynamically over time, and incorporating temporal information may improve prediction accuracy. This study investigates emotional dynamics in speech emotion recognition using multitask learning, where a model jointly predicts both the affect state and its temporal derivative. Experiments on the RECOLA and SEWA datasets show that incorporating dynamic information improves affect state prediction, particularly for valence, known to be challenging to model from audio alone. While CCC scores for affect dynamic predictions remain lower than those for affect state predictions, results indicate that learning dynamics as an auxiliary task enhances affect state estimation over time. These findings underscore the importance of modelling emotional dynamics to capture the temporal evolution of affect.
Parkinson's disease (PD) and mild cognitive impairment (MCI) affect both motor and cognitive linguistic abilities. This work proposes a multimodal framework for detecting PD and MCI from naturalistic retelling tasks by combining acoustic and linguistic features. Speech features capture prosodic, articulatory, and phonemic properties linked to hypokinetic dysarthria, while language features model lexical, syntactic, and semantic complexity, including a novel motility-based representation that quantifies the use of action-related verbs. Each modality is independently evaluated and combined following early and late fusion strategies based on support vector machines. The results confirm speech features as good biomarkers to model PD and MCI, and show the motility-based language features improve specificity, particularly in distinguishing cognitive decline in Parkinson's patients, i.e., PD patients with MCI vs. patients without MCI. Fusion strategies further improve classification performance, confirming the complementarity of speech and language. These findings support the use of retellingbased multimodal analyses as promising tools for early and non-invasive screening of neurodegenerative conditions.
With recent advances in natural language processing, part-of-speech (POS) tagging is one of the areas that has seen significant improvements. Contemporary state-of-the-art tools report accuracies approaching 100
Speech processing and translation technology have the potential to facilitate meetings of individuals who do not share any common language. To evaluate automatic systems for such a task, a versatile and realistic evaluation corpus is needed. Therefore, we create and present a corpus of cross-lingual dialogues between individuals without a common language who were facilitated by automatic simultaneous speech translation. The corpus consists of 5 h of speech recordings with ASR and gold transcripts in 12 original languages and automatic and corrected translations into English. For the purposes of research into cross-lingual summarization, our corpus also includes written summaries (minutes) of the meetings. Moreover, we propose automatic detection of misunderstandings. For an overview of this task and its complexity, we attempt to quantify misunderstandings in cross-lingual meetings. We annotate misunderstandings manually and also test the ability of current large language models to detect them automatically. The results show that the Gemini model is able to identify text spans with misunderstandings with recall of 77
We present a practical pipeline for multilingual domain adaptation in automatic speech recognition (ASR) that combines the Whisper model with large language models (LLMs). Using Aya-23-8B, Common Voice transcripts in 22 languages are automatically classified into the Law and Healthcare domains, producing high-quality domain labels at a fraction of the manual cost. These labels drive parameter-efficient (LoRA) fine-tuning of Whisper and deliver consistent relative Word Error Rate (WER) reductions of up to 14.3
Automatic speech recognition (ASR) systems often struggle with dysarthric speech, particularly from individuals with neurodegenerative conditions such as Parkinson's disease. Addressing this challenge, we explore the use of instruction-tuned large language models (LLMs) for ASR error correction. Our approach uses zero-shot prompting with only the 1-best greedy ASR output. Using speech data from the Speech Accessibility Project, we fine-tune a 250M parameter Flan-T5 model to improve transcription accuracy for dysarthric speakers. Evaluation against raw outputs from multiple open-source ASR systems demonstrates notable improvements in correction quality. We further assess larger models, such as the 8B parameter LLaMA 3.1 model, and observe additional gains with minimal hyperparameter tuning. These results highlight the potential of LLM-based correction to enhance ASR accessibility for individuals with speech impairments.
We present a comparative analysis of text complexity across domains using scale-free metrics. We quantify linguistic complexity via Heaps' exponent β (vocabulary growth), Taylor's exponent α (word-frequency fluctuation scaling), compression rate r (redundancy), and entropy. Our corpora span three domains: legal documents (statutes, cases, deeds) as a specialized domain, general natural language texts (literature, Wikipedia), and AI-generated (GPT) text. We find that legal texts exhibit slower vocabulary growth (lower β) and higher term consistency (higher α) than general texts. Within legal domain, statutory codes have the lowest β and highest α, reflecting strict drafting conventions, while cases and deeds show higher β and lower α. In contrast, GPT-generated text shows the statistics more aligning with general language patterns. These results demonstrate that legal texts exhibit domain-specific structures and complexities, which current generative models do not fully replicate.
Discourse relations represent a relatively ambiguous area of language that can be a challenge for computational and corpus linguistics. In our study, we examine the reliability of the Prague Dependency Treebank – Consolidated 2.0 by employing multiple annotations that can reveal possible different readings. It turns out that the complexity of sentence structure has a very significant influence on the distinction of the left discourse argument; in contrast, neither the mode of the text (written vs. spoken) nor the presence of attributive constructions appears to influence the variability in the interpretation of discourse structure. Furthermore, we identify the technical organization of the annotation process itself as a potential source of bias in discourse annotation.
We present a compact, single-model approach to multilingual inflection, the task of generating inflected word forms from base lemmas to express grammatical categories. Our model, trained jointly on data from 73 languages, is lightweight, robust to unseen words, and outperforms monolingual baselines in most languages. This demonstrates the effectiveness of multilingual modeling for inflection and highlights its practical benefits: simplifying deployment by eliminating the need to manage and retrain dozens of separate monolingual models. In addition to the standard SIGMORPHON shared task benchmarks, we evaluate our monolingual and multilingual models on 73 Universal Dependencies (UD) treebanks, extracting lemma-tag-form triples and their frequency counts. To ensure realistic data splits, we introduce a novel frequency-weighted, lemma-disjoint train-dev-test resampling procedure. Our work addresses the lack of an open-source, general-purpose, multilingual morphological inflection system capable of handling unseen words across a wide range of languages, including Czech. All code is publicly released at: https://github.com/tomsouri/multilingual-inflection.
End-to-end automatic speech recognition (ASR) systems perform well for high-resource languages but struggle in low-resource settings due to the scarcity of usable training data. Recent advances in speech synthesis, particularly voice cloning, offer promising solutions by augmenting training corpora with synthetic speech. However, the impact of such synthetic data on ASR performance in low-resource scenarios remains underexplored. In this study, we take Hungarian as a case study to systematically evaluate how synthetic speech can enhance ASR under data-scarce conditions. We investigate multiple strategies for generating synthetic data and assess their effectiveness through extensive experiments. Our results show that incorporating synthetic speech can significantly reduce word error rates. Compared to the baseline, the largest relative improvement on the in-domain test set (BEA-Base) reaches 12.1
In this contribution, we present two approaches to efficiently enhance end-to-end (E2E) automatic speech recognition (ASR) models for the Norwegian language. Both utilize multilingual models. First, we demonstrate that model performance can be significantly improved if trained with encoder parameters initialized from models created for other languages. This is true not only for closely related languages, like Swedish or Danish, but also for more or less distant ones, like German, English, Italian, or even Ukrainian. This type of model enhancement is achieved without any additional training data, so it requires no extra computation time. Second, having multiple and differently initialized models for Norwegian offers another advantage. They can be used in data harvesting as multiple checkers to validate correct annotations in parallel. This allows us to acquire a large amount of additional training data automatically from various public sources, such as YouTube, parliament, or government archives. We evaluate our final Norwegian model (trained on 2,599 h) on a diverse 18-h test set and compare its performance to major ASR service providers (Google, Microsoft, Speechmatics) and two fine-tuned Whisper models.
Clinical notes hold rich yet unstructured details about diagnoses, treatments, and outcomes that are vital to precision medicine but hard to exploit at scale. We introduce a method that represents each patient as a matrix built from aggregated embeddings of all their notes, enabling robust patient similarity computation based on their latent low-rank representations. Using clinical notes of 4,267 Czech breast-cancer patients and expert similarity labels from Masaryk Memorial Cancer Institute, we evaluate several matrix-based similarity measures and analyze their strengths and limitations across different similarity facets, such as clinical history, treatment, and adverse events. The results demonstrate the usefulness of the presented method for downstream tasks, such as personalized therapy recommendations or toxicity warnings.
Disfluency labeling plays a crucial role in improving end-to-end automatic speech recognition (ASR) of spontaneous speech, which is often characterized by non-lexical and disfluent acoustic events such as hesitations, backchannels, and broken words. In this study, we revisit disfluency labeling methods and evaluate their effectiveness across three distinct Hungarian conversational speech datasets, including a novel corpus of spontaneous speech from patients with mental illness. Our experiments demonstrate that disfluency-aware modeling significantly reduces overall error rates across all datasets. Furthermore, training on mixed datasets-including both typical and atypical speech-yields substantial improvements in recognition accuracy. These findings indicate that robust recognition of both lexical content and disfluent elements is attainable, even in lowresource settings, provided that sufficient training data is available.
We present a comparison of neural network-based morphological segmenters trained on morphologically segmented datasets from seven European languages: Czech, English, French, German, Italian, Dutch, and Slovak. Our aim is to investigate how different model architectures and dataset sizes influence segmentation quality, and how performance varies across languages. To this end, we evaluate recurrent and convolutional neural network models and compare them to widely used unsupervised baseline methods. In selecting the datasets, we prioritized linguistic accuracy and segmentation completeness. We also explore the impact of cross-lingual transfer learning on model performance. Our results show that neural models trained on as few as 125 words outperform unsupervised methods. Moreover, for closely related languages, zero-shot cross-lingual transfer learning can also surpass unsupervised baselines. Overall, we observe consistent performance patterns across languages.
Overlapping Speech Detection (OSD), which involves identifying segments where multiple speakers talk simultaneously, is essential for tasks such as speaker identification, diarization, and automatic speech recognition. However, existing methods often struggle, especially under real-world acoustic conditions that include noise, reverberation, and emotional fluctuations. This paper introduces TOSD-Net, a robust frame-level OSD approach designed to operate effectively in diverse acoustic environments. TOSD-Net incorporates a local feature extraction module based on 1D Convolutional Neural Network (CNNs), which captures short-term temporal patterns across neighboring frames and learns highlevel spectral representations by integrating information across frequency bins. These local features are then passed to a transformer encoder, which models global temporal dependencies by enabling each frame to attend to all others via multi-head self-attention, effectively capturing long-range interactions across the entire sequence. A final classification layer predicts whether each frame corresponds to overlapping or single-speaker speech. To evaluate the proposed approach, overlapping speech segments are simulated using the GRID corpus for neutral speech and the RAVDESS corpus for emotional speech. Both datasets are further augmented with ambient noise, reverberation, and their combinations at multiple signal-to-noise ratios, reflecting a wide range of real-world acoustic conditions. Experimental results show that TOSD-Net significantly outperforms state-of-the-art OSD baselines, demonstrating its effectiveness and robustness across diverse acoustic environments.
Timely detection of cognitive disorders is critical yet often limited by resource-heavy clinical assessments. This paper presents an automated framework for early cognitive screening based on the semantic analysis of spoken image descriptions in Czech. The system integrates automatic speech recognition, formal semantic parsing, and machine learning to evaluate deviations from an expert-defined reference description. The responses of the participants are analyzed for missing or incorrect semantic content, producing structured loss vectors used for classification. Evaluation on a clinically annotated dataset of 268 samples (split into train-test subsets) shows that semantic features outperform traditional lexical and morphological baselines, highlighting the potential of the method for scalable and interpretable cognitive assessment.
