Depression remains one of the most prevalent mental health conditions worldwide, yet most automated detection systems focus on English text and overlook the multilingual, multimodal nature of social media communication. This limitation is particularly significant in India, where users frequently write using Hinglish and other Hindi variants, posing substantial challenges for standard language processing pipelines. This study introduces a time-aware multilingual multimodal framework for detecting depression-related expressions in social media posts collected from X (formerly Twitter). The model jointly learns from text, images, and posting time, enabling it to capture how users express themselves across languages and modalities. To support this work, we construct and annotate a dataset of 15,739 posts from 3,471 users covering four linguistic varieties: English, Roman Hindi, Devanagari Hindi, and Hinglish. The contributions of this work are twofold: it presents a multilingual multimodal dataset for Indian social media depression-related content detection, and it provides a systematic evaluation of encoding and fusion strategies, achieving an F1-score of 0.782 with cross-lingual transfer capability demonstrated on a Bengali corpus. All outputs are intended as a content-screening aid and do not constitute clinical diagnosis.
Large Language Models (LLMs) have demonstrated remarkable capabilities across various natural language processing tasks, yet their effectiveness for Arabic language processing remains underexplored. This paper presents an in-depth evaluation of instruction tuning for Arabic NLP tasks using three prominent LLMs: LLaMA 3.1-8B, AceGPT-v2-8B, and Qwen3-8B. To support this evaluation and future Arabic NLP research, we introduce STAR (inStruction Tuning for ARabic), a publicly released collection of 355 Jinja2 instruction templates spanning 87 datasets and 20 NLP tasks, together with over 46 million rendered instruction–output pairs. Using a 60-template subset of STAR, we evaluate three model variants for each LLM—base, instruction-tuned, and custom LoRA-tuned—on six diverse Arabic NLP tasks (MCQ-based Natural Language Understanding, Natural Language Inference, Dialect Identification, Sarcasm Detection, Machine Translation, and Summarization) across 12 datasets. We employ both intra-dataset evaluation (generalization within training distribution) and intra-task evaluation (generalization across datasets of the same task) to assess model robustness. Results demonstrate that instruction tuning consistently improves performance across most tasks, with notable variations in effectiveness across different tasks and prompts. Models show sensitivity to cross-dataset generalization and prompt styles, while task-specific tuning noticeably reduces prompt sensitivity. Our findings reveal that optimal results depend on careful prompt engineering and task-specific settings, providing insights for developing more effective Arabic language modeling practices. Our code is available at https://github.com/KFUPM-JRCAI/instructions-tuning .
This article presents the Russian Language-Monolingual corpus (RusLan-M, v.1.0), a longitudinal multimedia collection of early child speech from two Russian-speaking monolingual children: Tosya (ages 0;10–3;10, 246 recordings) and Yasha (ages 1;04–3;00, 42 recordings). The corpus consists of approximately 41 h (2,454 min.) of video recordings and 35,386 child utterances, available with transcriptions in the CHAT format on TalkBank. The corpus adheres to strict ethics requirements for data sharing with anonymization. We also conducted two exploratory investigations of the acquisition of Russian morphology using the mean length of utterance (MLU) and the newly developed Index of Productive Syntax for evaluating grammatical complexity in the nominal system of Russian (IPSyn-NP-R). These investigations illustrate how a comprehensive analysis of syntactic and morphological structures in Russian language development can be conducted and highlight the kinds of questions that can be answered using the RusLan-M data. The RusLan-M corpus represents an application of corpus linguistics methods to the Russian language and fills a significant gap in the limited data and resources available for Russian child language research.
With the rise of conversational AI, sentiment-aware chatbots have become essential for enhancing human-machine interactions, allowing for more natural and engaging conversations by making the chatbot aware of the user’s current sentiment before providing a response. In this study, we present a sentiment-aware chatbot optimized for the Arabic language. The Arabic language is known for its complexity and is one of the most widely used languages worldwide, particularly among Arabic-speaking countries. Unlike traditional chatbots, which struggle with contextual understanding, our approach integrates sentiment analysis to generate emotionally adaptive responses. We employ large language models fine-tuned on two Arabic datasets for sentiment classification, achieving state-of-the-art results on the classification task with an accuracy of 91.63
Detecting sarcasm in Arabic remains a challenging task due to the complex linguistic and contextual nature of sarcastic expressions. Sarcasm often contradicts the literal sentiment of a statement, requiring models to understand both the surface meaning and the underlying emotional tone. This challenge is further compounded by Arabic’s rich morphology, diverse dialects, and nuanced social media language. To address these difficulties, we propose a hybrid cascading multi-task model that leverages sentiment information to enhance sarcasm detection. The model first learns sentiment representations through a dedicated head, which are then integrated with contextual embeddings in the sarcasm detection head, capturing the interaction between sentiment and sarcastic intent. Residual connections stabilize learning and preserve fine-grained semantics. Experiments conducted on Arabic datasets show that our model outperforms fine-tuned baseline models and is competitive with most existing approaches. The results confirm that cascading sentiment signals help improve sarcasm detection in Arabic social media text.
Arabic Semantic Textual Similarity (STS) research has been limited by the lack of large-scale, high-quality evaluation resources. We address this gap with the Comprehensive Arabic Semantic Similarity (CASS) dataset, comprising 3,048 manually annotated sentence pairs with fine-grained similarity scores (0–5) spanning six semantic categories and 42 subcategories. CASS is four times larger than existing Arabic STS datasets and provides structured taxonomic coverage supporting systematic model evaluation. We validate CASS by evaluating 21 large language models, including commercial systems, large open-source models, and Arabic-specific systems. Fine-tuning on CASS achieves state-of-the-art performance, with Fanar 9B reaching 0.93 Spearman correlation, exceeding GPT-4o’s zero-shot baseline (0.90). Cross-dataset experiments demonstrate CASS’s superior generalization, with models trained on CASS outperforming those trained on smaller benchmarks by 2–10
Generative Artificial Intelligence (GAI) and Large Language Models (LLMs) have demonstrated significant capabilities in generating human-like content; however, they exhibit a propensity to fabricate spurious information, a phenomenon often termed hallucination. This review paper provides a comprehensive overview of hallucinations in GAI and LLMs. More specifically, this review encompasses their definitions, underlying mechanisms, taxonomies, commonly used tests, and datasets for evaluating hallucinations. In addition, this review dives into intrinsic and extrinsic factors contributing to these inaccuracies, including limitations in model architectures, training data biases, and inference algorithms, as well as examines various detection strategies [e.g., post-hoc consistency checks, external fact-checking, contrastive learning, uncertainty calibration methods, and Retrieval-Augmented Generation (RAG)]. The review also synthesizes a range of correction and mitigation techniques, from proactive measures during training to hybrid approaches that combine detection and intervention. Finally, this review integrates qualitative assessments and comparative insights to delineate the impact of hallucinations on user trust and acceptability, and to shed light on current challenges and future research trends.
Transformer-based models are the current state-of-the-art in machine translation (MT) research. These systems can generate fluent and contextually appropriate translations for many translation tasks and languages. These models are trained on large amounts of unlabelled data, often consisting of billions of examples, and naturally, this data can be imbalanced or contain stereotypes. As a result, the models may produce inaccurate gender assignments in translation and reinforce societal stereotypes. This survey investigates the nature and impact of gender bias in neural MT (NMT). It begins with an overview of how such bias emerges through different data and models. The survey then examines various evaluation frameworks proposed to systematically identify and measure gender disparities in translation outputs. It also categorises mitigation techniques based on the stage at which they operate within an NMT pipeline, from data preprocessing, model adaptation, to inference-level interventions. This survey compiles existing research and highlights challenges to facilitate the creation of more unbiased and gender-aware NMT systems.
Evaluating the quality of long-form answers generated by Question Answering systems presents significant challenges. Traditional metrics, such as BLEU and ROUGE, often reduce the assessment to a single similarity score with a reference answer, failing to capture semantic and specific aspects of answer quality. This reliance on an aggregated score not only overlooks important dimensions but also depends heavily on the availability of reference answers, which may not always be practical or sufficient. Developing metrics capable of individually assessing specific criteria, particularly completeness and relevance, is crucial for identifying weaknesses and guiding improvements in these systems. To address these limitations, this paper introduces specialized metrics designed to evaluate completeness and relevance of long answers without the need for reference texts. We present a new dataset comprising long answers to instructional questions in Computer Science, annotated by human experts based on completeness and relevance. Building upon this, we propose three novel metric models: (1) a prompt-based strategy utilizing Large Language Models to assess answers, (2) an approach that adapts precision and recall concepts by segmenting answers into discrete information units, and (3) a regression model trained on synthetic data to predict completeness and relevance scores. Experimental results demonstrate that the proposed metrics closely align with human judgments and provide more detailed evaluations of completeness and relevance compared to traditional metrics. By enabling a more granular assessment, these metrics facilitate targeted refinements in QA systems, enhancing their ability to meet users’ informational needs more effectively.
Arabic clinical NLP systems often receive short, vague, or incomplete questions, which yields weak downstream answers even with strong encoders. We address this bottleneck by making question quality a first-class and measurable objective. Using domain-adaptive (continued) pretraining with a masked-language objective (DAPT-MLM) on AHQAD ( 808k Arabic health Q–A pairs), we adapt two widely used backbones—AraBERT and the generator variant of AraELECTRA—to the lexical, syntactic, and discourse patterns of well-formed medical questions. Evaluation is aligned with the learning signal: we report cross-entropy and perplexity only at masked tokens, top-k accuracy restricted to masked spans, and lexical-diversity measures to discourage formulaic phrasing. A length-controlled test design (Short/Long/Very Long) isolates modeling gains from verbosity. Results show consistent intrinsic improvements for the domain-adapted models; AraBERT-MLM is best overall (macro Top-5 = 0.8392, lowest CE/PPL), outperforming AraBERT (orig.) by + 6.0 pp Top-5 and AraELECTRA (orig.) by + 17.2 pp. A 200-item human study (clinician + linguist) corroborates these gains (mean ± 95
We present HistoriQA-ThirdRepublic: a French-language dataset of multi-hop historical questions derived from parliamentary debates and newspapers of the French Third Republic. Designed in collaboration with a historian, the corpus captures complex reasoning patterns typical of historical inquiry, including cross-source synthesis, temporal reasoning, and the integration of sparse evidence. The dataset is made of 1782 questions and emphasizes multi-hop connections across heterogeneous historical documents, providing a resource for evaluating retrieval-augmented and large language model systems in domain-specific contexts. We describe the methodology for constructing the corpus, including the selection and alignment of sources, question validation, and metadata integration. While the dataset focuses on French historical documents, our methodology can be readily adapted to other languages and national corpora. Finally, we demonstrate how the corpus can support realistic evaluation scenarios for multi-hop question answering, bridging the gap between NLP benchmarks and the needs of historical scholarship.
Texas German is a contact variety that is the result of dialect mixing of several German dialects brought to Texas from central Europe starting in the 1830s. Since 2001, the Texas German Dialect Project has been assembling a large collection of spoken data documenting this unique variety. The present paper describes how a substantial part of this collection was developed into an annotated corpus and how the corpus is now available through a corpus platform based on the ZuMult technology. We start with an outline of the project’s development and its established processes of data collection, transcription, and dissemination. We then explain the process by which the data were cleaned up and enriched with language tagging, orthographic normalization, lemmatization, and part-of-speech tagging. Finally, we illustrate how the new corpus platform makes these annotated data available for systematic browsing and querying. In the outlook, we sketch prospects for future development of the data and for their role in a larger landscape of comparable speech island data.
This paper presents BivalTyp, a typological database of bivalent verbs and their argument coding patterns across 140+ languages. Based on a questionnaire comprising 130 contextualized sentences with bivalent predicates, BivalTyp aims to facilitate cross-lingual comparison by combining broad language coverage with rich descriptive detail. The online database provides multiple query and visualization tools, including customizable maps, as well as downloadable datasets. We outline diverse research applications, from identifying transitivity hierarchies and semantic-role clustering to analyzing language-specific valency systems and their complexity. A case study demonstrates how BivalTyp data can disentangle genealogical inheritance from areal convergence in shaping argument-coding patterns, revealing that geographical proximity significantly influences valency pattern similarities even after controlling for phylogenetic relatedness.
Large Vision-Language Models are an extension of Large Language Models to process signals from the vision-language domain. Their performance is remarkable since they exploit the cutting-edge capabilities of pre-trained Large Language Models. However, most works using these models focus on English language data for training and evaluation. In light of this, several works have been released to extend current training mixtures to consider non-English data. As a matter of fact, several works also considered cultural aspects to decrease bias related to American and Chinese culture. However, evaluation in non-English languages is still limited w.r.t. English. Specifically, there are no datasets that focus on text-centric non-English images, since most available multimodal benchmarks focus on natural images. This limitation makes it unclear whether Large Vision-Language Models can be used for non-English text-centric tasks. In this work, we aim to understand the capabilities of these models for the Italian language and introduce a benchmark dataset focusing on text-centric Italian images. Therefore, we introduce our benchmark dataset ETCII (Evaluation of Text-Centric Italian Images), to evaluate Large Vision-Language Models on three diverse task categories for text-centric images.
