Depression remains one of the most prevalent mental health conditions worldwide, yet most automated detection systems focus on English text and overlook the multilingual, multimodal nature of social media communication. This limitation is particularly significant in India, where users frequently write using Hinglish and other Hindi variants, posing substantial challenges for standard language processing pipelines. This study introduces a time-aware multilingual multimodal framework for detecting depression-related expressions in social media posts collected from X (formerly Twitter). The model jointly learns from text, images, and posting time, enabling it to capture how users express themselves across languages and modalities. To support this work, we construct and annotate a dataset of 15,739 posts from 3,471 users covering four linguistic varieties: English, Roman Hindi, Devanagari Hindi, and Hinglish. The contributions of this work are twofold: it presents a multilingual multimodal dataset for Indian social media depression-related content detection, and it provides a systematic evaluation of encoding and fusion strategies, achieving an F1-score of 0.782 with cross-lingual transfer capability demonstrated on a Bengali corpus. All outputs are intended as a content-screening aid and do not constitute clinical diagnosis.
Large Language Models (LLMs) have demonstrated remarkable capabilities across various natural language processing tasks, yet their effectiveness for Arabic language processing remains underexplored. This paper presents an in-depth evaluation of instruction tuning for Arabic NLP tasks using three prominent LLMs: LLaMA 3.1-8B, AceGPT-v2-8B, and Qwen3-8B. To support this evaluation and future Arabic NLP research, we introduce STAR (inStruction Tuning for ARabic), a publicly released collection of 355 Jinja2 instruction templates spanning 87 datasets and 20 NLP tasks, together with over 46 million rendered instruction–output pairs. Using a 60-template subset of STAR, we evaluate three model variants for each LLM—base, instruction-tuned, and custom LoRA-tuned—on six diverse Arabic NLP tasks (MCQ-based Natural Language Understanding, Natural Language Inference, Dialect Identification, Sarcasm Detection, Machine Translation, and Summarization) across 12 datasets. We employ both intra-dataset evaluation (generalization within training distribution) and intra-task evaluation (generalization across datasets of the same task) to assess model robustness. Results demonstrate that instruction tuning consistently improves performance across most tasks, with notable variations in effectiveness across different tasks and prompts. Models show sensitivity to cross-dataset generalization and prompt styles, while task-specific tuning noticeably reduces prompt sensitivity. Our findings reveal that optimal results depend on careful prompt engineering and task-specific settings, providing insights for developing more effective Arabic language modeling practices. Our code is available at https://github.com/KFUPM-JRCAI/instructions-tuning .
This article presents the Russian Language-Monolingual corpus (RusLan-M, v.1.0), a longitudinal multimedia collection of early child speech from two Russian-speaking monolingual children: Tosya (ages 0;10–3;10, 246 recordings) and Yasha (ages 1;04–3;00, 42 recordings). The corpus consists of approximately 41 h (2,454 min.) of video recordings and 35,386 child utterances, available with transcriptions in the CHAT format on TalkBank. The corpus adheres to strict ethics requirements for data sharing with anonymization. We also conducted two exploratory investigations of the acquisition of Russian morphology using the mean length of utterance (MLU) and the newly developed Index of Productive Syntax for evaluating grammatical complexity in the nominal system of Russian (IPSyn-NP-R). These investigations illustrate how a comprehensive analysis of syntactic and morphological structures in Russian language development can be conducted and highlight the kinds of questions that can be answered using the RusLan-M data. The RusLan-M corpus represents an application of corpus linguistics methods to the Russian language and fills a significant gap in the limited data and resources available for Russian child language research.
With the rise of conversational AI, sentiment-aware chatbots have become essential for enhancing human-machine interactions, allowing for more natural and engaging conversations by making the chatbot aware of the user’s current sentiment before providing a response. In this study, we present a sentiment-aware chatbot optimized for the Arabic language. The Arabic language is known for its complexity and is one of the most widely used languages worldwide, particularly among Arabic-speaking countries. Unlike traditional chatbots, which struggle with contextual understanding, our approach integrates sentiment analysis to generate emotionally adaptive responses. We employ large language models fine-tuned on two Arabic datasets for sentiment classification, achieving state-of-the-art results on the classification task with an accuracy of 91.63
Detecting sarcasm in Arabic remains a challenging task due to the complex linguistic and contextual nature of sarcastic expressions. Sarcasm often contradicts the literal sentiment of a statement, requiring models to understand both the surface meaning and the underlying emotional tone. This challenge is further compounded by Arabic’s rich morphology, diverse dialects, and nuanced social media language. To address these difficulties, we propose a hybrid cascading multi-task model that leverages sentiment information to enhance sarcasm detection. The model first learns sentiment representations through a dedicated head, which are then integrated with contextual embeddings in the sarcasm detection head, capturing the interaction between sentiment and sarcastic intent. Residual connections stabilize learning and preserve fine-grained semantics. Experiments conducted on Arabic datasets show that our model outperforms fine-tuned baseline models and is competitive with most existing approaches. The results confirm that cascading sentiment signals help improve sarcasm detection in Arabic social media text.
Arabic Semantic Textual Similarity (STS) research has been limited by the lack of large-scale, high-quality evaluation resources. We address this gap with the Comprehensive Arabic Semantic Similarity (CASS) dataset, comprising 3,048 manually annotated sentence pairs with fine-grained similarity scores (0–5) spanning six semantic categories and 42 subcategories. CASS is four times larger than existing Arabic STS datasets and provides structured taxonomic coverage supporting systematic model evaluation. We validate CASS by evaluating 21 large language models, including commercial systems, large open-source models, and Arabic-specific systems. Fine-tuning on CASS achieves state-of-the-art performance, with Fanar 9B reaching 0.93 Spearman correlation, exceeding GPT-4o’s zero-shot baseline (0.90). Cross-dataset experiments demonstrate CASS’s superior generalization, with models trained on CASS outperforming those trained on smaller benchmarks by 2–10
Generative Artificial Intelligence (GAI) and Large Language Models (LLMs) have demonstrated significant capabilities in generating human-like content; however, they exhibit a propensity to fabricate spurious information, a phenomenon often termed hallucination. This review paper provides a comprehensive overview of hallucinations in GAI and LLMs. More specifically, this review encompasses their definitions, underlying mechanisms, taxonomies, commonly used tests, and datasets for evaluating hallucinations. In addition, this review dives into intrinsic and extrinsic factors contributing to these inaccuracies, including limitations in model architectures, training data biases, and inference algorithms, as well as examines various detection strategies [e.g., post-hoc consistency checks, external fact-checking, contrastive learning, uncertainty calibration methods, and Retrieval-Augmented Generation (RAG)]. The review also synthesizes a range of correction and mitigation techniques, from proactive measures during training to hybrid approaches that combine detection and intervention. Finally, this review integrates qualitative assessments and comparative insights to delineate the impact of hallucinations on user trust and acceptability, and to shed light on current challenges and future research trends.
Transformer-based models are the current state-of-the-art in machine translation (MT) research. These systems can generate fluent and contextually appropriate translations for many translation tasks and languages. These models are trained on large amounts of unlabelled data, often consisting of billions of examples, and naturally, this data can be imbalanced or contain stereotypes. As a result, the models may produce inaccurate gender assignments in translation and reinforce societal stereotypes. This survey investigates the nature and impact of gender bias in neural MT (NMT). It begins with an overview of how such bias emerges through different data and models. The survey then examines various evaluation frameworks proposed to systematically identify and measure gender disparities in translation outputs. It also categorises mitigation techniques based on the stage at which they operate within an NMT pipeline, from data preprocessing, model adaptation, to inference-level interventions. This survey compiles existing research and highlights challenges to facilitate the creation of more unbiased and gender-aware NMT systems.
Evaluating the quality of long-form answers generated by Question Answering systems presents significant challenges. Traditional metrics, such as BLEU and ROUGE, often reduce the assessment to a single similarity score with a reference answer, failing to capture semantic and specific aspects of answer quality. This reliance on an aggregated score not only overlooks important dimensions but also depends heavily on the availability of reference answers, which may not always be practical or sufficient. Developing metrics capable of individually assessing specific criteria, particularly completeness and relevance, is crucial for identifying weaknesses and guiding improvements in these systems. To address these limitations, this paper introduces specialized metrics designed to evaluate completeness and relevance of long answers without the need for reference texts. We present a new dataset comprising long answers to instructional questions in Computer Science, annotated by human experts based on completeness and relevance. Building upon this, we propose three novel metric models: (1) a prompt-based strategy utilizing Large Language Models to assess answers, (2) an approach that adapts precision and recall concepts by segmenting answers into discrete information units, and (3) a regression model trained on synthetic data to predict completeness and relevance scores. Experimental results demonstrate that the proposed metrics closely align with human judgments and provide more detailed evaluations of completeness and relevance compared to traditional metrics. By enabling a more granular assessment, these metrics facilitate targeted refinements in QA systems, enhancing their ability to meet users’ informational needs more effectively.
Arabic clinical NLP systems often receive short, vague, or incomplete questions, which yields weak downstream answers even with strong encoders. We address this bottleneck by making question quality a first-class and measurable objective. Using domain-adaptive (continued) pretraining with a masked-language objective (DAPT-MLM) on AHQAD ( 808k Arabic health Q–A pairs), we adapt two widely used backbones—AraBERT and the generator variant of AraELECTRA—to the lexical, syntactic, and discourse patterns of well-formed medical questions. Evaluation is aligned with the learning signal: we report cross-entropy and perplexity only at masked tokens, top-k accuracy restricted to masked spans, and lexical-diversity measures to discourage formulaic phrasing. A length-controlled test design (Short/Long/Very Long) isolates modeling gains from verbosity. Results show consistent intrinsic improvements for the domain-adapted models; AraBERT-MLM is best overall (macro Top-5 = 0.8392, lowest CE/PPL), outperforming AraBERT (orig.) by + 6.0 pp Top-5 and AraELECTRA (orig.) by + 17.2 pp. A 200-item human study (clinician + linguist) corroborates these gains (mean ± 95
Texas German is a contact variety that is the result of dialect mixing of several German dialects brought to Texas from central Europe starting in the 1830s. Since 2001, the Texas German Dialect Project has been assembling a large collection of spoken data documenting this unique variety. The present paper describes how a substantial part of this collection was developed into an annotated corpus and how the corpus is now available through a corpus platform based on the ZuMult technology. We start with an outline of the project’s development and its established processes of data collection, transcription, and dissemination. We then explain the process by which the data were cleaned up and enriched with language tagging, orthographic normalization, lemmatization, and part-of-speech tagging. Finally, we illustrate how the new corpus platform makes these annotated data available for systematic browsing and querying. In the outlook, we sketch prospects for future development of the data and for their role in a larger landscape of comparable speech island data.
This paper presents BivalTyp, a typological database of bivalent verbs and their argument coding patterns across 140+ languages. Based on a questionnaire comprising 130 contextualized sentences with bivalent predicates, BivalTyp aims to facilitate cross-lingual comparison by combining broad language coverage with rich descriptive detail. The online database provides multiple query and visualization tools, including customizable maps, as well as downloadable datasets. We outline diverse research applications, from identifying transitivity hierarchies and semantic-role clustering to analyzing language-specific valency systems and their complexity. A case study demonstrates how BivalTyp data can disentangle genealogical inheritance from areal convergence in shaping argument-coding patterns, revealing that geographical proximity significantly influences valency pattern similarities even after controlling for phylogenetic relatedness.
Large Vision-Language Models are an extension of Large Language Models to process signals from the vision-language domain. Their performance is remarkable since they exploit the cutting-edge capabilities of pre-trained Large Language Models. However, most works using these models focus on English language data for training and evaluation. In light of this, several works have been released to extend current training mixtures to consider non-English data. As a matter of fact, several works also considered cultural aspects to decrease bias related to American and Chinese culture. However, evaluation in non-English languages is still limited w.r.t. English. Specifically, there are no datasets that focus on text-centric non-English images, since most available multimodal benchmarks focus on natural images. This limitation makes it unclear whether Large Vision-Language Models can be used for non-English text-centric tasks. In this work, we aim to understand the capabilities of these models for the Italian language and introduce a benchmark dataset focusing on text-centric Italian images. Therefore, we introduce our benchmark dataset ETCII (Evaluation of Text-Centric Italian Images), to evaluate Large Vision-Language Models on three diverse task categories for text-centric images.
Panels are a fundamental unit of comics, yet basic data about their usage in comics from around the world has not been widely investigated. Here we analyze panel information in the TINTIN Corpus consisting of 1,030 comics from 144 countries—comprising over 14,000 pages with over 76,000 panels—all annotated using the Multimodal Annotation Software Tool (MAST). We examined both the number of panels per page and the relative size of those panels to their pages, finding that they varied in dimensions of the style that they are drawn in, the global region they come from, the typological properties of the languages spoken by their authors, and the year of their publication. In addition, a clear tradeoff occurred between these dimensions of structure, where larger panels appeared for fewer panels per page, and vice versa. This relationship appeared to be “universal”, persisting similarly no matter the variation across style, region, language, or publication date of the comics. Altogether, this work reveals that the structure of panels on comic pages involve a tension between variability across numerous sources and universal consistencies of properties that persist across all comics.
Recently, Large Language Models (LLMs) have gained substantial attention due to their exceptional capabilities in various Natural Language Processing (NLP) tasks, particularly for widely spoken global languages. The increasing adoption of LLMs is primarily attributed to their ability to achieve near-human-level proficiency in language understanding and generation. However, the effectiveness of LLMs in regional languages requires a thorough evaluation before their deployment in NLP applications. This study conducts a performance analysis of OpenAI’s Generative Pre-trained Transformer (GPT) model specifically for Marathi, India’s third most widely spoken regional language. The research focuses on crucial NLP tasks, including Sentiment Analysis, Text Classification, and Paraphrase generation. This paper addresses the challenges of fine-tuning GPT model for regional languages and provides a detailed performance evaluation. The contributions of this study are twofold: firstly, it presents a diverse and validated dataset specifically designed for Marathi NLP tasks; secondly, it offers a detailed performance benchmarking of OpenAI’s GPT model in the context of paraphrasing, text classification, and sentiment analysis for the Marathi text.
This study investigates Tibetan–Chinese poetry translation through a mixed-methods, cross-system comparison of dedicated neural machine translation (NMT) engines and a prompted large language model (LLM). Utilizing a curated corpus of 35 contemporary Tibetan poems paired with published Chinese reference translations, we evaluated four translation conditions: Google Translate, Bing Translator, and ChatGPT-4o subjected to two distinct prompt designs. Quantitative evaluation employing BERTScore, COMET-22, and LEPOR captured complementary dimensions of reference-based similarity and linguistic adequacy. The results revealed clear, metric-dependent hierarchies. Bing Translator achieved the highest BERTScore, Google Translate obtained the strongest performance under LEPOR, and ChatGPT-4o with Prompt 1 yielded the best COMET-22 scores. Within the ChatGPT-4o framework, prompt design produced measurable variation; notably, the more stylistically elaborative prompt exhibited reduced reference-based alignment, suggesting an inherent trade-off between rhetorical elaboration and semantic proximity to the reference. Complementing the quantitative metric findings, a qualitative error analysis grounded in Nord’s functionalist typology identified cross-system failure modes. These included numerical misrecognition, loss of symbolic and relational nuance, affective shifts, and disruptions to the text’s metaphorical architecture. Ultimately, these findings highlight both the potential and the prevailing limitations of automated systems for low-resource literary translation, underscoring the critical value of integrating automated metrics with theory-informed qualitative analysis for poetry evaluation.
In this paper, we introduce a novel computer-aided pronunciation training (CAPT) corpus that was designed to facilitate the construction of a high-performance CAPT system and includes the most frequent pronunciation errors made by nonnative Arabic speakers learning Arabic as a second language. This corpus is named the L2AraSpeech corpus and is designed to overcome the lack of nonnative Arabic databases needed to build a high-performance Arabic CAPT system. L2AraSpeech includes speech from 220 speakers from diverse backgrounds in terms of country and mother language. The methodology of building L2AraSpeech consisted of different phases that started with the selection of text that was carefully designed for CAPT systems by linguistic experts, and pronunciation errors were very carefully annotated by expert Arabic linguists. The analysis of pronunciation errors revealed that the most frequent pronunciation errors made by speakers were consistent with linguistic studies in the literature. The corpus was validated in an end-to-end mispronunciation detection and correction (MDD) system, and the results are encouraging.
The construction of lexical-semantic resources for low-resource languages is a crucial task in natural language processing, as it enables the development of linguistic technologies that are often unavailable for under-represented languages. In this study, we present a methodology for building a structured lexical-semantic resource for the Moroccan Arabic dialect (Darija) as a low-resource language. Our method consists of two steps. The first step involves mapping a lexico-semantic network for Darija to the Princeton WordNet and the Suggested Upper Merged Ontology (SUMO). We apply the Lexical Ontology Inference (LeOnI) framework to link Darija words using a bilingual resource and existing WordNet-SUMO mapping. Darija words are classified as monosemous or polysemous to guide the mapping process. The second step introduces a similarity-based refinement process, combining semantic similarity components with ontological-lexical adjustment factors. A scoring function reliably guides the automatic mapping and disambiguation of synsets and concepts. Our results demonstrate that the combination of symbolic and distributional semantics yields accurate and interpretable wordnet-like resources for dialects. We also analyze semantic coverage and translation gaps, highlighting concepts that are untranslatable or culturally specific to Darija. The proposed framework can be generalized to other low-resource languages, as the core mapping and refinement stages in our method are language-independent. Once the Darija lexical-semantic resource is finalized, the constructed dataset will be made publicly available to promote reproducibility and facilitate research into Arabic semantic processing and dialectal natural language processing.
Toxic language detection in Arabic remains an underexplored yet critical task, complicated by the language’s dialectal diversity, morphological richness, and the prevalence of overlapping toxicity types. In this paper, AraTox is presented as a large-scale, hand-annotated Arabic dataset designed for multi-label and multi-dialect toxicity classification. The dataset comprises around 36,700 social media comments spanning Gulf, Levantine, Nile Basin, North African, Yemeni, and Modern Standard Arabic (MSA) varieties, each manually labeled by expert annotators across seven categories, including hatred, cussing, racial, appearance-based, and sexual insults. The categories were derived from an iterative refinement of overlapping harassment types. AraTox is benchmarked using a range of deep learning models and classical classifiers, highlighting the superior performance of a stacked meta-learning ensemble over standalone models. The meta-classifier achieves a macro-averaged F1-score of 96
