Word Sense Disambiguation (WSD) is crucial in computational linguistics, particularly for languages with complex morphology such as Urdu. We introduce KAAN-GNN-BERT, an ensemble model that integrates Knowledge-Aware Attention Networks (KAAN), Graph Neural Networks (GNNs), and BERT transformer architecture within a semi-supervised learning framework. The model is evaluated on two benchmark Urdu datasets: ULS-WSD and UAW-WSD. KAAN-GNN-BERT leverages contextual embeddings from BERT to capture subtle semantic nuances, while GNNs infuse structured knowledge through a tripartite graph connecting pre-contexts, post-contexts, and sense identifiers. This structured representation provides rich relational insights. The KAAN mechanism aligns BERT embeddings with graph-based knowledge, enabling the model to highlight and assimilate the most relevant information. By operating in a semi-supervised manner, KAAN-GNN-BERT draws strength from both labeled and unlabeled data, improving generalizability and mitigating class imbalance issues. Experimental results demonstrate that KAAN-GNN-BERT significantly outperforms traditional supervised methods and current state-of-the-art models. It achieves F1 scores of 81.3% on ULS-WSD and 79.4% on UAW-WSD, reflecting over a 30% improvement compared to baseline models. Furthermore, the GNN component supports community detection, enhancing interpretability and providing insights into sense groupings. These findings underscore the robustness and precision of integrating transformer-based architectures with graph-driven relational knowledge and attention mechanisms. KAAN-GNN-BERT thus emerges as a powerful solution for WSD, offering strong performance even in resource-scarce linguistic contexts and paving the way for future advancements in semantic understanding.
Sequence labeling tasks such as part-of-speech (POS) tagging and named entity recognition (NER) have advanced significantly in high-resource languages and well-structured texts. However, low-resource languages, such as Urdu, face unique challenges due to limited annotated resources, complex morphology, and the informal, noisy nature of social media text. To address this, we introduce two relatively large-scale annotated datasets of Urdu tweets: a POS tagging dataset with 39 syntactic tags and an NER dataset covering three major entity types (person, location, and organization), capturing the linguistic diversity of user-generated content. We benchmark these datasets using a spectrum of approaches, from traditional conditional random fields (CRFs) to deep neural architectures with static embeddings and fine-tuned pre-trained language models (PLMs). Building on these findings, we propose a transformer-based hybrid architecture, XLM-R–CNN–BiLSTM–CRF, which integrates XLM-R embeddings for contextualized representation, convolutional neural networks (CNNs) for local feature extraction, bidirectional long short-term memory (BiLSTM) networks for sequential modeling, and a CRF layer for optimal sequence prediction. Our approach achieves state-of-the-art performance, with F1 scores of 95.39% for POS tagging and 91.91% for NER, significantly surpassing strong baselines. These resources and methods advance sequence labeling for Urdu while providing insights for other low-resource, noisy languages.
Advancements in natural language processing (NLP) have markedly improved paraphrase generation, an essential task for numerous applications. However, current methods face limitations due to model and constraint specificity, which hinder their flexibility and practical deployment. In this work, we introduce a unified prompt-driven approach to paraphrase generation that leverages diverse prompts, enabling fine-grained user control over aspects such as syntax and sentiment. Moreover, we incorporate translation to enable sophisticated cross-lingual text controls. Our system employs a data-centric paradigm which organizes prompts with natural language instructions. The proposed method is compatible with various sequence-to-sequence architectures and utilizes a novel training strategy to address the versatility of prompt combinations. Empirical results show that our approach not only demonstrates its capacity to adhere to multiple user-defined constraints but also maintains high performance in generation tasks without prompts. Moreover, extensive analysis shows that the model exhibits robustness to prompt variance such as language and quantity.
The effectiveness of Speech-to-Text (STT) models depends heavily on dataset-level audio and speech characteristics, yet the quantitative influence of these factors remains insufficiently explored, particularly for low-resource lauguages, such as Vietnamese. This study examines how specific audio and speech characteristics, including Speech Rate, Naturalness, Signal-To-Noise Ratio, Audio Coloration and Environmental Reverberation, affect STT performance for Vietnamese. Amongst them, naturalness is notably picked as a new evaluative characteristic with a dedicated metric for dataset selection. Experiments in a real-world setting with a social robots how that tailoring datasets based on these characteristics can respectively improve the accuracy of the trained models by approximately 2.66%, 4.72%, 8.36%, 5.89%, and 5.00% compared to training on untailored ones. Additionally, models trained on curated datasets can outperform conventional pre-trained models by up to approximately 8.7% accuracy-wise, highlighting the effectiveness of our approach. The methodology is most useful in practical deployments - such as social robots, voice assistants, and contact-center systems - where field audio is noisier, reverberant, and produced by diverse, non-uniform speakers; its benefit diminishes once sufficiently large, representative training datasets exist.
Current stance detection models predominantly rely on text analysis, often overlooking the wealth of information embedded in user profiles. This oversight is largely attributed to the scarcity of stance detection datasets that encompass detailed user profiles. Bridging this gap, we have created the first dataset for Chinese stance detection that includes user profile information (PC-STANCE). The dataset contains 31,033 Chinese microblogs annotated with stances toward 8 targets, as well as user gender, age, and location information. We have conducted a detailed analysis of the dataset and performed extensive empirical experiments using classic neural network models. Our experiments achieve state-of-the-art results, surpassing several large language models like Llama and ChatGPT. This confirms the effectiveness of integrating user profile information in stance detection and underscores the challenging nature of the dataset. To facilitate further research in stance detection, we have made the dataset publicly available. 1
The availability of high-quality corpora is foundational for advancements in Natural Language Processing (NLP), enabling the training and rigorous evaluation of computational models. While rich textual resources exist for high-resource languages, a significant scarcity persists for many natural languages, particularly understudied Arabic dialects such as Kuwaiti Arabic (KA). This article introduces Hazawi+ , a multi-domain textual corpus comprising over 7 million tokens of KA dialectal stories and novels. Unlike social network texts, Hazawi+ is specifically designed to capture the rich linguistic features inherent in narrative texts, including morphological complexity, informal syntax, and pragmatic nuances, making it an invaluable resource for developing NLP models in low-resource settings. The entire corpus underwent automatic morphological annotation using CAMeL tools specialized for Gulf Arabic, with annotation quality subsequently validated through a rigorous manual review of a 105,770-token sample by two language experts. To demonstrate Hazawi+ ’s immediate usability, we present a complementary empirical study involving the programmatic generation of a synthetic dataset of KA stories, which is then utilized in a downstream task to train a classifier capable of distinguishing between human-written and bot-generated narratives. This experiment serves as a crucial proof-of-concept, underscoring Hazawi+ ’s potential to provide researchers with deep insights into dialectal linguistic patterns and to significantly enhance the precision of various language processing tasks for the Kuwaiti Arabic dialect.
Arabic Sign Language (ArSL) is a formal language used by Deaf and Hard of Hearing communities. It helps them understand Arabic spoken by hearing people, particularly in formal speech, such as news or sermons. The production of sign language systems can support communication by translating spoken languages into ArSL videos. However, current systems often fail to meet user needs due to poor translation of grammatical structures and limited ability to capture the textual meaning of speech. This research focused on a neural machine translation (NMT) approach to convert spoken Arabic texts into Arabic Sign Language (ArSL) using a transformer-based model to overcome these difficulties. The ArSL video of the sign is produced as a series of 2D skeletal poses. The proposed methodology is based on an encoder-decoder architecture: texts are represented by a transformer model, and a posture decoder generates gestures. The model is evaluated using quantitative criteria, such as mean square error (MSE) and the dynamic time warping (DTW) measure, as well as qualitative evaluations by sign language experts and users from the Deaf and Hard of Hearing community. The study uses Friday sermon data as a reference to build the translation model and applies optimization techniques such as temporal modulation, noise addition, and resizing to enhance performance. The results show that the enhanced model improves the quality of gesture representation, reducing DTW by 43.4% relative to the baseline in the non-continuous ArSL test. On the other hand, in the Continuous ArSL test using Short Sermon, the DTW is reduced by around 2.25%. In addition, the enhanced model received high user acceptance, with 75.2% satisfaction. Moreover, unified qualitative evaluation criteria for sign language production are proposed, including clarity, accuracy, consistency, and naturalness, which could serve as a basis for future evaluations in this field. These criteria are used to evaluate the generated ArSL and measure user satisfaction.
Short videos follow the trend of creation, leading to a proliferation of homogenized video content. Textual overlays such as titles in short videos often reflect semantic homogeneity. This phenomenon manifests not only in the syntactic structure and overall thematic expression, but also in local semantic elements such as words and phrases. Based on information retrieval techniques, we propose a text semantic homogenization recognition network (TSHR-Net) for short video title overlays. The framework comprises key components: (1) a dynamic semantic representation that incorporates contextual information of title overlays using RoBERTa pre-trained word embeddings; (2) a dual-path semantic parser that integrates global semantics via BiLSTM-Attention and local semantics via multi-scale TextCNN; (3) a ranking loss optimization is designed to measure cosine similarity between semantic features, thereby improving homogenization recognition accuracy. Experimental results show that our TSHR-Net achieves the competitive performance in Chinese text semantic homogenization recognition, with ρ and ρ X,Y reaching 80.16% and 78.23% on LCQMC, 81.05% and 80.19% on STS-B(ZH), and 92.47% and 92.19% on our self-built BJUT-HCD. The model also exhibits generalization ability in English, attaining 79.68% and 79.89% on STS-B(EN), and 73.56% and 72.21% on SICK dataset, respectively.
Neural sequence models, despite their proficiency in generating highly fluent sentences, have also exhibited a tendency to hallucinate, introducing additional content that lacks grounding in the input data, as evidenced by recent investigations. This variety of fluent yet erroneous outputs poses a significant challenge, as it is difficult for users to discern the veracity of the presented content and identify inaccuracies. Several methods have been proposed to address this challenge. However, they sometimes detect the synonym of the true token in the generation as hallucination. To alleviate the above problem, we propose a novel architecture, i.e., Synonym Knowledge Graph Enhanced Language Model (SKGELM). We construct and prune a synonym knowledge graph according to the source sentence, which can help the subsequently graph attention network and classifier detect hallucination. Empirical results on SUMMAC benchmark and multi-domain Chinese–English translation benchmark show that our method achieves state-of-the-art performance and improves the best baseline significantly.
Named Entity Recognition (NER) for Arabic dialects faces persistent challenges due to the scarcity of annotated resources, severe class imbalance, and high linguistic variability. These factors hinder both model accuracy and cross-dialect generalization, creating a pressing need for more efficient annotation strategies and robust learning approaches. Motivated by these challenges, this work investigates the integration of active learning strategies with a semantic oversampling method to enhance performance on Algerian and Moroccan dialectal corpora. Three sampling strategies, Random, Uncertainty, and Diversity, are evaluated at incremental annotation levels (20%, 40%, 60%, and 80%), both with and without oversampling. The proposed semantic oversampling approach generates contextually coherent synthetic examples to alleviate underrepresented entity classes. Experiments conducted with three pre-trained language models, AraBERT, MARBERT, and Multi-dialect-BERT-Base-Arabic, demonstrate that semantic oversampling provides substantial early-stage improvements, particularly in recall, with consistent benefits observed across models. However, overall F1-scores remain modest (≤ 55%), and cross-dialect transfer performance is still limited. These findings indicate that while combining active learning with semantic oversampling improves annotation efficiency and model robustness, further progress in dialectal NER will require richer, more diverse datasets and dialect-aware modeling techniques.
Metaphor, as a common type of linguistic expression, helps people intuitively understand complex concepts in communication, writing, and cognition. Metaphor components, including source-domain words and target-domain words, are critical elements for metaphor identification and interpretation. This article focuses on metaphor components and proposes a metaphor components identification framework employing F eedback-enhanced F eature-driven I n- C ontext L earning (FF-ICL) based on the large language model (LLM). Specifically, in-context learning and feedback mechanisms inspired by human learning are integrated. Firstly, a machine feedback mechanism is designed to perform prior predictions on training samples, constructing a candidate demonstration pool enriched with prediction results and feedback information. Secondly, a multi-head graph attention network (GAT) is introduced to capture the linguistic and structural information embedded in metaphorical expressions, producing feature-rich representations and establishing a vector repository. Based on the repository, the framework retrieves demonstrations most relevant to the input query across different feature dimensions, incorporating in-context prompts to effectively fine-tune the LLM. Experiments and analyses on public datasets demonstrate the superiority of FF-ICL. Furthermore, the metaphor concept mapping experiment validates the crucial role of metaphor components in downstream computational metaphor tasks. Relevant data and codes are available at https://github.com/WXLJZ/FF-ICL .
Machine translation has increasingly shifted toward Neural Machine Translation (NMT) because of its ability to handle input and output sequences of varying lengths. The incorporation of attention mechanisms in NMT systems enables the model to focus on the most relevant parts of the source sentence, rather than relying solely on a fixed representation of the entire input. While NMT improves translation quality by addressing long-range dependencies and contextual understanding, it also requires a large parallel corpus for training, which is a challenge for languages with less resources. The main focus of this research is to give solution for the unique challenges of translating Ayurvedic texts using NMT. Ayurvedic texts have collection of special and scientific words related to medicines and treatments. This makes the translation process more complex and needs very efficient approach for accurate translations. Also, the content of ayurvedic text books is in the form shlokas which is formed using very complex and compound words. In order to simplify the translation process efficiently this work uses a sandhi splitter module and an Anvaya Generator/ word reordering module. In order to develop NMT system for low resource language pair Sanskrit-Malayalam, there is a need of developing a parallel corpus especially for Ayurvedic text books. Also, as the NMT model is proposed for translation it requires a minimum amount of parallel data in the corpus. So, a number of general domain Sanskrit text books with verses, called shlokas, were also considered for developing parallel corpora. The authors developed a parallel corpus for Anvaya Generator, sandhi splitter and translation. Mainly four NMT models were developed trained and tested especially for shlokas as input. The two models are basic transformer model with attention and an encoder-decoder model using Long-Short term Memory (LSTM) with attention. The other two are developed by adding two modules called Sandhi Splitter and Anvaya Generator in the pre-processing stages of the earlier models- Transformer based model and LSTM based model. The limitations of low resources and richness in grammatical structure of Sanskrit- Malayalam language pair are overcome by the concepts of deep learning and the additional modules used in preprocessing stages for developing the models. The models were tested with and without sandhi splitter and Anvaya Generator modules. The transformer-based model integrated with sandhi splitter and Anvaya Generator system achieved a higher average BLEU score of 73.11 and a uni-gram BLEU score of 76.93 for Sanskrit verses to Malayalam translation.
This article introduces TriLex-Darija , a large-scale affective lexicon suite and a hybrid lexicon–transformer framework for analyzing Moroccan Arabic (Darija) social media text across three complementary dimensions: sentiment, emotion, and pragmatic context. The resource is constructed from a corpus of 288,709 manually annotated comments and consists of three unigram lexicons, each mapping 147,565 words to normalized probability distributions over task-specific labels. We first evaluate a symbolic lexicon-based classifier (without machine learning) based on word-level score aggregation to assess the intrinsic quality of the proposed TriLex-Darija resource. Despite the absence of contextual modeling, this approach achieves competitive performance, demonstrating that corpus-derived lexical knowledge captures substantial affective information in Moroccan Darija. To further improve performance, we propose a unified hybrid framework that combines TriLex-Darija features with contextual embeddings extracted from MARBERT. All models are trained using a consistent LinearSVC classifier to ensure fair comparison and reproducibility. In addition to the symbolic model, we evaluate a lexicon-feature-based LinearSVC model, allowing a clear distinction between symbolic, feature-based, and hybrid approaches. Experimental results show that the hybrid model consistently outperforms both BERT-only and lexicon-feature-based baselines across all tasks. For sentiment classification, the hybrid model achieves a macro F1-score of 72.96% , compared to 59.15% for BERT-only and 67.94% for the lexicon-feature-based model. For emotion classification, it reaches 92.92% , outperforming BERT-only ( 79.17% ) and lexicon-feature-based ( 89.55% ) models. For pragmatic context classification, the hybrid model achieves 91.35% , compared to 75.93% for BERT-only and 87.72% for the lexicon-feature-based model. Bootstrap confidence intervals (95%) and McNemar’s tests confirm that all improvements are statistically significant ( p < 0.001). Overall, TriLex-Darija demonstrates that combining lexical knowledge with contextual embeddings leads to robust, interpretable, and statistically validated affective models for Moroccan Darija in low-resource settings.
The Arabic language presents unique modeling challenges, such as its morphological complexity, orthographic ambiguity, dialectal variations, and orthographic noise. Furthermore, the scarcity of linguistic resources dedicated to Arabic and the limited availability of detection tools compound the difficulties in effectively identifying and mitigating offensive language in Arabic text. Given Arabic's linguistic richness and diversity, it is essential to develop robust and accurate methods for detecting and addressing offensive language to ensure the safety and well-being of Arabic-speaking online communities. This study aims to enhance the performance of Arabic offensive tweet classification by proposing a novel framework combining advanced preprocessing techniques and state-of-the-art classification models in an ensemble methodology. It comprises two primary modules: preprocessing module and classification module. The preprocessing module incorporates AraBERT preprocessing, emoji-to-word interpretation, and punctuation removal. The classification module encompasses neural networks (NN), LSTM, and AraBERT. The proposed ensemble model combines a fine-tuned AraBERT with two NN layers, LSTM, ReLU, and Sigmoid activation functions. The model achieved state-of-the-art performance, surpassing all other approaches in terms of Accuracy and F1-score. These results highlight the effectiveness and potential of leveraging advanced models like AraBERT in detecting and classifying offensive language in Arabic text.
‘‘Is a Large Language Model (LLM) a good liar? Can we detect its lies for Turkish?” LLMs are capable of generating deceptive or misleading information, which can be considered as “lying”. In this article, we introduce a dataset called TQuADFake , consisting of correct and incorrect sentences generated by LLMs through the Question Answering (QA) task. We evaluate the quality of the dataset using both human and LLM evaluators. We explore linguistic features that may help distinguish incorrect sentences and build a lie detection model based on these features. Our analysis shows that while linguistic features are useful for identifying differences between correct and incorrect sentences, these features alone are insufficient to fully solve the lie detection problem. We also investigate whether LLMs are aware of their own inaccuracies and compare various models’ performance on this task, including SVM, pre-trained language models (PLMs), and LLMs. This study provides a foundation for future research on misinformation detection in Turkish and highlights the challenges in building effective lie detection systems. 1
Named Entity Recognition (NER) in Arabic mental-health text is constrained by the scarcity of gold annotations and dialectal variation. Identifying medical entities (e.g., Symptom, Diagnosis, drug, Dosage) in low-resource context is crucial for clinical triage and mental health safety. To address this gap, we propose and evaluate end-to-end framework for NER in Arabic patient-generated mental health questions. Our framework leverages state-of-the-art Large Language Models (LLMs), including GPT-4o, LLaMA, and ALLaM, utilizing Persona–Template prompting strategy, including zero-shot vs. few-shot tested across the models. We applied a clinically grounded nine-entity schema to extract structured information from mental health questions. We adopt a hybrid evaluation strategy that combines LLM-as-a-Judge scoring for scalability with targeted validation on a manually curated drug subset, compensating for the limited availability of fully gold-annotated data. Results show that few-shot prompting improves reliability across models. GPT-4o achieves the strongest drug test set accuracy (F1: 0.86→0.94), LLaMA remains stable and competitive (0.89→0.88), and Arabic-focused ALLaM shows the largest relative gain (0.58→0.73). This work not only advances Arabic NLP for mental health applications but also offers a reproducible framework for low-resource NER, demonstrating the feasibility of LLM-driven medical entity extraction in critical clinical contexts.
Real-time rumor detection methods that do not rely on propagation features have emerged as an effective strategy to curb the spread of misinformation. To address the pressing challenge of enhancing semantic understanding of short texts and extracting latent user features in real-time rumor detection, this article proposes a novel approach that integrates B ackground K nowledge and U ser F eatures (BKUF). First, relevant background knowledge is extracted from an external knowledge graph through knowledge distillation. To accommodate different granularities of knowledge, we design two fusion strategies: one based on graph attention networks and the other on co-attention mechanisms, effectively enriching the semantic representation of the text. In addition to traditional user features, we further introduce two novel latent user attributes—rationality and professionalism—which are inferred from users’ historical posts. Finally, the enhanced semantic and user features are adaptively integrated and passed into a multi-layer perceptron for classification. Experiments conducted on four widely used public rumor datasets—Weibo, PHEME, Twitter15, and Twitter16—show that our method achieves accuracies of 92.8%, 84.9%, 81.5%, and 82.7%, respectively, outperforming state-of-the-art baselines.
In today’s technologically advanced society, online services are rapidly expanding, with a growing emphasis on customer satisfaction. To enhance the value of cloud services for users, it is essential to provide relevant and authentic recommendations. To address this requirement, our model DeepHaB-MMF integrates an automated recommendation system with advanced contextual and sequential embedding, designed to handle multilingual inputs. The first phase of our model is DeepHaB, which processes the user reviews by generating embedding that combine contextual information with extracted features. These embedding are then passed through a deep BiLSTM network to capture the bidirectional dependencies in the data. An attention mechanism further enhances the process by highlighting the most informative features which further help in classifying the truthful or fake reviews. These results are then provided with other extracted features to the next phase of the model, i.e., DeepMMF which modifies the traditional matrix factorization technique to provide relevant recommendations. Thus our model, DeepHaB-MMF first filters out the fake review and based on only truthful reviews it provides authentic as well as relevant recommendations. This model is evaluated on three low resource languages like Hindi, Marathi and Bengali and the results clearly shows that it out performs other state-of-the-art approaches.
This study addresses the challenges of end-to-end (E2E) Speech-to-Text Translation (STT) for the low-resource Fongbe-to-French language pair using a transfer learning approach. We first establish robust baselines by integrating state-of-the-art pretrained speech encoders (HuBERT-147, AfriHuBERT, XLS-R, Whisper) with powerful text decoders (mBART, NLLB). This initial phase identified the AfriHuBERT–NLLB and XLS-R–NLLB combinations as the most competitive E2E configurations. To further enhance performance, we propose and evaluate three hybrid feature fusion strategies, the Bidirectional Co-Attention (BCOAT), the Feature-wise Linear Modulation (FiLM), and the Feature Sum (SUM). These methods are designed to strategically fuse intermediate representations extracted from two distinct and powerful encoders, AfriHuBERT and XLS-R, within the E2E architecture. The fusion process aims to enrich the linguistic and tonal information critical for accurate translation of the tonal Fongbe language. Experimental results demonstrate significant performance gains over the baselines. The BLEU score improved from 26.32 to a peak of 27.78 for the AfriHuBERT–NLLB configuration (using FiLM), and from 26.27 to a maximum of 28.05 for the XLS-R–NLLB configuration (using SUM). These findings confirm that translation quality for tonal languages like Fongbe can be substantially improved by extracting and combining high-quality, complementary features through advanced encoder fusion. Our hybrid feature fusion methods present a substantial advance in speech translation quality within resource-scarce linguistic environments.
Spell-checking, including misspelling detection and correction, is a classic problem in the natural language processing community. Most common soft spelling errors are typographical; they occur due to orthographic variations of some Arabic letters, given the identical phonetic sounds. This study aims at experimenting and applying a recent state-of-the-art attention-based transformer deep learning model with neural machine translation seq-to-seq loss on a Modern Standard Arabic spell-checking task. We used OpenNMT, an open-source neural network library, to train the Bidirectional Encoder Representations from the Transformers model and the Bidirectional Long short-term memory model as a baseline model for detecting and correcting soft spelling errors in Arabic. The seq-to-seq model converts corrupted text (input sequence) into clean, error-free text (output sequence). The synthetic dataset is generated from the “SCUT corpus Version 3” dataset, where we created and applied a random noise injection confusion function. This process involved substituting characters in the text at random positions to simulate spelling errors. The intention was to mimic common human typing or transcription errors, including typographical errors and cognitive misspellings. The corruption ratio injected into the data and the length of the input sequence were considered when assessing the models’ performance. The trained models’ results in terms of Accuracy and Bilingual Evaluation Understudy Score were promising and competitive compared to other solutions.
