
The rapid advancement of large language models such as GPT, Jais, and Llama has increasingly blurred the boundary between human and machine-generated text, creating significant challenges for ensuring the authenticity and reliability of digital content. Although research on AI-generated text detection has expanded considerably, Arabic remains insufficiently studied due to its morphological complexity and dialectal variation, which present additional challenges for reliable detection. This study presents a robust and generalizable framework for Arabic AI-text detection across multiple domains and text-generating large language models. We propose a two-level stacked ensemble that integrates transformer-based models, including AraBERT, AraELECTRA, XLM-RoBERTa, and DeBERTa, and stacks their outputs using meta-classifiers such as Logistic Regression, SVM, Random Forest, and XGBoost. The framework is evaluated on both formal academic abstracts and informal social media texts, covering outputs generated by diverse large language models such as ALLaM, Jais, GPT-4, and Llama. Our results show that fine-tuned Arabic models outperform multilingual counterparts, while models trained on informal text exhibit stronger cross-domain generalization. Furthermore, the choice of generative model impacts robustness, with Arabic-specific LLMs such as ALLaM and Jais achieving superior cross-domain and cross-model performance. Our findings provide valuable insights for developing robust AI-generated Arabic text detection systems adaptable across diverse domains.
We present Evalita-LLM, a novel benchmark designed to evaluate Large Language Models (LLMs) on Italian tasks. The key innovation of Evalita-LLM is the use of multiple prompts for each task, which mitigates the model’s sensitivity to specific prompt formulations and ensures a robust and objective performance assessment. We propose an iterative methodology for selecting and validating multiple prompts, and metrics that capture both high performance and stability of a prompt given a set of LLMs. The resulting Evalita-LLM multi-prompt benchmark is based on ten native Italian tasks, eliminating translation issues and potential cultural biases. It includes a mix of multiple-choice and generative tasks, enabling more natural interactions with LLMs. We report experimental results from the benchmark’s development phase, along with performance statistics for several state-of-the-art LLMs. Evalita-LLM is publicly available with a leaderboard hosted on Hugging Face to support transparent evaluation and comparison of LLMs.
Animacy determines grammatical choices in many languages worldwide, including Ancient Greek. In order to be able to study animacy on a large scale, this paper describes an annotation project of animacy for the GLAUx corpus (Keersmaekers, 2021). This consists of a manual annotation project on a small part of GLAUx and training automatic classifiers to fully annotate the corpus. The manual annotation is based on a modified version of the annotation scheme of Zaenen et al. (2004). For the automated annotation six different transformer models and two classification methods are compared: finetuning transformer models for animacy or extracting embeddings from them as features for a Multi-Layer Perceptron (MLP), with or without additional features. For transformers pretrained on less Ancient Greek data the feature-based strategy is clearly superior to finetuning, while both strategies perform roughly equally well for the transformer pretrained on the largest Ancient Greek dataset. With both strategies the best models’ accuracies ranged from 0.89 to 0.90, allowing us to perform relatively reliable queries on animacy in a large Ancient Greek corpus. Typical classification errors involve issues where the general semantics of the target word is not grasped well, polysemy, and problems in defining the boundaries of the different animacy classes. The results of this study may be generalizable to other low-resource languages as well.
Traditional linguistic atlases can be leveraged to investigate the dynamics of language diversification and change but are severely underexploited for this purpose. Here we present a cleaned up and augmented dataset derived from the AIS (Swiss-Italian Atlas) which allows us to explore the amount of innovation at different linguistic levels (lexicon vs. phonology), and the geographic, societal, and evolutionary factors predicting the amount of change in different varieties. Results point towards a significant correlation in the speed of lexical and phonological change, as well as the greater relevance of extralinguistic variables (most importantly population size) to predict lexical innovation.
Repeated references to a discourse entity that has already occurred in a text are generally established through the pronouns called anaphors. Resolution of such anaphors to their respective antecedent (discourse entity) is known as Anaphora Resolution (AR). Examining how these pronouns are resolved unlocks a deeper understanding of language mechanics and assists in meaning construction. Considering the diverse linguistic facets of a text, this paper proposes a hybrid model for automatic AR in Hindi texts. A systematic approach for identifying the antecedent of an anaphor is presented in this work. The proposed model combines rule-based and machine learning approaches. It uses a rule-based approach for the resolution of reflexive and locative pronouns and supervised machine learning classifiers for resolving demonstrative pronouns and relative pronouns. The supervised classifiers investigated in this work are Extreme Gradient Boost (XGBOOST) and Distributed Random Forest (RF). The work further employed a novel heuristic: Class Distribution Aware Classification Imbalance Factor (CDACIF) for handling the data imbalance problem for the machine learning based AR module for the Hindi discourse. Experiments are conducted on publicly available Hindi tourism and Hindi Dependency Tree Bank data (HDTB). Analysis of the results shows that ensemble classifiers performed better than existing proposed classifiers. An improvement of 3.91