Legal judgment prediction (LJP) systems support legal professionals by forecasting case outcomes, aiding in legal preparation and strategy. The development of such systems has been driven largely by benchmark datasets that support the training and evaluation of machine learning (ML) models. However, most existing datasets only contain final judgment texts that include facts written after the court hearing, which are unavailable to practitioners during the pre-hearing phase, limiting the real-world applicability of models trained on them. Additionally, benchmark datasets for Supreme Courts remain scarce, with few notable exceptions. To address both issues, we introduce the first LJP benchmark dataset for the United Kingdom Supreme Court: UKSC-JP, covering 821 cases and supporting two subtasks: (i) legal judgment classification, and (ii) court view generation. Each UKSC case includes a press summary that provides background and does not contain all adjudicated facts, offering a more realistic input for LJP systems. We evaluate several ML models, including large language models (LLMs), across both subtasks. Our results show that, despite recent advances, LJP remains a challenging task, especially when deprived of post-hearing content. We release our code and data resources publicly available at: https://github.com/DamithDR/uksc .

Large language models have become the dominant paradigm in Natural Language Processing, yet their explainability remains fundamentally limited, making automatic error detection challenging. This study conducts a systematic comparison of explainability signatures between discrete graph-based and continuous token-based architectures for error pattern recognition in text classification. Using text-to-graph conversion, LLM-as-teacher distillation, and unified Shapley-based attribution, we identify four complementary architectural patterns that enable reliable discrimination between correct and incorrect predictions: AUC-based confidence trajectories, feature concentration profiles, margin-preservation patterns, and fidelity-asymmetry signatures.We demonstrate that hierarchical graph-based architectures produce substantially sharper discriminative patterns (99.7--100.0\% error detection accuracy) compared to token-based architectures (88.1--89.6\% accuracy) under fair comparison. This performance gap persists across datasets, graph topologies, and prediction correctness conditions. Integrated logistic regression analysis reveals that all architectures rely on necessity-based features, but hierarchical discrete structures produce significantly stronger necessity signals through binary node participation, while continuous token embeddings produce attenuated signals through distributed compensatory pathways.For high-stakes applications requiring transparent automated error detection, hierarchical graph-based explainability patterns offer clear advantages over token-based approaches. Our four-dimensional evaluation framework provides a reproducible methodology for architectural signature analysis in explainability research.

In common law systems, legal professionals such as lawyers and judges rely on precedents to build their arguments. As the volume of cases has grown massively over time, effectively retrieving prior cases has become essential. Prior case retrieval (PCR) is an information retrieval (IR) task that aims to automatically identify the most relevant court cases for a specific query from a large pool of potential candidates. While IR methods have seen several paradigm shifts over the last few years, the vast majority of PCR methods continue to rely on traditional IR methods, such as BM25. The state-of-the-art deep learning IR methods have not been successful in PCR due to two key challenges: i. Lengthy legal text limitation; when using the powerful BERT-based transformer models, there is a limit of input text lengths, which inevitably requires to shorten the input via truncation or division with a loss of legal context information. ii. Lack of legal training data; due to data privacy concerns, available PCR datasets are often limited in size, making it difficult to train deep learning-based models effectively. In this research, we address these challenges by leveraging LLM-based text embedders in PCR. LLM-based embedders support longer input lengths, and since we use them in an unsupervised manner, they do not require training data, addressing both challenges simultaneously. In this paper, we evaluate state-of-the-art LLM-based text embedders in four PCR benchmark datasets and show that they outperform BM25 and supervised transformer-based models.

The first Workshop on Language Models for Low-Resource Languages (LoResLM 2025) was held in conjunction with the 31st International Conference on Computational Linguistics (COLING 2025) in Abu Dhabi, United Arab Emirates. This workshop mainly aimed to provide a forum for researchers to share and discuss their ongoing work on language models (LMs) focusing on low-resource languages, following the recent advancements in neural language models and their linguistic biases towards high-resource languages. LoResLM 2025 attracted notable interest from the natural language processing (NLP) community, resulting in 35 accepted papers from 52 submissions. These contributions cover a broad range of low-resource languages from eight language families and 13 diverse research areas, paving the way for future possibilities and promoting linguistic inclusivity in NLP.

Experts in machine learning leverage domain knowledge to navigate decisions in model selection, hyperparameter optimization, and resource allocation. This is particularly critical for fine-tuning language models (LMs), where repeated trials incur substantial computational overhead and environmental impact. However, no existing automated framework simultaneously tackles the entire model selection and hyperparameter optimization (HPO) task for resource-efficient LM fine-tuning. We introduce XAutoLM, a meta-learning-augmented AutoML framework that reuses past experiences to optimize discriminative and generative LM fine-tuning pipelines efficiently. XAutoLM learns from stored successes and failures by extracting task- and system-level meta-features to bias its sampling toward valuable configurations and away from costly dead ends. On four text classification and two question-answering benchmarks, XAutoLM surpasses zero-shot optimizer’s peak F1 on five of six tasks, cuts mean evaluation time of pipelines by up to 4.5x, reduces search error ratios by up to sevenfold, and uncovers up to 50% more pipelines above the zero-shot Pareto front. In contrast, simpler memory-based baselines suffer negative transfer. We release XAutoLM and our experience store to catalyze resource-efficient, Green AI fine-tuning in the NLP community.

Distinguishing between human-and machine-produced text is crucial for tasks like authorship verification, content moderation, and quality assessment. We introduce IBERMAT, a novel dataset of human and machine translations across three specialised domains (clinical, legal and literary) and four official languages in Spain (Basque, Catalan, Galician and Spanish) and outlines a case study of its exploitation. We evaluate the performance of classifying translation origin using a range of machine learning techniques. We evaluate three approaches: (1) traditional machine learning pipelines, (2) fine-tuned transformer-based language models using full and low-rank adaptation strategies, and (3) LLMs for zero-shot classification. The results show that fine-tuned transformers outperform both traditional ML and zero-shot LLMs, but not with substantial differences. These results highlight both the increasing quality of MT output and the limitations of current models in detecting subtle distinctions, especially when translations may involve post-editing. Our findings also suggest that machine-translated content may be harder to identify than general AI-generated text.

Predicting semantic textual similarity (STS) is a complex and ongoing challenge in natural language processing (NLP). Over the years, researchers have developed a variety of supervised and unsupervised approaches to calculate STS automatically. Additionally, various benchmarks, which include STS datasets, have been established to consistently evaluate and compare these STS methods. However, they largely focus on high-resource languages, mixed with datasets annotated focusing on relatedness instead of similarity and containing automatically translated instances. Therefore, no dedicated benchmark for multilingual STS exists. To solve this gap, we introduce the Multilingual Semantic Textual Similarity Benchmark (MUSTS), which spans 13 languages, including low-resource languages. By evaluating more than 25 models on MUSTS, we establish the most comprehensive benchmark of multilingual STS methods. Our findings confirm that STS remains a challenging task, particularly for low-resource languages.

Recently, language models (LMs) have produced excellent results in many natural language processing (NLP) tasks. However, their effectiveness is highly dependent on available pre-training resources, which is particularly challenging for low-resource languages such as Sinhala. Furthermore, the scarcity of benchmarks to evaluate LMs is also a major concern for low-resource languages. In this paper, we address these two challenges for Sinhala by (i) collecting the largest monolingual corpus for Sinhala, (ii) training multiple LMs on this corpus and (iii) compiling the first Sinhala NLP benchmark (Sinhala-GLUE) and evaluating LMs on it. We show the Sinhala LMs trained in this paper outperform the popular multilingual LMs, such as XLM-R and existing Sinhala LMs in downstream NLP tasks. All the trained LMs are publicly available. We also make Sinhala-GLUE publicly available as a public leaderboard, and we hope that it will enable further advancements in developing and evaluating LMs for Sinhala.

This article presents an approach to readability estimation that focuses on conceptual rather than linguistic complexity, using the extensive SaudiTextBooks textbooks. We introduce DARES 2.0, an enhanced concept-based readability training dataset designed to estimate the readability of Saudi educational texts. Building on DARES 1.0, DARES 2.0 extends the scope of conceptual complexity by replacing repetitive concepts and manually revising the input features with unique terms and their surrounding contexts from the SaudiTextBooks, spanning grades 1 to 12. The refined DARES 2.0 is employed to fine-tune pre-trained transformer models, including XLM-R Base, mBERT, AraELECTRA, AraBERTv2, and CAMeLBERTmix. The findings suggest that both the dataset and experimental setup require further development to ensure a larger, higher-quality dataset and to support more extensive fine-tuning experiments, in addition to exploring transfer learning from other languages and enhancing the diversity and richness of Arabic concepts. These developments pave the way for further advancements in concept-based readability estimation in educational contexts in future work.

Researchers have relegated natural language processing tasks to Transformer-type models, particularly generative models, because these models exhibit high versatility when performing generation and classification tasks. As the size of these models increases, they achieve outstanding results. Given their widespread use, many explainability techniques are developed based on these models. However, this process becomes computationally expensive due to the large size of the models. Additionally, transformers interpret input information through tokens that fragment input words into sequences lacking inherent semantic meaning, complicating the explanation of the model from the very beginning. This study proposes a novel methodology to achieve explainability in natural language processing tasks by automatically converting sentences into graphs and maintaining semantics through nodes and relations that express fundamental linguistic concepts. It also allows the subsequent exploitation of this knowledge in subsequent tasks, making it possible to obtain trends and understand how the model associates the different elements inside the text with the explained task. The experiments delivered promising results in determining the most critical components within the text structure for a given classification.

The goal of information extraction is to extract structural knowledge (such as entities, relations and events) from plain and unstructured texts. Information extraction in legal documents has recently gained a lot of attention in the natural language processing (NLP) community due to the high demand for efficient information extraction for legal practitioners and companies. Given that the legal documents are unique and their processing is challenging, there is a pressing need for applications of NLP techniques to tackle these challenges. In this research, we present a survey on the recent advancements in legal information extraction focusing on three tasks: named entity recognition, relationship extraction and event detection. We report language resources and systems in multiple jurisdictions and languages for each task. Based on the thorough review conducted, we identify insights into the techniques employed and promising research directions that merit further exploration in future studies. We maintain a public repository and consistently update related resources at https://github.com/DamithDR/legalinformationextraction .

The hospitality industry in the Arab world increasingly relies on customer feedback to shape services, driving the need for advanced Arabic sentiment analysis tools. To address this challenge, the Sentiment Analysis on Arabic Dialects in the Hospitality Domain shared task focuses on Sentiment Detection in Arabic Dialects. This task leverages a multi-dialect, manually curated dataset derived from hotel reviews originally written in Modern Standard Arabic (MSA) and translated into Saudi and Moroccan (Darija) dialects. The dataset consists of 538 sentiment-balanced reviews spanning positive, neutral, and negative categories. Translations were validated by native speakers to ensure dialectal accuracy and sentiment preservation. This resource supports the development of dialect-aware NLP systems for real-world applications in customer experience analysis. More than 40 teams have registered for the shared task, with 12 submitting systems during the evaluation phase. The top-performing system achieved an F1 score of 0.81, demonstrating the feasibility and ongoing challenges of sentiment analysis across Arabic dialects.

Definition modelling (DM) is the task of automatically generating a dictionary definition for a specific word. Computational systems that are capable of DM can have numerous applications benefiting a wide range of audiences. As DM is considered a supervised natural language generation problem, these systems require large annotated datasets to train the machine learning (ML) models. Several DM datasets have been released for English and other high-resource languages. While Portuguese is considered a mid/high-resource language in most natural language processing tasks and is spoken by more than 200 million native speakers, there is no DM dataset available for Portuguese. In this research, we fill this gap by introducing DORE; the first dataset for Definition MOdelling for PoRtuguEse containing more than 100,000 definitions. We also evaluate several deep learning based DM models on DORE and report the results. The dataset and the findings of this paper will facilitate research and study of Portuguese in wider contexts.

Idioms present a formidable challenge for machine translation (MT) due to their figurative, culture-specific, and linguistic complexity. In this study, we compiled a corpus of 100 Chinese idioms from the Dictionary of Chinese Idioms and conducted quantitative analyses of nine state-of-the-art MT systems. Recognising the linguistic complexity of idioms, we introduced AIE, a new evaluation metric for translations, derived from its three assessment criteria: Accuracy, Intelligibility, and Elegance. In this framework, we suggest assigning distinct weights to its metrics, supported by empirical evidence. Additionally, we employed automatic metrics ROUGE, BLEU, BLEURT, and METEOR, to assess translation quality. Our analysis revealed that while BLEURT and BLEU exhibited stronger correlations with human scores, the overall correlation remained weak. Furthermore, recognising the significance of automatic evaluation in natural language processing (NLP), we hypothesised that combining existing automatic metrics could yield improved assessment scores compared to individual metrics. To validate this hypothesis, we computed average scores of automatic metrics, which demonstrated a positive correlation with human scores, suggesting a promising alternative. Our findings indicate that GPT4 and GLM4 outperform other state-of-the-art models even in translating less commonly used idioms. Keywords: Chinese idioms · Machine translation · Large language model.

Given the increase in production of data for the biomedical field and the unstoppable growth of the internet, the need for Information Extraction (IE) techniques has skyrocketed. Named Entity Recognition (NER) is one of such IE tasks useful for professionals in different areas. There are several settings where biomedical NER is needed, for instance, extraction and analysis of biomedical literature, relation extraction, organisation of biomedical documents, and knowledge-base completion. However, the computational treatment of entities in the biomedical domain has faced a number of challenges including its high cost of annotation, ambiguity, and lack of biomedical NER datasets in languages other than English. These difficulties have hampered data development, affecting both the domain itself and its multilingual coverage. The purpose of this study is to overcome the scarcity of biomedical data for NER in Spanish, for which only two datasets exist, by developing a robust bilingual NER model. Inspired by back-translation, this paper leverages the progress in Neural Machine Translation (NMT) to create a synthetic version of the Colorado Richly Annotated Full-Text (CRAFT) dataset in Spanish. Additionally, a new CRAFT dataset is constructed by replacing 20% of the entities in the original dataset generating a new augmented dataset. We evaluate two training methods: concatenation of datasets and continuous training to assess the transfer learning capabilities of transformers using the newly obtained datasets. The best performing NER system in the development set achieved an F-1 score of 86.39%. The novel methodology proposed in this paper presents the first bilingual NER system and it has the potential to improve applications across under-resourced languages.

In this paper, we evaluate and compare new variants of a popular rule-based anaphora resolution algorithm with the original version. We seek to establish data (always) outperform the original rule-based algorithm. The results of this study suggest that while algorithms based in Deep Learning and LLMs usually perform better than rule-based ones, this is not always the case, and we argue that rule-based approaches still have a place in today's research.