Entity Resolution (ER) is a fundamental problem in data management, playing a critical role in tasks like data cleaning and knowledge graph construction. The existing ER approaches range from traditional rule-based to deep learning techniques and LLM-based methods, but typically operate under a “passive paradigm”, as duplicates are detected through static, one-shot similarity computations. Such approaches fail to capture the inherently uncertain and context-dependent nature of real-world ER tasks, especially in data lakes with streaming content in heterogeneous formats such as CSV files, JSON files, RDF dumps, and free text. In such settings, resolving ambiguity often requires iterative evidence gathering, reasoning across multiple sources, even selective human involvement. To cover this gap, we advocate a paradigm shift from passive to Agentic ER, which frames ER as a sequential decision-making process that is performed by autonomous agents. These agents actively plan ER strategies, acquire external evidence, decide when to query additional sources or humans, and optimize trade-offs between accuracy, cost, and latency. We formalize Agentic ER as a decision-theoretic problem, we propose a reference architecture, we identify core research challenges, and outline new evaluation dimensions tailored to agentic behavior. By introducing Agentic ER, we aim to establish a new research direction at the intersection of data management and intelligent agents.

Entity Resolution (ER) constitutes a fundamental task in data management, involving the identification and linking of entities that refer to the same real-world object across different datasets. It is typically addressed through the Filtering-Verification framework, where the first step (also called Blocking) reduces the computational cost to the most similar pairs of entities, which are then analytically compared by the second step (also called Entity Matching). The latest advancements in Verification leverage Large Language Models (LLMs), which operate in an unsupervised manner and require no labelled training data. However, LLM-based matching is typically addressed as a classification task, independently of Filtering and without examining conflicts among the detected matches. Moreover, the substantial computational requirements of LLMs often limit their practical applicability in Entity Matching.In this work, we propose an open-source system called ChatMatcher, that implements end-to-end ER pipelines combining LLM-based Matching with Filtering and Entity Clustering. ChatMatcher provides a design space for matching prompts by combining different example orderings in few-shot prompts and several example selection strategies. To ensure that ChatMatcher runs on commodity hardware, it exclusively supports 7B-parameter LLMs with 4-bit quantization for Entity Matching. The performance of the resulting ER pipelines is evaluated on eight established ER datasets, with the experiments examining the relative performance of numerous configurations. In fact, our experimental study is the first to analyse how the precision–recall trade-off induced by Filtering affects LLM-based matching performance, and to demonstrate that Entity Clustering is essential to reconcile the inconsistent predictions produced by LLMs. The experimental results demonstrate that ChatMatcher can achieve state-of-the-art ER performance with resource-constrained models, despite hardware limitations.

Recent advances in Entity Resolution (ER) have leveraged Large Language Models (LLMs), achieving strong performance but at the cost of substantial computational resources or high financial overhead. Existing LLM-based ER approaches operate either in unsupervised settings and rely on very large and costly models, or in supervised settings and require ground-truth annotations, leaving a critical gap between time efficiency and effectiveness. To make LLM-powered ER more practical, we investigate Knowledge Distillation (KD) as a means to transfer knowledge from large, effective models (Teachers) to smaller, more efficient models (Students) without requiring gold labels. We introduce DistillER, the first framework that systematically bridges this gap across three dimensions: (i) Data Selection, where we study strategies for identifying informative subsets of data; (ii) Knowledge Elicitation, where we compare single- and multi-teacher settings across LLMs and smaller language models (SLMs); and (iii) Distillation Algorithms, where we evaluate supervised fine-tuning and reinforcement learning approaches. Our experiments reveal that supervised fine-tuning of Students on noisy labels generated by LLM Teachers consistently outperforms alternative KD strategies, while also enabling high-quality explanation generation. Finally, we benchmark DistillER against established supervised and unsupervised ER methods based on LLMs and SLMs, demonstrating significant improvements in both effectiveness and efficiency.

In our primary work (Gagliardelli et al., 2024), we presented Generalized Supervised Meta-blocking, a novel approach that is based on the task of associating every pair of candidates with a probabilistic estimation of their matching likelihood. It formulates Meta-blocking as a probabilistic binary classification task, using a wide variety of weighting schemes as features. The resulting probability estimations can be combined with any pruning algorithm that improves blocking by retaining the best candidate comparisons, drastically reducing false positives without any significant impact on true positives. They can also be used by state-of-the-art progressive Entity Resolution methods to identify the most promising candidates as early as possible. We experimentally identified the best pruning algorithms, their optimal sets of features, and the minimum possible size of the training set. Our experiments demonstrate that the resulting approaches achieve excellent performance in several established benchmark datasets. This work introduces a companion reproducible paper of our previous work (Gagliardelli et al., 2024) to describe how to reproduce the entire experimental study and discuss how to extend it with new features, classification algorithms, and datasets. The presented reproducibility methodology is based on a Docker image and the NVIDIA CUDA Toolkit 12.0, which are evaluated on several Linux-based platforms equipped with at least 128 GB RAM, a GPU and at least 300 GB of free disk space. The reproducibility methodology leads to a weak reproducibility, with minor deviations from the experimental results reported in our primary work (Gagliardelli et al., 2024).

Entity Resolution (ER) constitutes a challenging data integration task that is typically addressed through the Filtering-Verification framework. Filtering reduces the quadratic search space in an unsupervised manner that relies on heuristics, whereas verification performs matching, usually through a machine or a deep learning-based approach. Numerous solutions have been proposed for each step, but analyzing their combined performance constitutes a non-trivial task, due to technical and methodological challenges, while the literature typically examines them as orthogonal tasks. We facilitate the benchmarking of state-of-the-art verification algorithms under realistic settings, applying them to the candidate pairs generated by established filtering approaches from popular real-world datasets. To democratize this benchmarking, we developed an open-source, hands-off Web application, called SMBench, which allows users to perform a wealth of experiments through an intuitive user interface that requires no coding or ER expertise. SMBench is publicly available at https://smbench.kbs.uni-hannover.de, while its code is released through https://github.com/erbench/erbench. We delve into its frontend and backend, elaborating on the technologies used for their implementation as well as on the state-of-the-art ER methods they support. Using SMBench, we perform an extended experimental analysis that combines 3 filtering methods with 7 verification approaches, applying them to 9 datasets. The experimental results lead to interesting insights into the relative effectiveness, time and memory efficiency of the considered methods.

Entity Resolution (ER) is typically implemented as a batch task that processes all available data before identifying duplicate records. However, applications with time or computational constraints, e.g., those running in the cloud, require a progressive approach that produces results in a pay-as-you-go fashion. Numerous algorithms have been proposed for Progressive ER in the literature. In this work, we propose a novel framework for Progressive Entity Matching that organizes relevant techniques into four consecutive steps: (i) filtering, which reduces the search space to the most likely candidate matches, (ii) weighting, which associates every pair of candidate matches with a similarity score, (iii) scheduling, which prioritizes the execution of the candidate matches so that the real duplicates precede the non-matching pairs, and (iv) matching, which applies a complex, matching function to the pairs in the order defined by the previous step. We associate each step with existing and novel techniques, illustrating that our framework overall generates a superset of the main existing works in the field. We select the most representative combinations resulting from our framework and fine-tune them over 10 established datasets for Record Linkage and 8 for Deduplication, with our results indicating that our taxonomy yields a wide range of high performing progressive techniques both in terms of effectiveness and time efficiency.

The same real-world entity (e.g., a movie, a restaurant, a person) may be described in various ways on different datasets. Entity Resolution (ER) aims to find such different descriptions of the same entity, this way improving data quality and, therefore, data value. However, an ER pipeline typically involves several steps (e.g., blocking, similarity estimation, clustering), with each step requiring its own configurations and tuning. The choice of the best configuration, among a vast number of possible combinations, is a dataset-specific and labor-intensive task both for novice and expert users, while it often requires some ground truth knowledge of real matches. In this work, we examine ways of automatically configuring a state of-the-art end-to-end ER pipeline based on pre-trained language models under two settings: (i) When ground truth is available. In this case, sampling strategies that are typically used for hyperparameter optimization can significantly restrict the search of the configuration space. We experimentally compare their relative effectiveness and time efficiency, applying them to ER pipelines for the first time. (ii) When no ground truth is available. In this case, labelled data extracted from other datasets with available ground truth can be used to train a regression model that predicts the relative effectiveness of parameter configurations. Experimenting with 11 ER benchmark datasets, we evaluate the relative performance of existing techniques that address each problem, but have not been applied to ER before.

Link Discovery plays a vital role in enhancing connections between data sources within the Linked Open Data Cloud. A key aspect of this process is Entity Resolution (ER), which focuses on identifying owl:sameAs relationships between different entity descriptions representing the same real-world object. Recent works on ER have investigated the use of Large Language Models (LLMs) for Entity Matching, showing promising results. In the simplest case, a pair of entities is given as input to an LLM, asking whether these entities match or not. A recent approach introduced SELECT prompts, which include the query entity along with multiple candidates generated by a Blocking method. However, this increases the complexity of the questions posed to LLMs, while being susceptible to position bias among the presented candidates. To address these issues, we introduce AvengER, a novel approach to SELECT prompts for LLM-based Matching that effectively handles both unsupervised and supervised settings. For the former, we introduce a hybrid approach that utilizes an ensemble of open-source medium-size models (8b) and also selectively leverages (in just 12

Entity Resolution (ER) is typically implemented as a batch task that processes all available data before identifying duplicate records. However, applications with time or computational constraints, e.g., those running in the cloud, require a progressive approach that produces results in a pay-as-you-go fashion. Numerous algorithms have been proposed for Progressive ER in the literature. In this work, we propose a novel framework for Progressive Entity Resolution that organizes relevant techniques into four consecutive steps: (i) filtering, which reduces the search space to the most likely candidate matches, (ii) weighting, which associates every pair of candidate matches with a similarity score, (iii) scheduling, which prioritizes the execution of the candidate matches so that the real duplicates precede the non-matching pairs, and (iv) matching, which applies a complex, matching function to the pairs in the order defined by the previous step. We associate each step with existing and novel techniques, illustrating that our framework overall generates a superset of the main existing works in the field. We select the most representative combinations resulting from our framework and fine-tune them over 10 established datasets for Record Linkage and 8 for Deduplication, with our results indicating that our taxonomy yields a wide range of high performing progressive techniques both in terms of effectiveness and time efficiency.

Entity Resolution (ER) is a critical data cleaning task for identifying records that refer to the same real-world entity. In the era of Big Data, traditional batch ER is often infeasible due to volume and velocity constraints, necessitating Progressive ER methods that maximize recall within a limited computational budget. However, existing progressive approaches fail to scale to high-velocity streams because they rely on deterministic sorting to prioritize candidate pairs, a process that incurs prohibitive super-linear complexity and heavy initialization costs. To address this scalability wall, we introduce SPER (Stochastic Progressive ER), a novel framework that redefines prioritization as a sampling problem rather than a ranking problem. By replacing global sorting with a continuous stochastic bipartite maximization strategy, SPER acts as a probabilistic high-pass filter that selects high-utility pairs in strictly linear time. Extensive experiments on eight real-world datasets demonstrate that SPER achieves significant speedups (3x to >6x) over state-of-the-art baselines while maintaining comparable recall and precision.

Entity Resolution constitutes a core data integration task that has attracted a bulk of works on improving its effectiveness and time efficiency. This tutorial provides a comprehensive overview of the field, distinguishing relevant methods into five main generations. The first one targets Veracity in the context of structured data with a clean schema. The second generation extends its focus to cover Volume, as well, leveraging multi-core or massive parallelization to process large-scale datasets. The third generation addresses the additional challenge of Variety, targeting voluminous, noisy, semi-structured, and highly heterogeneous data from the Semantic Web. The fourth generation also tackles Velocity so as to process data collections of a continuously increasing volume. The latest works, though, belong to the fifth generation, involving pre-trained (large) language models which heavily rely on external knowledge to address all four Vs with high effectiveness.

Automatically extracting food entities from cooking recipes has recently gained significant attention as a means of facilitating data-and AI-driven solutions for healthy and sustainable diets. The state-of-the-art approaches in the literature involve a high overhead, as their supervised functionality requires a large number of labelled instances. To facilitate food entity extraction applications, we explore the use of Large Language Model (LLMs) under zero-shot settings, where no labelled instance is required. Instead, we propose a generic methodology that exclusively focuses on prompt and response engineering. We apply it to small LLMs with just 7b parameters so as to increase their time-efficiency, while minimizing the necessary resources. Our experimental analysis yields promising results, but there is room for significant improvements.

This work presents an open-source Python library, named pyJedAI, which provides functionalities supporting the creation of algorithms related to Product Entity Resolution. Building over existing state-of-the-art resolution algorithms (Papadakis et al. 2021a), the tool offers a plethora of important tasks required for processing product data collections. It be can easily used by researchers and practitioners for creating algorithms analyzing products, such as real-time ads bidding, sponsored search, or pricing determination. In essence, it allows to easily import product data from the possible sources, compare products in order to detect either similar or identical products, generate a graph representation using the products and desired relationships, and either visualize or export the outcome in various forms. Our experimental evaluation on data from well-known online retailers illustrates high accuracy and low execution time for the supported tasks. To the best of our knowledge this is the first Python package to focus on product entities and provide this range of Product Entity Resolution functionalities.

Entity Resolution (ER) constitutes a core data integration task that relies on Blocking in order to tame its quadratic time complexity. Schema-agnostic blocking achieves very high recall, requires no domain knowledge and applies to data of any structuredness and schema heterogeneity. This comes at the cost of many irrelevant candidate pairs (i.e., comparisons), which can be significantly reduced through Meta-blocking techniques, i.e., techniques that leverage the co-occurrence patterns of entities inside the blocks: first, a weighting scheme assigns a score to every pair of candidate entities in proportion to the likelihood that they are matching and then, a pruning algorithm discards the pairs with the lowest scores. Supervised Meta-blocking goes beyond this approach by combining multiple scores per comparison into a feature vector that is fed to a binary classifier. By using probabilistic classifiers, Generalized Supervised Meta-blocking associates every pair of candidates with a score that can be used: (i) by any pruning algorithm for retaining the set of candidate comparisons; and (ii) by state-of-the-art progressive ER methods to identify the most promising candidates as early as possible (when time is a critical component for the downstream applications that consume the data). For higher effectiveness, new weighting schemes are examined as features. Through an extensive experimental analysis, we identify the best pruning algorithms, their optimal sets of features as well as the minimum possible size of the training set. The resulting approaches achieve excellent performance across several established benchmark datasets.

Entity Resolution identifies entity profiles that represent the same real-world object. A brute-force approach that considers all pairs of entities suffers from quadratic time complexity. To ameliorate this issue, filtering techniques reduce the search space to highly similar and, thus, highly likely matches. Such techniques come in two forms: (i) blocking workflows group together entity profiles with identical or similar signatures, and (ii) nearest-neighbor workflows convert all entity profiles into vectors and detect the ones closest to every query entity. The main techniques of these two types have never been juxtaposed in a systematic way and, thus, their relative performance is unknown. To cover this gap, we perform an extensive experimental study that investigates the relative performance of the main representatives per type over numerous established datasets. Comparing techniques of different types in a fair way is a non-trivial task, because the configuration parameters of each approach have a significant impact on its performance, but are hard to fine-tune. We consider a plethora of parameter configurations per methods, optimizing each workflow with respect to recall and precision in both schema-agnostic and schema-aware settings. The experimental results provide novel insights into the effectiveness, the time efficiency, the memory footprint, and the scalability of the considered techniques.

Entity resolution (ER) is the process of identifying records that refer to the same entities within one or across multiple databases. Numerous techniques have been developed to tackle ER challenges over the years, with recent emphasis placed on machine and deep learning methods for the matching phase. However, the quality of the benchmark datasets typically used in the experimental evaluations of learning-based matching algorithms has not been examined in the literature. To cover this gap, we propose four complementary approaches to assessing the difficulty and appropriateness of 13 commonly used datasets: two theoretical ones, which involve new measures of linearity and existing measures of complexity, and two practical ones - the difference between the best non-linear and linear matchers, as well as the difference between the best learning-based matcher and the perfect oracle. Our analysis demonstrates that most existing benchmark datasets pose rather easy classification tasks. As a result, they are not suitable for properly evaluating learning-based matching algorithms. To address this issue, we propose a new methodology for yielding benchmark datasets. We put it into practice by creating four new matching tasks, and we verify that these new benchmarks are more challenging and therefore more suitable for further advancements in the field.