Knowledge Graph Embeddings (KGEs) support a wide range of downstream tasks over Knowledge Graphs (KGs). In practice, KGs evolve as new entities and facts are added, motivating Continual Knowledge Graph Embedding (CKGE) methods that update embeddings over time. Current CKGE approaches address catastrophic forgetting (i.e., the performance degradation on previously learned tasks) primarily by limiting changes to existing embeddings. However, we show that this view is incomplete. When new entities are introduced, their embeddings can interfere with previously learned ones, causing the model to predict them in place of previously correct answers. This phenomenon, which we call entity interference, has been largely overlooked and is not accounted for in current CKGE evaluation protocols. As a result, the assessment of catastrophic forgetting becomes misleading, and CKGE methods performance is systematically overestimated. To address this issue, we introduce a corrected CKGE evaluation protocol that accounts for entity interference. Through experiments on multiple benchmarks, we show that ignoring this effect can lead to performance overestimation of up to 25

In today's data-driven world, the ability to extract meaningful information from data is becoming essential for businesses, organizations and researchers. For this purpose, a wide range of tools and systems exists addressing data-related tasks, from data integration, preprocessing and modeling, to the interpretation and evaluation of the results. As data continues to grow in volume and complexity, there is an increasing need for advanced yet userfriendly tools, such as intelligent discovery assistants (IDAs) or automated machine learning (AutoML) systems, that facilitate the user's interaction with data. This enables non-expert users to effectively leverage powerful data analytics techniques. However, the use of these tools still requires non-trivial user input that cannot be anticipated from the analytical problem's data alone, but must be tailored to each individual user and their specific intents. To this end, this work explores the use of Knowledge Graphs (KG) as a foundational representation for capturing complex analytics workflows, as well as information about the users, their intents and their feedback, in order to facilitate user interaction with IDAs or AutoML tools. This is achieved through established techniques from recommender systems, in particular a link prediction approach based on KG embeddings and graph neural networks. Experimental results show that the proposed method effectively captures the graph structure and produces meaningful suggestions for users. To demonstrate the feasibility of the approach, a working prototype is presented.

Knowledge Graphs (KGs) represent knowledge as structured graphs of entities and their relationships. Knowledge Graph Embeddings (KGEs) enable learning and inference over KGs by representing their entities and relations as continuous vectors. Many KGs are frequently updated, requiring their KGEs to adapt to these changes. Continual learning methods for KGEs address this by incorporating new embeddings while updating existing ones to account for the new facts without retraining from scratch. A necessary step in these methods is the initialization of new embeddings, which has an important impact both on the accuracy of the resulting embeddings, and in the time required for their training. Current continual learning methods either use a random initialization as in non-continual scenarios, ignoring relevant information in existing embeddings, or rely on a model-dependent initialization, which is only compatible with specific KGE training models. We propose a novel informed embedding initialization strategy that can be seamlessly integrated into existing continual learning methods for KGEs. Our approach leverages the KG schema along with previously learned embeddings to generate semantically informed initial representations at every continual learning step, enhancing knowledge acquisition while reducing catastrophic forgetting in the final embeddings obtained. Experimental results show that our strategy improves the predictive performance of the resulting KGEs and enhances knowledge retention. Moreover, it accelerates the learning process, reducing the number of epochs, and thus the time required to incrementally learn new embeddings. Finally, our evaluation demonstrates that these benefits are consistent regardless of the KGE learning model used.

Despite advancements in MLOps and AutoML, ML development still remains challenging for data scientists. First, there is poor support for and limited control over optimizing and evolving ML models. Second, there is lack of efficient mechanisms for continuous evolution of ML models which would leverage the knowledge gained in previous optimizations of the same or different models. We propose an experiment-driven MLOps approach which tackles these problems. Our approach relies on the concept of an experiment, which embodies a fully controllable optimization process. It introduces full traceability and repeatability to the optimization process, allows humans to be in full control of it, and enables continuous improvement of the ML system. Importantly, it also establishes knowledge, which is carried over and built across a series of experiments and allows for improving the efficiency of experimentation over time. We demonstrate our approach through its realization and application in the ExtremeXP1 project (Horizon Europe).

Big data comes with the challenge of containing irrelevant and redundant information (i.e., features). Given that a single objective can not fully capture a feature's relevance, a Many-Objective Feature Selection (MOFS) approach able to accommodate various relevant perspectives is preferred for identifying the most appropriate features in a given context. However, MOFS produces a large set of solutions whose interpretability has been largely overlooked.First, we demonstrate the relevance of MOFS and establish its necessity by considering up to six objectives using a genetic algorithm and Naive Bayes on ten datasets for classification tasks. Then, we propose a novel methodology to improve the interpretability of MOFS results in order to support the data scientist in selecting the subset of features pertinent to their use case. Our methodology is instantiated as an intuitive and interactive dashboard that provides insights into the results beyond the pure numerical representation of the objectives being considered and evaluated with 50 participants. The outcome shows that it addresses the need for a methodological approach and comprehensive visualization to achieve interoperability.

Many Knowledege Graphs (KGs) are frequently updated, forcing their Knowledge Graph Embeddings (KGEs) to adapt to these changes. To address this problem, continual learning techniques for KGEs incorporate embeddings for new entities while updating the old ones. One necessary step in these methods is the initialization of the embeddings, as an input to the KGE learning process, which can have an important impact in the accuracy of the final embeddings, as well as in the time required to train them. This is especially relevant for relatively small and frequent updates. We propose a novel informed embedding initialization strategy, which can be seamlessly integrated into existing continual learning methods for KGE, that enhances the acquisition of new knowledge while reducing catastrophic forgetting. Specifically, the KG schema and the previously learned embeddings are utilized to obtain initial representations for the new entities, based on the classes the entities belong to. Our extensive experimental analysis shows that the proposed initialization strategy improves the predictive performance of the resulting KGEs, while also enhancing knowledge retention. Furthermore, our approach accelerates knowledge acquisition, reducing the number of epochs, and therefore time, required to incrementally learn new embeddings. Finally, its benefits across various types of KGE learning models are demonstrated.

In recent years, the availability of GPS-equipped mobile devices and other inexpensive location-tracking technologies have enabled the ubiquitous capturing of the location of moving objects. As a result, trajectory data are abundantly available and there is an increasing trend in analyzing them in the context of mobility data science. However, the abundant availability of trajectory data makes them compelling for other tasks too. In this paper, we propose the use of these data to tackle the data scarcity problem in data analysis by appropriately transforming them to extract relevant knowledge. The challenge lies not just in leveraging these abundant trajectory data, but in accurately deriving information from them that closely approximates the target variable of interest. Such knowledge can be used to generate or supplement the scarcely available datasets in a data analytics problem, thereby enhancing model learning. We showcase the feasibility of our approach in the domain of fishing where there is an abundance of trajectory data but a scarcity of detailed catch information. By using environmental data as explanatory variables, we build and compare models to predict fishing productivity using the actual catches from fishing reports and/or the inferred knowledge from the vessel’s trajectories. The results show that, mainly due to trajectory data being larger in volume than fishing data, models trained with the former obtain a precision 7.9% higher, despite the simplicity of the applied transformations.

This paper introduces Many-Objective Feature Selection (MOFS) as a novel approach for enhancing fairness in machine learning models. MOFS simultaneously optimizes model complexity, performance, and fairness using evolutionary algorithms to ensure that the selected features lead to a balanced and equitable model. We evaluate MOFS across 15 diverse fairness datasets, comparing it to three baseline methods, including IBM AI Fairness 360. Statistical analysis of our results demonstrates that MOFS consistently and significantly achieves higher accuracy and improves fairness metrics such as demographic parity and equalized odds. The study further examines the robustness and scalability of the MOFS approach across the datasets. The consistently high hypervolume metric indicator confirms that a balance between model complexity, performance, and fairness is achieved. This work highlights the potential of MOFS to develop fairer machine learning models while reducing model complexity and without compromising performance.

The ability to extract valuable information from data is crucial for organizations and individuals who want to remain competitive in a constantly evolving data-driven environment. However, some of them lack the skills required to appropriately leverage the existing data analytics tools and methods. This problem is aggravated when the users are domain-experts but completely unfamiliar with data analytics terminology, as existing assistant tools, such as AutoML or Intelligent Discovery Assistants, require them to state their analytical intent (i.e., the type of data analysis they want to perform). To address this problem, we propose to capture the underlying analytical intent from textual problem descriptions by leveraging Large Language Models (LLMs). To this end, we propose a hierarchical categorization of analytical intents, along with a data collection methodology to obtain analytical problem descriptions for all of them in order to validate different approaches that aim to extract such intents from text. Next, we compare the performance of state-of-the-art approaches with LLMs, and then study the performance of different LLMs based on their characteristics and the impact of the source of validation data. Finally, we develop a prototype to showcase how our method could interact with existing AutoML systems.

Recent advances in Large Language Models (LLMs) have positioned them as a prominent solution for Natural Language Processing tasks. Notably, they can approach these problems in a zero or few-shot manner, thereby eliminating the need for training or fine-tuning task-specific models. However, LLMs face some challenges, including hallucination and the presence of outdated knowledge or missing information from specific domains in the training data. These problems cannot be easily solved by retraining the models with new data as it is a time-consuming and expensive process. To mitigate these issues, Knowledge Graphs (KGs) have been proposed as a structured external source of information to enrich LLMs. With this idea, in this work we use KGs to enhance LLMs for zero-shot Entity Disambiguation (ED). For that purpose, we leverage the hierarchical representation of the entities' classes in a KG to gradually prune the candidate space as well as the entities' descriptions to enrich the input prompt with additional factual knowledge. Our evaluation on popular ED datasets shows that the proposed method outperforms non-enhanced and description-only enhanced LLMs, and has a higher degree of adaptability than task-specific models. Furthermore, we conduct an error analysis and discuss the impact of the leveraged KG's semantic expressivity on the ED performance.

This work is a companion reproducibility paper of the experiments and results reported in Giovanelli et al. (2022), where data pre-processing pipelines are evaluated in order to find pipeline prototypes that reduce the classification error of supervised learning algorithms. With the recent shift towards data-centric approaches, where instead of the model, the dataset is systematically changed for better model performance, data pre-processing is receiving a lot of attention. Yet, its impact over the final analysis is not widely recognized, primarily due to the lack of publicly available experiments that quantify it. To bridge this gap, this work introduces a set of reproducible experiments on the impact of data pre-processing by providing a detailed reproducibility protocol together with a software tool and a set of extensible datasets, which allow for all the experiments and results of our aforementioned work to be reproduced. We introduce a set of strongly reproducible experiments based on a collection of intermediate results, and a set of weakly reproducible experiments (Lastra-Dıaz, 0000) that allows reproducing our end-to-end optimization process and evaluation of all the methods reported in our primary paper. The reproducibility protocol is created in Docker and tested in Windows and Linux. In brief, our primary work (i) develops a method for generating effective prototypes, as templates or logical sequences of pre-processing transformations, and (ii) instantiates the prototypes into pipelines, in the form of executable or physical sequences of actual operators that implement the respective transformations. For the first, a set of heuristic rules learned from extensive experiments are used, and for the second techniques from Automated Machine Learning (AutoML) are applied.

The ongoing data boom has democratized the use of data for improved decision-making. Beyond gathering voluminous data, preprocessing the data is crucial to ensure that their most relevant aspects are considered during the analysis. Feature Selection (FS) is one integral step in data preprocessing for reducing data dimensionality and preserving the most relevant features of the data. FS can be done by inspecting inherent associations among the features in the data (filter methods) or using the model performance of a concrete learning algorithm (wrapper methods). In this work, we extensively evaluate a set of FS methods on 32 datasets and measure their effect on model performance, stability, scalability and memory usage. The results re-establish the superiority of wrapper methods over filter methods in model performance. We further investigate the unique role of wrapper methods in multi-objective FS with a focus on two traditional metrics - accuracy and Area Under the ROC Curve (AUC). On model performance, our experiments showed that optimizing for both metrics simultaneously, rather than using a single metric, led to improvements in the accuracy and AUC trade-off 1 up to 5% and 10%, respectively.

Introduction: Malaria is one of the most prevalent infectious diseases in sub-Saharan Africa, with 247 million cases reported worldwide in 2021 according to the World Health Organization. Optical microscopy remains the gold standard technique for malaria diagnosis, however, it requires expertise, is time-consuming and difficult to reproduce. Therefore, new diagnostic techniques based on digital image analysis using artificial intelligence tools can improve diagnosis and help automate it.Methods: In this study, a dataset of 2571 labeled thick blood smear images were created. YOLOv5x, Faster R-CNN, SSD, and RetinaNet object detection neural networks were trained on the same dataset to evaluate their performance in Plasmodium parasite detection. Attention modules were applied and compared with YOLOv5x results. To automate the entire diagnostic process, a prototype of 3D-printed pieces was designed for the robotization of conventional optical microscopy, capable of auto-focusing the sample and tracking the entire slide.Results: Comparative analysis yielded a performance for YOLOv5x on a test set of 92.10% precision, 93.50% recall, 92.79% F-score, and 94.40% mAP0.5 for leukocyte, early and mature Plasmodium trophozoites overall detection. F-score values of each category were 99.0% for leukocytes, 88.6% for early trophozoites and 87.3% for mature trophozoites detection. Attention modules performance show non-significant statistical differences when compared to YOLOv5x original trained model. The predictive models were integrated into a smartphone-computer application for the purpose of image-based diagnostics in the laboratory. The system can perform a fully automated diagnosis by the auto-focus and X-Y movements of the robotized microscope, the CNN models trained for digital image analysis, and the smartphone device. The new prototype would determine whether a Giemsa-stained thick blood smear sample is positive/negative for Plasmodium infection and its parasite levels. The whole system was integrated into the iMAGING smartphone application.Conclusion: The coalescence of the fully-automated system via auto-focus and slide movements and the autonomous detection of Plasmodium parasites in digital images with a smartphone software and AI algorithms confers the prototype the optimal features to join the global effort against malaria, neglected tropical diseases and other infectious diseases.

The ability to cross data from multiple sources represents a competitive advantage for organizations. Yet, the governance of the data lifecycle, from the data sources into valuable insights, is largely performed in an ad-hoc or manual manner. This is specifically concerning in scenarios where tens or hundreds of continuously evolving data sources produce semi-structured data. To overcome this challenge, we develop a framework for operationalizing and automating data governance. For the first, we propose a zoned data lake architecture and a set of data governance processes that allow the systematic ingestion, transformation and integration of data from heterogeneous sources, in order to make them readily available for business users. For the second, we propose a set of metadata artifacts that allow the automatic execution of data governance processes, addressing a wide range of data management challenges. We showcase the usefulness of the proposed approach using a real world use case, stemming from the collaborative project with the World Health Organization for the management and analysis of data about Neglected Tropical Diseases. Overall, this work contributes on facilitating organizations the adoption of data-driven strategies into a cohesive framework operationalizing and automating data governance.

Malaria is an infectious disease caused by parasites of the genus Plasmodium spp. It is transmitted to humans by the bite of an infected female Anopheles mosquito. It is the most common disease in resource-poor settings, with 241 million malaria cases reported in 2020 according to the World Health Organization. Optical microscopy examination of blood smears is the gold standard technique for malaria diagnosis; however, it is a time-consuming method and a well-trained microscopist is needed to perform the microbiological diagnosis. New techniques based on digital imaging analysis by deep learning and artificial intelligence methods are a challenging alternative tool for the diagnosis of infectious diseases. In particular, systems based on Convolutional Neural Networks for image detection of the malaria parasites emulate the microscopy visualization of an expert. Microscope automation provides a fast and low-cost diagnosis, requiring less supervision. Smartphones are a suitable option for microscopic diagnosis, allowing image capture and software identification of parasites. In addition, image analysis techniques could be a fast and optimal solution for the diagnosis of malaria, tuberculosis, or Neglected Tropical Diseases in endemic areas with low resources. The implementation of automated diagnosis by using smartphone applications and new digital imaging technologies in low-income areas is a challenge to achieve. Moreover, automating the movement of the microscope slide and image autofocusing of the samples by hardware implementation would systemize the procedure. These new diagnostic tools would join the global effort to fight against pandemic malaria and other infectious and poverty-related diseases.