In this paper, we present a novel Bayesian optimization method named Conditional Local Bayesian Optimization (CLBO) designed specifically to address challenges in optimizing Automated Machine Learning (AutoML) tasks. Inspired by a controller-responder architecture, our method leverages a controller that selects promising pipelines based on an acquisition function. Local responders employ Bayesian optimization to refine the search within sub-spaces. CLBO introduces a progressive budget system, which dynamically allocates the optimization budget. Our method utilizes a group of surrogate models to initially optimize a simpler objective function. Notably, CLBO outperforms current state-of-the-art algorithms on 35 classification datasets from the OpenML repository. Finally, CLBO reports unbiased performance estimates through the use of a performance correction method.
In facility location problems we seek to locate a set of facilities in an area, where clients may be present, so that some criterion is optimized. In the p-median problem we seek to minimize the sum of distances between demand points and their nearest facility, whereas in the p-dispersion problem we seek to maximize the closest distance between any two facilities. Recently, a variant of p-dispersion where distance constraints exist between facilities was studied from a Constraint Programming (CP) and Integer Linear Programming (ILP) perspective. An incomplete CP solver that uses a greedy heuristic to prune branches during search was shown to significantly outperform the ILP solver Gurobi and the CP solver OR-Tools in terms of execution time. Following that work, we consider a variant of the p-median problem where distance constraints exist between facilities and between facilities and demand points. This problem can be used to model the requirements that arise when locating semi-obnoxious facilities. We first introduce ILP and CP models and implement them in Gurobi and OR-Tools. Then, we demonstrate how a heuristic CP solver can be developed and applied on the p-median problem with distance constraints, comparing it to Gurobi and OR-Tools.
In this work we investigate zero shot learning in side effects prediction of a pair of drugs. Often combined usage of drugs has undesired side effects which have not yet been observed during clinical trials. Whereas the prediction of side effects can be formulated as a supervised machine learning problem, there are cases where there are no samples for certain sides effects, i.e. for certain classes. Thus, a zero shot learning approach can be pursued. We adapted an existing zero shot learning approach to the problem of prediction of side effects of pairs of drugs. One particular aspect of the problem is that a pair of drugs may have multiple side effects, which turns the problem into a multilabel, zero shot prediction. Finally, extensive experimentation has been conducted to gain insights.
This paper presents an improved approach to wildfire protection by integrating heterogeneous data sources under a novel fire detection ontology. The introduced technique automatically unifies data from IoT devices into a single Semantic Knowledge Graph (SemKG), improving user experience by removing the need for users to examine several data sources. To make easier quick and well-informed decision-making, the system generates real-time notifications with different degrees of severity. The validation of the introduced technique has already taken place in various pilot sites of the SILVANUS H2020 project, which focus on developing a climate-resilient forest management platform to prevent and suppress wildfires, involving 49 partners from the European Union, Brazil, Indonesia, and Australia. In this paper, we demonstrate the effectiveness of this approach through scenarios of active fire detection and gas leakage.
The integration of machine learning (ML) within the realm of nanomanufacturing processes, specifically through the applications of thickness prediction during deposition and pattern fidelity during etching, may present transformative potential. This paper discusses two ML-based methodologies that enhance the precision and efficiency of these critical processes. In the context of etching, we revisit the 3D geometrical modeling of etch-induced Line Edge Roughness (LER) transfer from photoresist lines to the substrate, refining previous models to account for realistic three-dimensional surface topographies. Meanwhile, in the deposition segment, we employ a deep neural network (DNN) to predict the thickness of films deposited on rough substrates using binarized top-down Scanning Electron Microscopy (SEM) images. The outcomes demonstrate how ML-based methodologies not only predict but also potentially control the nanofabrication parameters, leading to improved manufacturing outcomes.
The development of materials with tailored properties is often viewed as the holy grail of chemistry, due to the rather complex structure-property correlations. Artificial Intelligence (AI) and Machine Learning (ML) predictive models have been introduced in the field, as means of unveiling intricate said correlations. However, the application of these models in the physical sciences is hindered by the high cost and substantial time required to generate the necessary data for training AI algorithms. Given this backdrop of data scarcity, methods that efficiently sample and explore an unknown material's design space can significantly reduce laboratory costs and time.
The digitalisation of medicine through artificial intelligence (AI)-based tools promises to revolutionise patient management by supporting big data analysis and decision-making capabilities. Central to these advancements is the overarching requirement of aligning the design and development of AI with established values and principles of AI and biomedical ethics. Motivated by an ongoing H2020 research project that aims to develop explainable and verifiable analytic AI models for personalised clinical management of heart failure (HF) patients, this work focuses on investigating and proposing assessment metrics aimed at evaluating the AI system's adherence to the requirement of transparency, in line with the European Commission's High-Level Expert Group (HLEG) Ethics Guidelines for Trustworthy Artificial Intelligence and its respective Assessment List for Trustworthy Artificial Intelligence (ALTAI). Our study combines existing metrics and proposes new ones to assess transparency in healthcare AI applications, offering insights transferable to other domains.
This study investigates the performance of Physics-Informed-Neural-Networks (PINNs) in addressing elasto-static plate bending problems under various boundary conditions. To that effect, the bending of a square isotropic plate was simulated using the Mindlin model. The accuracy of the predictions is compared to the established method of Finite Element Analysis (FEA). For ensuring boundary condition compliance, a hard-enforced boundary method is adopted from the literature. Additionally, Fourier Feature Embeddings and Self-Scalable hyperbolic-tangent are employed for increased training stability. The findings confirm the results from previous studies regarding the ability of PINNs to successfully tackle electrostatic problems and confirm that PINNs show great promise as a novel method for solving Partial Differential Equations (PDEs).
Bibliometric analysis in the field of cybersecurity and Cyber Threat Intelligence (CTI) is crucial for identifying research trends, key themes, and collaborative networks, which can guide future research directions and policy decisions. This paper presents a comprehensive bibliometric analysis of the current status of research on knowledge graphs in cybersecurity, highlighting significant trends and thematic clusters. The analysis reveals a rapidly growing interest in integrating knowledge graphs with advanced machine learning and AI techniques, such as deep learning and neural networks, to enhance cyber threat intelligence and response strategies. Key findings include the prominence of natural language processing, entity recognition, and relation extraction as critical methodologies in this field. Thematic evolution analysis shows the adoption of large language models (LLMs) and an ongoing focus on structured knowledge representation. The study underscores the potential of knowledge graphs to improve cybersecurity through better data organization, threat detection, and intelligence extraction.
Modern Knowledge Graphs (KG) are typically constructed via automated workflows that mine information from vast amounts of, often heterogeneous, resources. Such KG are typically large and also contain formal inconsistencies, i.e. assertions that induce conflicts when combined with given axioms. Such formal inconsistencies hinder the application of classical Description Logics reasoners, as no meaningful results can be obtained by reasoning. Classical reasoners can be used with such KG to retrieve explanations for the inconsistencies, however as the KG size grows larger, the time required for this operation increases dramatically. In this paper we put forward an open-source system for detecting and fixing formal inconsistencies in large real-world knowledge graphs. We integrate and extend the state-of-the-art for parallel KG inconsistency detection and fixing in a single framework. The empirical evaluation of our method on variants of the LUBM dataset reveals its potential for effective inconsistency detection and fixing in large KG.
The Electric Network Frequency (ENF) is an essential forensic signature embedded in multimedia content. Its accurate estimation is of paramount importance. For this purpose, a relaxation (RELAX) algorithm is proposed for enhanced ENF estimation. The RELAX algorithm works by iteratively refining the frequency estimates based on cyclic minimization, starting with initial frequency and amplitude estimates. It then sequentially improves these estimates through an iterative process, adjusting the parameters until convergence is achieved. The ENF-WHU dataset is employed to demonstrate the algorithm’s efficacy, showing enhanced estimation through extensive evaluation compared to state-of-the-art methods. Paired t-tests on average mean square error confirm a statistically significant improvement. The proposed method makes a significant contribution by leveraging nonlinear spectral estimation and enhancing the reliability and accuracy of ENF estimation in forensic applications.
Li-ion batteries are an important component of the Battery Management System (BMS) in Electric Vehicles (EVs) and their performance is crucial for the efficient function of EVs. State of Charge (SoC) prediction is a significant task since this state of the battery is indicative for its performance. Data-driven methods for SoC prediction are gaining increasing interest in both academia and industry and have been driven by recent advances in Machine Learning (ML) algorithms and methods. In this paper, we perform a comparative analysis of ML pipelines and methods for SoC prediction based on a structured methodology. To do this, we build the general pipeline for SoC prediction, including data preprocessing, model development and evaluation, while we also define and configure the critical dimensions of this pipeline. We use two different datasets, and we evaluate and compare the various experimental approaches that we have applied. In other words, we assembled several experiments with different attributes and configurable dimensions, enabling the evaluation of the experiments based on these dimensions. This way, we were able to compare the performance of each approach in terms of different aspects.
Accurate political information is vital for voters to make informed decisions. However, due to the plethora of data and biased sources, accessing concise, factual information still remains a challenge. To tackle this problem, we present an open-access, deployed digital assistant powered by Large Language Models (LLMs), specifically tailored to answer voters’ questions and help them vote for the political party they mostly align with. The user can select up to 3 parties, input their question, and get short, summarized answers from the parties’ published political agendas, which contain hundreds of pages and, thus, are difficult to navigate for the typical citizen. Our NLP system architecture leverages OpenAI’s GPT-4 and incorporates Retrieval-Augmented Generation with Citations (RAG+C) to integrate custom data into LLMs effectively and build user trust. We also describe our database design, underlining the use of an open-source vector database, optimized for high-dimensional semantic search across multiple documents, and a semantic-rich LLM cache, reducing operational expenses and end-user latency time. Our open-access system supports Greek and English and has been deployed live at https://toraksero.gr/for the Greek 2023 Elections, which gathered 30K user sessions and 74% user satisfaction.
Fluid mechanics’ simulations are widely incorporated in place of expensive and complex experiments, providing accurate property fields in a dense grid fashion. It is a fact that experimental measurements due to their discrete and, most of the times, sparse data representation, cannot fully define velocity, pressure, temperature, vorticity, which are common properties of interest, especially in turbulent flows. The need to upscale flow features from coarse-grained data has been successfully addressed during the past years with aid of novel artificial intelligence -based methods. This upscaling refers either to sparse experimental data or to coarse simulation data, bypassing computationally intensive classical direct numerical simulations. This study presents a deep learning approach that accepts low resolution vorticity fields, from an open channel flow simulation, and reconstructs the field to resemble a fine grid result. This super resolution approach shows that reconstruction is possible as long as the network training is performed using pairs of low/high resolution fields. The proposed model is based on the U-Net architecture and is investigated over the effect of the resolution scale factor of the input images.
This work addresses a challenge related to Multi-Objective Optimization in machine learning model training, specifically the problem of loss coefficients weight determination for physics grounded tasks. We propose a comprehensive comparative methodology for the analysis of balancing methods for loss function coefficients in deep learning models, to enhance replicability and comparisons across diverse applications, emphasizing the use of physical parameters as figures of merit. The proposed methodology is illustrated through the evaluation of self-adaptive methods for multicomponent loss coefficients in Graph Convolutional Neural Network (GCNN) models. The GCNN are trained to reproduce the interactions between the particles of the systems under study during coarse-grained molecular dynamics simulations. Criteria are outlined both for individual model assessment and for a statistical comparison between methods, highlighting the differences in training-related characteristics, and performance metrics for the downstream task, across various self- balancing approaches.
The integration of artificial intelligence (AI) into educational systems has the potential to revolutionize teaching and learning processes, offering personalized experiences, adaptive assessments, and administrative efficiency. However, ensuring trust in AI-based education is essential for its successful implementation. This research paper explores the importance of AI in education and identifies challenges to trustworthiness based on the Assessment List for Trustworthy Artificial Intelligence (ALTAI). Drawing on this analysis, the paper proposes strategies for promoting trust in AI-driven education. By addressing these challenges and implementing the proposed strategies, educational institutions can build an environment of trust, transparency, and accountability in AI-driven education.
With the increasingly ubiquitous presence of AI systems in modern society, new threats have arisen for fundamental rights. This short paper argues that the positive obligations doctrine offers a promising conceptual framework to address some of the challenges posed by AI systems to fundamental rights. It does so in three steps. First, it describes some of the novel challenges that AI systems create for the effective protection of fundamental rights in the EU, highlighting how they put under pressure the current legal framework. Second, it argues that the imposition of positive obligations on both public and private actors to ensure the fulfilment of fundamental rights can provide an answer to some of these challenges. Third, it discusses how positive obligations could be operationalised in the framework of the newly adopted AI Act. Finally, conclusions are drawn.
Association Rule Mining is an important subfield of data mining, which consists of extracting interesting associations between items that coexist in transactions on databases. The transactions dataset may be of different types, like (a) a market basket list, where each line represents a transaction, (b) invoice detail, directly derived from ERP company prints, (c) a sparse matrix with as many columns as the different types considered for mining, and (d) nominal attributes, mainly consisting of categorical features. The classification of a given input into the correct dataset type is crucial in automated machine learning tasks. In this paper, we report on the development of an automatic dataset type recognition mechanism. A specialized "Dataset of Datasets" is created from a variety of datasets distributed by well-known repositories. Ultimately, we build a hybrid classification model consisting of a procedural programming component and a pre-trained Supervised Machine Learning model based on the Random Forest algorithm. The classification accuracy achieved is of the order of 98%. The Random Forest algorithm has been chosen after considering a number of popular machine learning algorithms like the Naive Bayes, Decision Tree, K-Nearest Neighbor (K-NN), SVM, as well as their variants.
In the research area of Natural Language Processing (NLP), text summarization (TS) has been defined as the automatic composition of a cohesive and articulate summary, which encapsulates the main ideas and themes from a single or multiple documents. Recent developments from the areas of Deep Learning (DL) and Natural Language Understanding (NLU) have facilitated the development of abstractive TS transformer-based models, which demonstrate better performance than classical extractive ones. In any case, the vast majority of current NLP research concentrates on high-resource languages (e.g., English). Dealing with modern Greek, this paper introduces and elaborates a novel abstractive TS dataset comprising 93,433 Greek Wikipedia articles and their summaries assigned by human editors. The paper also proposes a series of DL abstractive TS models that were fine-tuned on this dataset for the task of Greek article summarization. A thorough experimentation was conducted for the comparative assessment of the proposed models against well-known extractive summarization ones, using the test subset of the dataset. The results reveal that the proposed abstractive models outperform the extractive ones across various TS evaluation metrics. To enhance the reproducibility of our work, we make publicly available the corresponding dataset, our best performing model and the experimentation code.
In this paper we evaluate the performance of an efficient second order algorithm for training deep neural networks. Unlike traditional SGD with momentum, the algorithm incorporates two significant modifications originating from treating the training task as a constrained optimization problem: Firstly, it scales the momentum term with curvature information, derived from the Hessian. Secondly, it dynamically adjusts coefficients for the learning rate and the scaled momentum term. The implementation of the algorithm requires minimal additional computations compared to classical SGD with momentum iterations since it only involves computing a Hessian-vector product rather than the entire Hessian matrix, which can be efficiently computed within any modern computational graph framework. We present experimental results on various neural network architectures trained on standard benchmarks which demonstrate the effectiveness of the proposed approach.
