Statistical Relational Learning (SRL) relies on statistical and probabilistic modeling to represent, learn, and reason about domains with complex relational and rich probability structures. Although SRL techniques have succeeded in many real-world applications, they follow the same assumption as most ML techniques by assuming training and testing data have the same distribution and are sampled from the same feature space. Changes between these distributions might require training a new model using new data. Transfer Learning adapts knowledge already learned to other tasks and domains to help create new models, particularly in a low-data regime setting. Many recent works have succeeded in applying Transfer Learning to relational domains. However, most focus on what and how to transfer. When to transfer is still an open research problem as a pre-trained model is not guaranteed to help or improve performance for learning a new model. Besides, testing every possible pair of source and target domains to perform transference is costly. In this work, we focus on when by proposing a method that relies on probabilistic representations of relational databases and distributions learned by models to indicate the most suitable source domain for transferring. To evaluate our approach, we analyze the performances of two transfer learning-based algorithms given the most similar target domain to a source domain according to our proposal. In the experimental results, our method has succeeded as both algorithms reach their best performance when transferring between the most similar pair of source and target domains.
This study presents a review of applications of Inductive Logic Programming (ILP) for robotic systems. The aim of the paper is to demonstrate the different methods of applying ILP to a robotic system and to also highlight some of the limitations that already exist. ILP can aid in the development of explainable and trustworthy robotics systems.
Traditional machine learning methods heavily rely on large amounts of labelled data for effective generalisation, posing a challenge in few-shot learning scenarios. In many real-world applications, acquiring large amounts of training data can be difficult or impossible. This paper presents an efficient and explainable method for few-shot learning from images using inductive logic programming (ILP). ILP utilises logical representations and reasoning to capture complex relationships and generalise from sparse data. We demonstrate the effectiveness of our proposed ILP-based approach through an experimental evaluation focused on detecting neurodegenerative diseases from fundus images. By extending our previous work on neurodegenerative disease detection, including Alzheimer’s disease, Parkinson’s disease, and vascular dementia disease, we achieve improved explainability in identifying these diseases using fundus images collected from the UK Biobank dataset. The logical representation and reasoning inherent in ILP enhances the interpretability of the detection process. The results highlight the efficacy of ILP in few-shot learning scenarios, showcasing its remarkable generalisation performance compared to a range of other machine learning algorithms. This research contributes to the field of few-shot learning using ILP and paves the way for addressing challenging real-world problems.
Learning settings are crucial for most Inductive Logic Programming (ILP) systems to learn efficiently. Hypothesis spaces can be huge, and ILP systems take a long time to output solutions or even cannot terminate within time limits. Therefore, users must set suitable learning settings for each ILP task to bring the best performance of the system. However, most users struggle to set appropriate settings for the task they see for the first time. In this paper, we propose a method to make an ILP system more adaptable to tasks with weak learning biases. In particular, we attempt to learn efficient strategies for an ILP system using reinforcement learning (RL). We use Popper, a state-of-the-art ILP system that implements the concept of learning from failures (LFF). We introduce RL-Popper, which divides the hypothesis space into subspaces more minutely than Popper. RL is used to learn the efficient search order of the divided spaces. We provide the details of RL-Popper and show some empirical results.
Markov Logic Networks (MLNs) combine relational specifications with probabilistic learning and reasoning. Although they can be specified without reference to a particular domain, their performance across domains of different sizes is generally unfavorable. Domain-size Aware Markov Logic Networks (DA-MLNs) address this issue by scaling down weight parameters based on the domain size. This allows for faster learning by training models on a random sample of the original domain. DA-MLNs also enable transferring models between naturally occurring domains of different sizes. This study proposes a combination of functional gradient boosting and weight scaling for single-target structure learning on large domains. It also evaluates performance and runtime on two benchmark domains of contrasting sizes. The results demonstrate that training a DA-MLN from a sample can significantly reduce learning time for large domains with minor performance trade-offs, which decrease with the size of the original domain. Additionally, the study explores how scaling reacts to varying domain sizes in a synthetic social network domain. It is observed that DA-MLNs outperform unscaled MLNs when the number of connections between individuals grows with domain size, but perform worse when the number of connections remains constant. This justifies the use of unscaled MLNs when sampling isolated subcommunities in areas such as social sciences research.
Neural-symbolic AI is the field that seeks to integrate deep learning with symbolic, logic-based methods, as they have complementary strengths. Lately, more and more researchers have encountered the limitations of deep learning, which has led to a rise in the popularity of neural-symbolic AI, with a wide variety of systems being developed. However, many of these systems are either evaluated on different benchmarks, or introduce new benchmarks that other systems have not been tested on. As a result, it is unclear which systems are suited to which tasks, and whether the difference between systems is actually significant. In this paper, we give an overview and classification of the tasks used in state-of-the-art neural-symbolic system. We show that most tasks fall in one of five categories, and that very few systems are compared on the same benchmarks. We also provide a methodological experimental comparison of a variety of systems on two popular tasks: learning with distant supervision and structured prediction. Our results show that a systems based on (probabilistic) logic programming achieve superior performance on these tasks, and that the performance amongst these methods does not differ significantly. Finally, we also discuss how the properties of the (probabilistic) logic programming-based systems are desirable for most neural-symbolic tasks.
In Boolean Satisfiability (SAT), Minimal Unsatisfiable Subsets (MUSes) are unsatisfiable subsets of constraints that serve as explanations for the unsatisfiability which, as a result, have been used in various applications. Although various systematic algorithms for the extraction of MUSes have been proposed, few heuristic methods have been studied, as the process of designing efficient heuristics requires extensive experience and expertise. In this research, we propose the first trainable heuristic based on Graph Neural Networks (GNNs). We design a new network structure along with loss functions and learning strategies specifically tuned to learn the process of MUS extraction, which we implement in a model called GNN-MUS. Furthermore, we introduce a new algorithm called NeuroMUSX that uses GNN-MUS as a heuristic and combines it with other systematic search methods to make the extraction process more efficient. We conduct experiments to compare our proposed method with existing methods on the MUS Track of the 2011 SAT Competition. From the results, NeuroMUSX is shown to achieve significantly better performance across a wide range of problem instances. In addition, training NeuroMUSX on specific instances of a class improves the algorithm’s performance against other problems in the same class, highlighting the advantages of the learnable approach. Overall, these results underscore the potential of using simple GNN architectures to drastically improve the procedures for extracting minimal subsets.
Fractals are geometric patterns with identical characteristics in each of their component parts. They are used to depict features which have recurring patterns at ever-smaller scales. This study offers a technique for learning from fractal images using Meta-Interpretative Learning (MIL). MIL has previously been employed for few-shot learning from geometrical shapes (e.g. regular polygons) and has exhibited significantly higher accuracy when compared to Convolutional Neural Networks (CNN). Our objective is to illustrate the application of MIL in learning from fractal images. We first generate a dataset of images of simple fractal and non-fractal geometries and then we implement a technique to learn recursive rules which describe fractal geometries. Our approach uses graphs extracted from images as background knowledge. Finally, we evaluate our approach against CNN-based approaches, such as Siamese Net, VGG19, ResNet50 and DenseNet169.
Probabilistic Answer Set Programming under the credal semantics has emerged as one of the possible formalisms to encode uncertain domains described by an answer set program extended with probabilistic facts. Some problems require associating probability values to probabilistic facts such that the probability of a query is above a certain threshold. To solve this, we propose a new class of programs, called Probabilistic Optimizable Answer Set Programs, together with a practical algorithm based on constrained optimization to solve the task.
Inferring a LTLf formula from a set of example traces, also known as passive learning, is a challenging task for model-based techniques. Despite the combinatorial nature of the problem, current state-of-the-art solutions are based on exhaustive search. They use an example at the time to discard a single candidate formula at the time, instead of exploiting the full set of examples to prune the search space. This hinders their applicability when examples involve many atomic propositions or when the target formula is not small. This short paper proposes the first ILP-based approach for learning LTLf formula from a set of example traces, using a learning from answer sets system called ILASP. It compares it to both pure SAT-based techniques and the exhaustive search method. Preliminary experimental results show that our approach improves on previous SAT-based techniques and that has the potential to overcome the limitation of an exhaustive search by optimizing over the full set of examples. Further research directions for the ILP-based LTLf passive learning problem are also discussed.
A ProbLog program is a logic program with facts that only hold with a specified probability. Each ProbLog program gives rise to probability estimations for counterfactual statements of the form “A would be true, if we had forced B”. This contribution studies program equivalence with respect to this counterfactual reasoning in the sense of Judea Pearl. Our main result reveals that each well-written ProbLog program with non-trivial probabilities is uniquely determined by its associated counterfactual estimations. More precisely, we give a procedure to reconstruct such a probabilistic logic program from its counterfactual output. As counterfactuals are part of our everyday language, our result indicates that they may also be a good language to express domain knowledge or readable program specifications.
Probabilistic Logic Programming combines uncertainty and logic-based languages. Liftable Probabilistic Logic Programs have been recently proposed to perform inference in a lifted way. LIFTCOVER is an algorithm used to perform parameter and structure learning of liftable probabilistic logic programs. In particular, it performs parameter learning via Expectation Maximization and LBFGS. In this paper, we present an updated version of LIFTCOVER, called LIFTCOVER+, in which regularization was added to improve the quality of the solutions and LBFGS was replaced by gradient descent. We tested LIFTCOVER+ on the same 12 datasets on which LIFTCOVER was tested and compared the performances in terms of AUC-ROC, AUC-PR, and execution times. Results show that in most cases Expectation Maximization with regularization improves the quality of the solutions.
We propose a program synthesis challenge inspired by the Abstraction and Reasoning Corpus (ARC) [3]. The ARC is intended as a touchstone for human intelligence. It consists of 400 tasks, each with very small numbers (3–5) of 'input-output' image pairs. It is known that the tasks are 'human-solvable' in the sense that, for any of the tasks, there exists a human-authored description that transforms input images in the task to the corresponding output images. Besides the 'small data problem', other features of ARC make it hard to use as a yardstick for machine learning. The solutions are not provided, nor is it known if they are unique. The use of some basic prior knowledge is acknowledged, but no definitions are available. The solutions are known also to apply to images that may be significantly different to those provided, but those images are not described. Inspired by ARC, but motivated to address some of these issues, in this paper we propose the Inductive Program Synthesis Challenge for ARC-like tasks (IPARC). The IPARC challenge is much more controlled, focusing on the inductive synthesis of structured programs. We specify for the challenge a set of 'ARC-like' tasks characterised by: training and test example sets drawn from a clearly-defined set of 'ARC-like' input-output image pairs; a set of image transformation functions from the image-processing field of Mathematical Morphology (MM); and target programs known to solve the tasks by transforming input to output images. The IPARC tasks rely on a result known as the 'Structured Program Theorem' that identifies a small set of rules as sufficient for construction of a wide class of programs. Tasks in the IPARC challenge are intended for machine learning methods of program synthesis able to address instances of these rules. In principle, Inductive Logic Programming (ILP) has the techniques needed to identify the constructs implied by the Structured Program Theorem. But, in practice, is there an ILP implementation that can achieve this? The purpose of the IPARC challenge is to determine if this is the case.
In this paper, we review hierarchical problem networks, which encode knowledge about how to decompose planning tasks, and report an approach to learning this expertise from sample solutions. In this framework, procedural knowledge comprises a set of conditional methods that decompose problems – sets of goals – into subproblems. Problem solving involves search through a space of hierarchical plans that achieve top-level goals. Acquisition involves creation of new methods, including state conditions for when they are relevant and goal conditions for when to avoid them. We describe HPNL, a system that learns new methods by analyzing sample hierarchical plans, using violated constraints to identify state conditions and ordering conflicts to determine goal conditions. Experiments with on-line learning in three planning domains demonstrate that HPNL acquires expertise that reduces search on novel problems and examine the importance of learning goal conditions. In closing, we contrast the approach with earlier methods for acquiring search-control knowledge, including explanation-based learning and inductive logic programming. We also discuss limitations and plans for future research.
Adding to the budding landscape of advanced analysis tools for Probabilistic Soft Logic (PSL), we present a graphical explorer for grounded PSL models. It exposes the structure of the model from the perspective of any single atom, listing the ground rules in which it occurs. The other atoms in these rules serve as links for navigation through the resulting rule-atom graph (RAG). As additional diagnostic criteria, each associated rule is further classified as exerting upward or downward pressure on the atom's value, and as active or inactive depending on its importance for the MAP estimate. Our RAG viewer further includes a general infrastructure for making PSL results explainable by stating the reasoning patterns in terms of domain language. For this purpose, we provide a Java interface for "talking" predicates and rules which can generate verbalized explanations of the atom interactions effected by each rule. If the model's rules are structured similarly to the way the domain is conceptualized by users, they will receive an intuitive explanation of the result in natural language. As an example application, we present the current state of the loanword detection component of EtInEn, our upcoming software for machine-assisted etymological theory development.
Complex Event Recognition and Forecasting (CER/F) techniques attempt to detect, or even forecast ahead of time, event occurrences in streaming input using predefined event patterns. Such patterns are not always known in advance, or they frequently change over time, making machine learning techniques, capable of extracting such patterns from data, highly desirable in CER/F. Since many CER/F systems use symbolic automata to represent such patterns, we propose a family of such automata where the transition-enabling conditions are defined by Answer Set Programming (ASP) rules, and which, thanks to the strong connections of ASP to symbolic learning, are directly learnable from data. We present such a learning approach in ASP and an incremental version thereof that trades optimality for efficiency and is capable to scale to large datasets. We evaluate our approach on two CER datasets and compare it to state-of-the-art automata learning techniques, demonstrating empirically a superior performance, both in terms of predictive accuracy and scalability.
Abductive reasoning plays an essential part in day-to-day problem-solving. It has been considered a powerful mechanism for hypothetical reasoning in the presence of incomplete knowledge; a form of "common sense" reasoning. In machine learning, abduction is viewed as a conceptual method in which data and the bond that jointly brings the different types of inference. The traditional Mode-Directed Inverse Entailment (MDIE) based systems such as Progol and Aleph for the abduction were not data-efficient since their execution time with the large dataset was too long. We present a new abductive learning procedure using Meta Inverse Entailment (MIE). MIE is similar to Mode-Directed Inverse Entailment (MDIE) but does not require user-defined mode declarations. In this paper, we use an implementation of MIE in Python called PyGol. We evaluate and compare this approach to reveal the microbial interactions in the ecosystem with state-of-art-of methods for abduction, such as Progol and Aleph. Our results show that PyGol has comparable predictive accuracies but is significantly faster than Progol and Aleph.
Probabilistic Answer Set Programming (PASP) is a powerful formalism that allows to model uncertain scenarios with answer set programs. One of the possible semantics for PASP is the credal semantics, where a query is associated with a probability interval rather than a sharp probability value. In this paper, we extend the learning from interpretations task, usually considered for Probabilistic Logic Programming, to PASP: the goal is, given a set of (partial) interpretations, to learn the parameters of a PASP program such that the product of the lower bounds of the probability intervals of the interpretations is maximized. Experimental results show that the execution time of the algorithm is heavily dependent on the number of parameters rather than on the number of interpretations.
Dependencies on the relative frequency of a state in the domain are common when modelling probabilistic dependencies on relational data. For instance, the likelihood of a school closure during an epidemic might depend on the proportion of infected pupils exceeding a threshold. Often, rather than depending on discrete thresholds, dependencies are continuous: for instance, the likelihood of any one mosquito bite transmitting an illness depends on the proportion of carrier mosquitoes. Current approaches usually only consider probabilities over possible worlds rather than over domain elements themselves. We introduce functional lifted Bayesian networks, a formalism that explicitly incorporates relative frequencies into statistical relational artificial intelligence. Incorporating relative frequencies is not only beneficial to modelling; it also provides a more rigorous approach to learning problems where training and test or application domains have different sizes. To this end, we provide a representation of the asymptotic probability distributions induced by functional lifted Bayesian networks on domains of increasing sizes. Since that representation has well-understood scaling behaviour across domain sizes, it can be used to estimate parameters for a large domain consistently from randomly sampled subpopulations.
Statistical relational learning (SRL) algorithms have succeeded in many real-world applications as real-world data is relational and consists of different entities characterized by different sets of attributes. Real-world data can also be noisy and have incomplete information. Like traditional machine learning models, SRL models also assume training and testing data are sampled from the same distribution. If distributions differ, a new model must be trained using newly collected data. Employing Transfer Learning to machine learning models has become a great asset in handling such issues. It aims to leverage the knowledge learned in a source domain to train a model in a target domain. Moreover, SRL models may suffer from insufficient high-quality data instances and a long training time. Recent work has shown that applying transfer learning is suitable for SRL models as it admits training and testing data sampled from different distributions. However, an essential challenge is how to transfer the learned structure, mapping the vocabulary across different domains. This work relies on a previous approach that uses the similarity between pre-trained word embeddings to guide the mapping and applies theory revision to improve its inferential capacities. However, choosing the most suitable similarity metrics for a specific pair of source and target datasets is not trivial. Thus, we propose to combine different similarity metrics to map predicates. Experimental results showed that combining distinct similarity metrics has improved or equated performance compared to previous methods. It also requires less training time for some experiments.
