Neurosymbolic approaches can add robustness to opaque neural systems by incorporating explainable symbolic representations. However, previous approaches have not used formal logic to contextualize queries to and validate outputs of large language models (LLMs). We propose ProSLM, a novel neurosymbolic framework, to improve the robustness and reliability of LLMs in question-answering tasks. We provide ProSLM with a domain-specific knowledge base, a logical reasoning system, and an integration to an existing LLM. This framework has two capabilities (1) context gathering: generating explainable and relevant context for a given query, and (2) validation: confirming and validating the factual accuracy of a statement in accordance with a knowledge base (KB). Our work opens a new area of neurosymbolic generative AI text validation and user personalization.
Despite Large Language Models (LLMs) have revolutionised Natural Language Processing (NLP), their capability of performing logical reasoning and automated planning is still debated. In this context, the state of the art is PlanGPT, a GPT-2 model specifically trained for planning tasks. This recent approach provides GPT-based planning policies with remarkable performance, but it can generate invalid plans containing violated action preconditions or unsatisfied goals. To address this limitation, we propose an extension of PlanGPT that integrates a plan validator into the generation process. The validator is exploited to prune invalid plan prefixes during the GPT token generation, obtaining a more robust and powerful solution to planning via GPT. We empirically evaluate the effectiveness of our approach and demonstrate its potential in various planning domains.
Pre-training machine learning models on molecular properties has proven effective for generating robust and generalizable representations, which is critical for advancements in drug discovery and materials science. While recent work has primarily focused on data-driven approaches, the KANO model introduces a novel paradigm by incorporating knowledge-enhanced pre-training. In this work, we expand upon KANO by integrating the large-scale ChEBI knowledge graph, which includes 2,840 functional groups – significantly more than the original 82 used in KANO. We explore two approaches, Replace and Integrate, to incorporate this extensive knowledge into the KANO framework. Our results demonstrate that including ChEBI leads to improved performance on 9 out of 14 molecular property prediction datasets. This highlights the importance of utilizing a larger and more diverse set of functional groups to enhance molecular representations for property predictions.
Neuro-symbolic approaches have garnered much interest recently as a path toward endowing neural systems with robust reasoning capabilities. Most proposed end-to-end methods assume knowledge to be given in advance and do not scale up over many latent concepts. The recently proposed Embed2Sym tackles the scalability limitation by performing end-to-end neural training of a visual perception component from downstream labels to generate clusters in the latent space of symbolic concepts. These are later used to perform downstream symbolic reasoning but symbolic knowledge is still engineered. Taking inspiration from Embed2Sym, this paper introduces a novel method for scalable neuro-symbolic learning of first-order logic programs from raw data. The learned clusters are optimally labelled using sampled predictions of a pre-trained vision-language model. A SOTA symbolic learner, robust to noise, uses these labels to learn an answer set program that solves the reasoning task. Our approach, called Embed2Rule, is shown to achieve better accuracy than SOTA neuro-symbolic systems on existing benchmark tasks in most cases while scaling up to tasks that require far more complex reasoning and a large number of latent concepts.
While remarkable recent developments in deep neural networks have significantly contributed to advancing the state-of-the-art in Computer Vision (CV), several studies have also shown their limitations and defects. In particular, CV models often make systematic errors on important subsets of data called slices, which are groups of data sharing a set of attributes. The slice discovery problem involves detecting semantically meaningful slices on which the model performs poorly, called rare slices. We propose a modular Neurosymbolic AI approach whose distinct advantage is the extraction of human-readable logical rules that describe rare slices, and thus enhances explainability of CV models. To this end, we present a methodology to induce rare slice occurrences in a model. Experiments on datasets from our data generator leveraging on Super-CLEVR show that the approach can correctly identify rare slices and produce logical rules describing them. The rules can be fruitfully used to generate new training data to mend model behavior or may be integrated into the model to enhance its inference capabilities. (The code for reproducing our experiments is available as an online repository: https://gitlab.tuwien.ac.at/kbs/nesy- ai/ilp4sd).
While the meaning of "semantic parsing" covers a wide spectrum, we consider converting English to first order predicate logic (FOL) with the help of large language models (LLMs). The paper focuses on experiments with different approaches of using an LLM for semantic parsing to FOL: from standalone zero-shot and multishot scenarios to the use as a specialized component in several stages of the semantic parser pipeline. The goal of the experiments is to determine promising approaches for including LLM components into a question answering pipeline built around a logical reasoner with extensions for commonsense reasoning.
This paper enhances Logic Tensor Networks through the integration of uninorm based fuzzy operators. Uninorms, a class of operators that bridge the gap between t-norms and t-conorms, offer unparalleled flexibility and adaptability, making them ideal for modeling the complex, often ambiguous relationships inherent in real-world data. By embedding these operators into Logic Tensor Networks, we present a methodology that significantly increases the network's capability to handle nuanced logical operations, thereby improving its applicability across different domains. Through a series of experiments, we demonstrate the efficacy of uninorm based operators in enhancing the precision of Logic Tensor Networks. Our findings suggest that the inclusion of uninorms not only broadens the scope of problems that Logic Tensor Networks can address but also deepens their reasoning capabilities, paving the way for more sophisticated artificial intelligence systems. This work lays a foundational stone for future research in the intersection of fuzzy logic and neural-symbolic computing, suggesting directions for further exploration and integration of fuzzy systems elements into Logic Tensor Networks.GitHub: https://github.com/IDA-FBK/UniLTN
Recently, the question of what types of computation and cognition large language models (LLMs) are capable of has received increasing attention. With models clearly capable of convincingly faking true reasoning behavior, the question of whether they are also capable of real reasoning-and how the difference should be defined-becomes increasingly vexed. Here we introduce a new tool, Logic Tensor Probes (LTP), that may help to shed light on the problem. Logic Tensor Networks (LTN) serve as a neural symbolic framework designed for differentiable fuzzy logics. Using a pretrained LLM with frozen weights, an LTP uses the LTN framework as a diagnostic tool. This allows for the detection and localization of logical deductions within LLMs, enabling the use of first-order logic as a versatile modeling language for investigating the internal mechanisms of LLMs. The LTP can make deductions from basic assertions, and track if the model makes the same deductions from the natural language equivalent, and if so, where in the model this happens. We validate our approach through proof-of-concept experiments on hand-crafted knowledge bases derived from WordNet and on smaller samples from FrameNet.
Classifying parts of time series is an important task when it comes to the usage of Artificial Neural Networks (ANN), e.g. for analyzing the power consumption of households. To make it possible to adapt such ANN for Non Intrusive Load Monitoring (NILM) for the household in which they are deployed is crucial but not easy to manage. The Neurosymbolic Artificial Intelligence (AI) approach in this paper makes it possible to do that by combining ANN modules with Probabilistic Logic which is used as a supervise process to check the outputs of the ANN in case of plausibility. This on the one hand filters implausible results out which is helpful for productive usage, on the other hand these post processed results can be used to retrain the network and adapt it for a specific household in a continual learning process.
Despite their widespread use, machine learning (ML) methods often exhibit sub-optimal performance. The accuracy of these models is primarily hindered by insufficient training data and poor data quality, with particularly severe consequences in critical areas such as medical diagnosis prediction. Our hypothesis is that enhancing ML pipelines with semantic information such as those available in knowledge graphs (KG) can address these challenges and improve ML prediction accuracy. To that end, we extend the state of the art through a novel approach that uses KG embeddings to augment tabular data in various innovative ways within ML pipelines. Concretely, we introduce and examine several integration techniques of KG embeddings and the influence of KG characteristics on model performance, specifically accuracy and F2 scores. We evaluate our approach with four ML algorithms and two embedding techniques, applied to heart and chronic kidney disease prediction. Our results indicate consistent improvements in model performance across various ML models and tasks, thus confirming our hypothesis, e.g. we increased the F2 score for the KNN from 70
Knowledge graphs (KGs) are an established paradigm for integrating heterogeneous data and representing knowledge. As such, there are many different methodologies for producing KGs, which span notions of expressivity, and are tailored for different use-cases and domains. Now, as neurosymbolic methods rise in prominence, it is important to understand how the development of KGs according to these methodologies impact downstream tasks, such as link prediction using KG embeddings (KGE). In this paper, we modify FB15k-237 in several ways (e.g., by increasingly including semantic metadata). This significantly changes the graph structure (e.g., centrality). We assess how these changes impact the link prediction task, using six KGE models.
Knowledge Graph Embeddings Models project entities and relations from Knowledge Graphs into a vector space. Despite their widespread application, concerns persist about the ability of these models to capture entity similarity effectively. To address this, we introduce InterpretE, a novel neuro-symbolic approach to derive interpretable vector spaces with human-understandable dimensions in terms of the features of the entities. We demonstrate the efficacy of InterpretE in encapsulating desired semantic features, presenting evaluations both in the vector space as well as in terms of semantic similarity measurements.
This paper presents a reduced version of a currently in-use questionnaire, the GENCAT scale, to determine the level of quality of life of people with intellectual disability and uses a framework in the literature of neurosymbolic AI, specifically the family of interpretable DL named logic explained networks, to provide explanations for the predictions. By integrating explainability, our research enhances the richness of the predictions and qualitatively evaluates the reduced questionnaire's effectiveness, also illustrating the importance of explainable AI in improving assessment tools for vulnerable populations such as people with disability. The work is understood as a step to initiate dialogue with experts and practitioners using the GENCAT scale, with the ultimate goal of refining the questionnaire by discussing the explanations generated.
Large language models have been used to translate natural language questions to SQL queries. Without hard constraints on syntax and database schema, they occasionally produce invalid queries that are not executable. These failures limit the usage of these systems in real-life scenarios. We propose a neurosymbolic framework that imposes SQL syntax and schema constraints with unification-based definite clause grammars and thus guarantees the generation of valid queries. Our framework also builds a bi-directional interface to language models to leverage their natural language understanding abilities. The evaluation results on a subset of SQL grammars show that all our output queries are valid. This work is the first step towards extending language models with unificationbased grammars. We demonstrate this extension enhances the validity, execution accuracy, and ground truth alignment of the underlying language model by a large margin. Our code is available at https://github. com/ML- KULeuven/deepstochlog- lm.
Humans excel at building generalizations of new concepts from just one single example. Contrary to this, current computer vision models typically require large amount of training samples to achieve a comparable accuracy. In this work we present a Bayesian model of perception that learns using only minimal data, a prototypical probabilistic program of an object. Specifically, we propose a generative inverse graphics model of primitive shapes, to infer posterior distributions over physically consistent parameters from one or several images. We show how this representation can be used for downstream tasks such as few-shot classification and pose estimation. Our model outperforms existing few-shot neural-only classification algorithms and demonstrates generalization across varying lighting conditions, backgrounds, and out-of-distribution shapes. By design, our model is uncertainty-aware and uses our new differentiable renderer for optimizing global scene parameters through gradient descent, sampling posterior distributions over object parameters with Markov Chain Monte Carlo (MCMC), and using a neural based likelihood function. The code and datasets are available at github.com/oarriaga/bayesian-inverse-graphics ).
Graph Neural Networks (GNNs) excel at complex, graph-structured tasks by leveraging node features and graph topology through recursive neural message passing. However, their opaque internal mechanisms limit their applicability. We propose the Functional-Semantic Activation Mapping (FSAM) approach, building on the co-activation graph concept but leveraging GNN structure to explore transparency and better understand how deep representations are learned from the input graph structure. FSAM encapsulates network behaviour within a functional semantic graph, representing functional relationships between nodes based on activation values. This graph encodes statistical correlations between neuron activations across GNN layers, revealing neuron relationships from input to output. Our approach involves two experimental setups in node classification tasks: identifying critical neurons for class prediction on each layer and applying graph analysis to study class similarities and their network impact. We also visualise class pair interactions within community layers, enhancing layer-wise understanding. Our findings integrate neural and symbolic representations, demonstrating that FSAM can provide insights into the inner workings of GNNs and improve transparency in the input graph structure.
Neuro-symbolic techniques such as logic tensor networks (LTNs) enable the integration of symbolic knowledge to improve the learning capabilities of deep neural networks. LTNs in particular ground first-order logic languages into differentiable tensor operations, redefining learning as maximizing the satisfiability of a grounded theory. Despite the promising results achieved so far, the optimization task is highly sensitive to the choice of functions for grounding logical operators and aggregators, limiting their practical adoption. The present study focuses on learning in the presence of class imbalance (in object detection tasks, class imbalance arises between background vs foreground samples). In particular, we seek to combine the recently proposed logLTN with the weighting scheme introduced by the focal loss as an enhancement of the original cross-entropy loss. Preliminary experiments on an object detection benchmark show that the focal logLTN aggregator achieves higher performance and stability than its standard counterpart, with potential application in many other practical scenarios.
Contemporary deep learning architectures lack principled means for capturing and handling fundamental visual concepts, like objects, shapes, geometric transforms, and other higher-level structures. We propose a neurosymbolic architecture that uses a domain-specific language to capture selected priors of image formation, including object shape, appearance, categorization, and geometric transforms. We express template programs in that language and learn their parameterization with features extracted from the scene by a convolutional neural network. When executed, the parameterized program produces geometric primitives which are rendered and assessed for correspondence with the scene content and trained via auto-association with gradient. We confront our approach with a baseline method on a synthetic benchmark and demonstrate its capacity to disentangle selected aspects of the image formation process, learn from small data, correct inference in the presence of noise, and out-of-sample generalization.
An emerging class of neurosymbolic methods relies on the use of neural networks to determine the parameters of symbolic probabilistic models. To train these hybrid models, these methods use a knowledge compiler to turn the symbolic model into a differentiable arithmetic circuit, after which gradient descent can be performed. However, these methods require compiling a reasonably sized circuit, which is not always possible, as for many symbolic probabilistic models calculating a gradient towards the parameters is #P-hard. We introduce a new approach for learning parameters using partially compiled circuits with approximation nodes. We show that, if the errors made in the approximation nodes are bounded, the error on the gradient of partially compiled circuits can also be bounded. We evaluate the impact of various approximation guarantees on this approach's learning and generalization performance. Using approximation allows more complex queries to be compiled and our experiments show that their addition helps reduce the training loss. However, we observe that there is a limit to the addition of partial circuits after which there is no more improvement.
The Ethical Reward Machine investigates reward design involving ethical constraints with reinforcement learning. Designed to promote good behaviour across specific domains, such as simulated driving and search-and-rescue scenarios, the Ethical Reward Machine explores ethical constraints based on Act Deontology and Utilitarianism. Our contribution to the literature is a novel algorithmic pipeline integrating ethical constraints into reinforcement learning through symbolic language. Our findings indicate ethical principles impact the system significantly if there is a dilemma, and that incorporating ethical principles does not increase runtime. Therefore, our results suggest that ethical considerations do not substantially burden computational resources. Ultimately, the overarching objective is to develop and validate a learning framework that ensures AI alignment with human learning and ethical policies.
