In Hybrid AI, machine learning and knowledge engineering are combined to have the best of both worlds. Insights obtained from data are combined with complementary expert knowledge, which can be represented in a graph structure. Graph networks are a recently new development in machine learning and cover methods that learn on graph structured data. This paper researches how knowledge can be incorporated in graph networks for the use case of scene classification. The aim is to detect novel scenes, of which only a few examples and noisy object detections are available. The results show that both using a graph network and adding knowledge can improve performance, however, this is not always necessarily the case. The novelty of this paper is threefold: 1. Using GNNs for scene classification; 2. Combining data and knowledge in GNNs by constructing one input graph; 3. Using GNNs in cases with few training samples and noisy inputs.
We describe a method “Kg2Causal” for using a large-scale, general-purpose biomedical knowledge graph as a prior for data-driven causal network structure learning. Given a set of observed nodes in a dataset, and some relationship edges between the nodes derived from a knowledge graph, Kg2Causal uses the knowledge graph-derived edges to guide the data-driven inference of a causal Bayesian network. We tested Kg2Causal on several real-world biological datasets with known ground-truth networks and demonstrate improvement in network learning accuracy, relative to a baseline of an uninformative network structure prior. We also demonstrate the application of our method if data are collected under different experimental conditions including interventions on the observed variables.
Deep learning has achieved state-of-the-art results in various application domains ranging from image recognition to language translation and game playing. However, it is now generally accepted that deep learning alone has not been able to satisfy the requirement of fairness and, ultimately, trust in Artificial Intelligence (AI). In this paper, we propose an interactive neural-symbolic approach for fairness in AI based on the Logic Tensor Network (LTN) framework. We show that the extraction of symbolic knowledge from LTN-based deep networks combined with fairness constraints offer a general method for instilling fairness into deep networks via continual learning. Explainable AI approaches which otherwise could identify but not fix fairness issues are shown to be enriched with an ability to improve fairness results. Experimental results on three real-world data sets used to predict income, credit risk and recidivism in financial applications show that our approach can satisfy fairness metrics while maintaining state-of-the-art classification performance.
We report on experiences in adding ML-trained visual recognition modules to a human-oriented image semantic annotation tool which creates RDF descriptions of images and scene contents. We conclude that ML cannot replace expert humans but can aid them in various ways, some unexpected. Semantic markup systems can be to designed to align human and machine blind spots. Finally, we briefly outline directions for future work.
Integration of symbolic and sub-symbolic approaches is rapidly emerging as an Artificial Intelligence (AI) paradigm. This paper presents a proof-of-concept approach towards an unsupervised learning method, based on Restricted Boltzmann Machines (RBMs), for extracting semantic associations among prominent entities within data. Validation of the approach is performed in two datasets that connect language and vision, namely Visual Genome and GQA. A methodology to formally structure the extracted knowledge for subsequent use through reasoning engines is also offered.
In order to keep complex railway systems fail-safe, sophisticated maintenance of the rolling stock and infrastructure are most essential. Although AI-based predictive maintenance systems exist in many different industries, there is still quite large potential for different application scenarios. The current research shows such an example, where machine learning can be applied to detect anomalies in the pantograph-catenary system by using a simple convolutional neural network that is able to detect arc ignitions during train operation. The paper provides some insights into the process of the system development life cycle. Starting from the initial idea to use machine learning for anomaly detection, over the system design of a prototype and the training of the Keras-based machine-learning model, up until the evaluation of the conducted experiments. The arcVision system prototype provides valuable insights into how a predictive maintenance process could be established by combining the results from the machine-learning model with rules and insights from manual inspections.
Record linkage, often called entity resolution or de-duplication, refers to identifying the same entities across one or more databases. As the amount of data that is generated grows at an exponential rate, it becomes increasingly important to be able to integrate data from several sources to perform richer analysis. In this paper, we present an open source comprehensive end to end hybrid record linkage framework that combines the automatic and manual review process. Using this framework, we train several models based on different machine learning algorithms such as random forests, linear SVM, Radial SVM, and Dense Neural Networks and compare the effectiveness and efficiency of these models for record linkage in different settings. We evaluate model performance based on Recall, F1-score (quality of linkages) and number of uncertain pairs which is the number of pairs that need manual review. We also test our trained models in a new dataset to test how different trained models transfer to a new setting. The RF, linear SVM and radial SVM models transfer much better compared to the DNN. Finally, we study the effect of name2vec (n2v) feature, a letter embedding in names, on model performance. Using n2v results in smaller manual review set with slightly less F1-score. Overall the SVM models performed best in all experiments.
Marketers need new ways of generating campaigns artificially for their marketing activities. Many marketers assume proprietary systems are individualized enough. This article investigates an order ...
Optimizing the hyperparameters is one of the most important and time-consuming activities to do when training machine learning models. But the lack of guidance available to optimization algorithms means that finding values for these hyperparameters is left to black-box methods. Black-box methods can be made more efficient by incorporating an understanding of where good hyperparameter values might be located for a specific model. In this paper, we visualize hyperparameter performance-landscapes in several datasets to discover how the XGBoost algorithm behaves for many combinations of hyperparameter values across these datasets. Using this knowledge, it might be possible to design more efficient search strategies for optimizing the hyperparameters of XGBoost.
Real time human posture estimation using reduced number of sensors is a challenging and highly sought after problem. Various model-based methods have been developed over the years in this direction which utilize optical and/or inertial sensor data. Although these methods have proven effective in laboratory settings, their applicability in the real world is limited due to the difficulty in information gathering, high intrusiveness and higher cost. This non-position paper deals with a hybrid approach involving full-body inverse kinematics (IK) and deep learning in order to estimate physiologically feasible joint angles in real time, based on orientation information from 6 inertial measurement units (IMUs). IK is performed on a kinematically constrained 3D human body model, to obtain joint angles of the body model, given orientation data of 17 sensors attached to different bone segments of the body. A bidirectional recurrent neural network (bi-RNN) is then trained using a newly collected IMU dataset to regress from the orientation data of 6 sensors to the joint angles obtained from IK. The training converged to a mean squared error (MSE) of 5.98 degrees.
Knowledge graph (KG) embedding models have recently gained increased attention. However, most of the existing models for KG embeddings ignore the structure and characteristics of the underlying ontology. In this work, we present EmEL++ embeddings – an ontology-based embedding model for the ++ description logic. EmEL++ maps the classes and the relations in an ontology to an n-dimensional vector space such that the relations between classes and relations in the ontology are preserved in the vector space. We evaluate the proposed embeddings on four different datasets and show that the proposed embeddings outperform the traditional knowledge graph embeddings on the subsumption reasoning task.
A variety of research on theory and knowledge refinement that integrated knowledge engineering and machine learning was conducted in the 1990’s. This work developed a variety of techniques for taking engineered knowledge in the form of propositional or first-order logical rule bases and revising them to fit empirical data using symbolic, probabilistic, and/or neural-network learning methods. We review this work to provide historical context for expanding these techniques to integrate modern knowledge engineering and machine learning methods.
The job market is extremely flexible and constantly evolving. If information is represented in a machinereadable way, it is easier to add new terms or job titles and relate that to the existing terms. Several different representations of this field already exist, but those are not aligned yet. This paper examines the automatic alignment of two occupation ontologies ESCO and O*NET using Natural Language Processing methods. We specifically focus on a contextualized embedding model named BERT, and compare performance of five alignment systems. The novelty of this paper is twofold: 1) ontology alignment is applied in a real-word use-case in the labour market field; 2) BERT is applied for ontology alignment. It is found that, while their performance is not good enough yet to yield a useful alignment on their own, BERT-based embeddings mostly outperform word2vec-based embeddings. It is concluded that a hybrid approach is needed, where automatic alignment techniques are combined with manual alignment techniques, in order to improve coverage and eliminate errors.
Increasing automation in the healthcare sector calls for a Hybrid Intelligence (HI) approach to closely study and design the collaboration of humans and autonomous machines. Ensuring that medical HI systems’ decision-making is ethical is key. The use of Team Design Patterns (TDPs) can advance this goal by describing successful and reusable configurations of design problems in which decisions have a moral component and facilitating communication in multidisciplinary teams designing HI systems. For this research, TDPs were developed describing a set of solutions for a design problem in a medical HI system: mitigating harmful biases in machine learning algorithms. The Socio-Cognitive Engineering (SCE) methodology was employed, integrating operational demands, human factors knowledge, and a technological analysis into a set of TDPs. A survey was created to assess the usability of the patterns with regards to their understandability, effectiveness, and generalizability. Results showed that TDPs are a useful method to unambiguously describe solutions for diverse HI design problems with a moral component on varying abstraction levels, usable by a heterogeneous group of multidisciplinary researchers. Additionally, results indicated that the SCE approach and the developed questionnaire are suitable methods for creating and assessing TDPs.
Combining symbolic-reasoning and data learning in a unified double-loop learning system can contribute to the emergence of artificial intelligence solutions that are more adaptive to social and behavioural context. This paper presents a hybrid user modeling framework that relies on the integration of machine learning and reasoning methods equipped with formally represented domain knowledge. We find that this approach contributes to the design of context-aware systems that require less data, manage bias better, provide better transparency and can handle data sparsity more effectively. We present the impact of our work in different social domains from building trusted digital surrogates to decentralization of social recommendation services. Our approach can construct software agents from identity and expertise of users and allows such entities to become more digitally portable. Our approach also contributes to the emergence of expertise sharing paradigms that are less prone to biases and more privacy preserving. The paper uses these domain applications to validate the scalability and versatility of our approach augmented with principles of open and transparent algorithms.
We present an approach to improve individual and self-regulated learning in group assignments. We focus on supporting individual reflection by providing feedback through a conversational system. Our approach leverages machine learning techniques to recognize concepts in student utterances and combines them with knowledge representation to infer the student’s understanding of an assignment’s cognitive requirements. The conversational agent conducts end-to-end conversations with the students and prompts them to reflect and improve their understanding of an assignment. The conversational agent not only triggers reflection but also encourages explanations for partial solutions
The company FutureTV produces and distributes self-produced videos in the fashion domain. It creates revenue through the placement of relevant advertising. The placement of apposite ads, though, requires an understanding of the contents of the videos. Until now, this tagging is created manually in a labor-intensive process. We believe that image recognition technologies can significantly decrease the need for manual involvement in the tagging process. However, the tagging of videos comes with additional challenges: Preliminary, new deep-learning models need to be trained on vast amounts of data obtained in a labor-intensive data-collection process. We suggest a new approach for the combining of deep-learning-based recognition with a semantic reasoning engine. Through the explicit declaration of knowledge fitting to the fashion categories present in the training data of the recognition system, we argue that it is possible to refine the recognition results and win extra knowledge beyond what is found in the neural net.
Creation of machine-usable, high-quality knowledge-bases is a critical prerequisite for many important applications that rely on availability of high-level of autonomous decision-making and reasoning capabilities. Manual construction of knowledge-bases for complex applications is a time-consuming and expensive process. In such application domains, however, a vast amount of knowledge is available in human-readable format, and it could be leveraged to build knowledge-bases automatically. Natural Language Processing (NLP)-based techniques provide an attractive option for this process. The field of NLP has made rapid strides in last several years and resulted in increased usage across a variety of consumer-facing applications. However, their usage for knowledge-base construction in the aviation industry remains rather limited to date. We present our assessment of using various NLP-based tools for the creation of aviation-focused, high-quality, machineprocessable, and human-legible knowledge bases (KBs) for various applications. We identify several gaps, both at the application and fundamental levels, and also identify potential directions for future research that could help overcome the challenges.
A significant and recent development in neural-symbolic learning are deep neural networks that can reason over symbolic knowledge graphs (KGs). A particular task of interest is KG entailment , which is to infer the set of all facts that are a logical consequence of current and potential facts of a KG. Initial neural-symbolic systems that can deduce the entailment of a KG have been presented, but they are limited: current systems learn fact relations and entailment patterns specific to a particular KG and hence do not truly generalize, and must be retrained for each KG they are tasked with entailing. We propose a neural-symbolic system to address this limitation in this paper. It is designed as a differentiable end-to-end deep memory network that learns over abstract, generic symbols to discover entailment patterns common to any reasoning task. A key component of the system is a simple but highly effective normalization process for continuous representation learning of KG entities within memory networks. Our results show how the model, trained over a set of KGs, can effectively entail facts from KGs excluded from the training, even when the vocabulary or the domain of test KGs is completely different from the training KGs.
