Humans regularly explain observations of their environment in terms of background knowledge. This process is better characterized as abduction than as deduction, since it often requires introduction of assumptions about unobserved relations. In this paper, we present a theory of abductive explanation that builds on earlier work but extends it in new directions. The theory distinguishes between definitions and constraints, with the former used to elaborate existing explanations and the latter used to detect and repair inconsistencies. We also describe PENUMBRA, an implemented system that instantiates the theory, and demonstrate its behavior on a number of domains. The system carries out heuristic search through a space of explanations, processing observations incrementally, and generating alternative accounts for a given set of inputs. We conclude by discussing related approaches to explanation, limitations of the implementation, and directions for future research. 1. Background and Motivation One distinctive feature of human cognition is the ability to understand complex situations and events. This invariably involves explaining observations in terms of available knowledge. Moreover, these explanations are typically abductive in character, in that they incorporate plausible assumptions that are neither observed nor derived deductively. Abductive explanation is a general ability that arises in many contexts, from sentence processing and story understanding (Winston, 2012) to scene interpretation and plan recognition (Blaylock & Allen, 2005) to diagnosis (Reggia, Nau, & Wang, 1985). We would like a computational theory of the structures that underlie this capacity and the processes that operate over them. Ultimately, this should contribute to a more comprehensive cognitive architecture (Langley, Laird, & Rogers, 2009) that supports goal-directed activity over time, but here we will focus only on conceptual understanding. Let us consider a simple example. Suppose we are told that Abe possesses some cash and Bob possesses a car, but that later Abe possesses the same car. Although we did not observe any transaction, we can reasonably assume that one took place. Two explanations come immediately to mind. One is that Abe bought the car from Bob using money; another is that Abe stole the car from Bob by threatening him in some way. We also know these two explanations are mutually c © 2019 Cognitive Systems Foundation. All rights reserved. P. LANGLEY AND B. MEADOWS exclusive, in that purchases and robbery are two distinct ways to transfer possession of objects. This means that we must not only introduce plausible assumptions about unobserved events, but consider the competing explanations and keep them separate. Later, we may hear that Abe actually gave money to Bob, eliminating theft as an alternative. More complex examples would involve multi-step inference chains that generate hierarchical accounts of observations. In this paper, we present a cognitive systems account of such abductive explanations. Our analysis draws on standard ideas from the paradigm, including a focus on high-level cognition, the importance of structured representations and knowledge, a reliance on heuristic search, and incorporation of constraints from human behavior, such as incremental processing of observations. The approach that we describe builds directly on two earlier efforts (Bridewell & Langley, 2011; Meadows, Langley, & Emery, 2014) in this area, but extends them to incorporate richer representations and novel reasoning mechanisms. In the next section, we discuss two formulations of the abductive explanation task, along with prior results in each framework, and clarify our reasons for selecting one of them. After this, we describe a new theory for this ability, focusing first on assumptions about cognitive structures and then on processes that inspect and manipulate them. Next we report PENUMBRA, an implemented system that instantiates these theoretical ideas, along with its behavior on multiple scenarios that demonstrate its coverage. We close by discussing links to earlier work, noting limits of the implementation, and proposing directions for additional research in this area. 2. Two Formulations of Abductive Explanation We can define any cognitive task in terms of the information provided as inputs and the content generated as outputs. However, there are often different ways to translate an informal problem into a formal specification. The literature on abductive explanation has explored two distinct statements of this mental task that we should discuss before proceeding further. These treatments are orthogonal to whether inputs are processed incrementally, an important feature of human processing whose discussion we will delay until a later section. The first formulation borrows from classic treatments of abduction in logic and the philosophy of science (Peirce, 1878; Hempel, 1966). We can state it as: • Given: A set of general knowledge elements K (e.g., relational rules) • Given: A set of specific observed facts O (e.g., relational literals) • Find: A set of specific plausible assumptions A (e.g., relational literals) • Find: A set of proof trees P that derive elements of O from A and other elements of O with K The key idea here is that the resulting explanation, a set of linked proof trees, must contain a proof for each observed fact. These may include default assumptions as terminal nodes, which can be shared across different proof trees, but each observation must follow deductively from these assumptions and from other observations by reasoning over available knowledge. Proof trees may correspond to causal chains, as in many scientific explanations, but this is not a requirement. We will refer to this formulation as derivational abduction because observations must be derived from other beliefs. This paradigm has received considerable attention in the AI community. For example, Reggia et al. (1985) adopted the approach in their work on diagnosis, which inferred unobserved diseases that caused observed symptoms, and Hobbs et al. (1993) used it in their ap-

This paper makes two contributions towards enabling a robot to provide explanatory descriptions of its decisions, the underlying knowledge and beliefs, and the experiences that informed these beliefs. First, we present a theory of explanations comprising (i) claims about representing, reasoning with, and learning domain knowledge to support the construction of explanations; (ii) three fundamental axes to characterize explanations; and (iii) a methodology for constructing these explanations. Second, we describe an architecture for robots that implements this theory and supports scalability to complex domains and explanations. We demonstrate the architecture’s capabilities in the context of a simulated robot (a) moving target objects to desired locations or people; or (b) following recipes to bake biscuits.

We review the psychological notion of affordances and examine it anew from a cognitive systems perspective. We distinguish between environmental affordances and their internal representation, choosing to focus on the latter. We consider issues that arise in representing mental affordances, using them to understand and generate plans, and learning them from experience. In each case, we present theoretical claims that, together, form an incipient theory of affordance in cognitive systems. We close by noting related research and proposing directions for future work in this arena.

Robots assisting humans in complex domains often have to reason with different descriptions of incomplete domain knowledge. It is difficult to equip such robots with comprehensive knowledge about the domain and axioms governing the domain dynamics. This paper presents a combined architecture that enables interactive and cumulative discovery of axioms governing action capabilities, and the preconditions and effects of actions in the domain. Specifically, Answer Set Prolog is used to represent the incomplete domain knowledge, and to reason with this knowledge for planning and diagnostics. Unexpected outcomes observed during plan execution trigger reinforcement learning to interactively discover specific instances of previously unknown axioms and to revise the existing axioms. Furthermore, a decision tree induction approach based on the relational domain representation constructs generic versions of the discovered axioms, which are then used for subsequent reasoning. The architecture’s capabilities are illustrated and evaluated in a simulated domain of a robot moving objects to specific places or people in an indoor domain.

Robots assisting humans in complex domains often have to reason with different descriptions of incomplete domain knowledge. It is difficult to equip such robots with comprehensive knowledge about the domain and axioms governing the domain dynamics. This paper presents a combined architecture that enables interactive and cumulative discovery of axioms governing action capabilities, and the preconditions and effects of actions in the domain. Specifically, Answer Set Prolog is used to represent the incomplete domain knowledge, and to reason with this knowledge for planning and diagnostics. Unexpected outcomes observed during plan execution trigger reinforcement learning to interactively discover specific instances of previously unknown axioms and to revise the existing axioms. Furthermore, a decision tree induction approach based on the relational domain representation constructs generic versions of the discovered axioms, which are then used for subsequent reasoning. The architecture’s capabilities are illustrated and evaluated in a simulated domain of a robot moving objects to specific places or people in an indoor domain.

This paper describes an architecture for an agent to learn and reason about affordances. In this architecture, Answer Set Prolog, a declarative language, is used to represent and reason with incomplete domain knowledge that includes a representation of affordances as relations defined jointly over objects and actions. Reinforcement learning and decision-tree induction based on this relational representation and observations of action outcomes are used to interactively and cumulatively (a) acquire knowledge of affordances of specific objects being operated upon by specific agents; and (b) generalize from these specific learned instances. The capabilities of this architecture are illustrated and evaluated in two simulated domains, a variant of the classic Blocks World domain, and a robot assisting humans in an office environment.

As intelligent agents become more autonomous, sophisti- cated, and prevalent, it becomes increasingly important that humans interact with them effectively. Machine learning is now used regularly to acquire expertise, but common techniques produce opaque content whose behavior is difficult to interpret. Before they will be trusted by humans, autonomous agents must be able to explain their decisions and the reasoning that produced their choices. We will refer to this general ability as explainable agency. This capacity for explaining decisions is not an academic exercise. When a self-driving vehicle takes an unfamiliar turn, its passenger may desire to know its reasons. When a synthetic ally in a computer game blocks a player’s path, he may want to understand its purpose. When an autonomous military robot has abandoned a high-priority goal to pursue another one, its commander may request justification. As robots, vehicles, and synthetic characters become more selfreliant, people will require that they explain their behaviors on demand. The more impressive these agents’ abilities, the more essential that we be able to understand them.

This paper describes an architecture that enables a robot to represent, reason about, and learn affordances. Specifically, Answer Set Prolog is used to represent and reason with incomplete domain knowledge that includes affordances modeled as relations between attributes of the robot and the object(s) in the context of specific actions. The learning of affordance relations from observations obtained through reactive execution or active exploration is formulated as a reinforcement learning problem. A sampling-based approach and decision-tree regression with the underlying relational representation are used to obtain generic affordance relations that are added to the Answer Set Prolog program for subsequent reasoning. The capabilities of this architecture are illustrated and evaluated in the context of a simulated robot assisting humans in an indoor domain.

In this paper, we present an architecture for task-oriented dialogue that integrates the processes of interpretation and generation. We analyze implemented systems based on this architecture-one formeeting support and another for assisting military medics-and discuss results obtained with the first. In closing, we review some related dialogue architectures and outline plans for future research.

Robots assisting humans in complex domains need the ability to represent, reason with, and learn from, different descriptions of incomplete domain knowledge and uncertainty. This paper focuses on the challenge of incrementally and interactively discovering previously unknown axioms governing domain dynamics, and describes an architecture that integrates declarative programming and relational reinforcement learning to address this challenge. Answer Set Prolog (ASP), a declarative programming paradigm, is used to represent and reason with incomplete domain knowledge for planning and diagnostics. For any given goal, unexplained failure of plans created by ASP-based inference is taken to indicate the existence of unknown domain axioms. The task of discovering these axioms is formulated as a reinforcement learning problem, and a relational representation is used to incrementally generalize from specific axioms identified over time. These generic axioms are then added to the ASP-based representation for subsequent inference. The architecture's capabilities are demonstrated and evaluated in two domains, Blocks World and Robot Butler.

A fundamental challenge in robotics is to reason with incomplete domain knowledge to explain unexpected observations, and partial descriptions of domain objects and events extracted from sensor observations. Existing explanation generation systems are based on ideas drawn from two broad classes of systems, and do not support all the desired explanation generation capabilities for robots. The objective of this paper is to first compare the explanation generation capabilities of a state of the art system from each of these two classes, using execution scenarios of a robot waiter assisting in a restaurant. Specifically, we investigate KRASP, a system based on the declarative language Answer Set Prolog, which uses an elaborate system description and observations of system behavior to explain unexpected observations and partial descriptions. We also explore UMBRA, an architecture that provides explanations using a weaker system description, a heuristic representation of past experience, and other heuristics for selectively and incrementally searching through relevant ground literals. Based on this study, this paper identifies some key criteria, and provides some recommendations, for developing an explanation generation system for robots that exploits the complementary strengths of the two classes of explanation generation systems.

A fundamental challenge in robotics is to reason with incomplete domain knowledge to explain unexpected observations and partial descriptions extracted from sensor observations. Existing explanation generation systems draw on ideas that can be mapped to a multidimensional space of system characteristics, defined by distinctions, such as how they represent knowledge and if and how they reason with heuristic guidance. Instances in this multidimensional space corresponding to existing systems do not support all of the desired explanation generation capabilities for robots. We seek to address this limitation by thoroughly understanding the range of explanation generation capabilities and the interplay between the distinctions that characterize them. Towards this objective, this paper first specifies three fundamental distinctions that can be used to characterize many existing explanation generation systems. We explore and understand the effects of these distinctions by comparing the capabilities of two systems that differ substantially along these axes, using execution scenarios involving a robot waiter assisting in seating people and delivering orders in a restaurant. The second part of the paper uses this study to argue that the desired explanation generation capabilities corresponding to these three distinctions can mostly be achieved by exploiting the complementary strengths of the two systems that were explored. This is followed by a discussion of the capabilities related to other major distinctions to provide detailed recommendations for developing an explanation generation system for robots.

Many high-level cognitive tasks involve understanding – the mechanisms by which an agent attempts to construct accurate mental representations of its world. In this paper, we discuss two such processes: explanation and question answering. We propose four theoretical assumptions about representation and processing that arise in these tasks: both involve inference, this inference requires making default assumptions, it occurs in an incremental manner, and it produces structures that can be expressed as directed graphs of conceptual ground literals. We analyze two models of explanation and question answering in terms of these commonalities and evaluate experimental claims about them using reading comprehension passages. In closing, we discuss our findings in light of related research.

This paper examines the task of understanding dialogues in terms of the mental states of the participating agents. We present a motivating example that clarifies the challenges this problem involves and then outline a theory of dialogue interpretation based on abductive inference of these unobserved beliefs and goals, incremental construction of explanations, and reliance on domain-independent knowledge. After this, we describe UMBRA, an implementation of the theory that embodies these assumptions. We report experiments with the system that demonstrate its ability to accurately infer the conversants' mental states even when some speech acts are unavailable. We conclude by reviewing related research on dialogue and discussing avenues for future study.

In this paper, we discuss a computational approach to the cognitivetask of social planning. First, we specify a class of planningproblems that involve an agent who attempts to achieve its goalsby altering other agents' mental states. Next, we describe SFPS,a flexible problem solver that generates social plans of this sort,including ones that include deception and reasoning about otheragents' beliefs. We report the results for experiments on socialscenarios that involve different levels of sophistication and thatdemonstrate both SFPS's capabilities and the sources of its power.Finally, we discuss how our approach to social planning has beeninformed by earlier work in the area and propose directions foradditional research on the topic.

There is general agreement that knowledge plays a key role in intelligent behavior, but most work on this topic has emphasized domain-specific expertise. We argue, in contrast, that cognitive systems also benefit from meta-level knowledge that has a domain-independent character. In this paper, we propose a representational framework that distinguishes between these two forms of content, along with an integrated architecture that supports their use for abductive interpretation and hierarchical skill execution. We demonstrate this framework’s viability on high-level aspects of extended dialogue that require reasoning about, and altering, participants’ beliefs and goals. Furthermore, we demonstrate its generality by showing that the meta-level knowledge operates with different domain-level content. We conclude by reviewing related work on these topics and discussing promising directions for future research.

In this paper, we analyze the task of complex moral judgement from a computational perspective. We present a theoretical framework that posits this process often involves the construction of a coherent explanation for observed behavior in terms of the mental states of the agents involved. We extend the framework to incorporate moral values as numeric annotations on cognitive structures and mitigating factors as influences on these weights that modulate overall moral judgement. In closing, we discuss other work related to moral cognition and behavior understanding, along with directions for additional research.

In this paper, we present new experimental results supporting the Seeding Genetic Algorithm (SGA). We evaluate the algorithm’s performance with various parameterisations, making comparisons to the Canonical Genetic Algorithm (CGA), and use these as guidelines as we establish reasonable parameters for the seeding algorithm. We present experimental results confirming aspects of the theoretical basis, such as the exclusion of the deleterious mutation operator from the new algorithm, and report results on GA-difficult problems which demonstrate the SGA’s ability to overcome local optima and systematic deception.

There is general agreement that knowledge plays a key role in intelligent behavior, but most work on this topic has emphasized domain-specific expertise. We argue, in contrast, that cognitive systems also benefit from meta-level knowledge that has a domain-independent character. In this paper, we propose a representational framework that distinguishes be-tween these two forms of content, along with an integrated architecture that supports their use for abductive interpretation and hierarchical skill execution. We demonstrate this frame-work’s viability on high-level aspects of extended dialogue that require reasoning about, and altering, participants’ beliefs and goals. Furthermore, we demonstrate its generality by showing that the meta-level knowledge operates with different domain-level content. We conclude by reviewing related work on these topics and discussing promising directions for future research.