Visual data analysis is composed of a collection of cognitive actions and tasks to decompose, internalize, and recombine data to produce knowledge and insight. Visual analytic tools provide interactive visual interfaces to data to support discovery and sensemaking tasks, including forming hypotheses, asking questions, and evaluating and organizing evidence. Myriad analytic models can be incorporated into visual analytic systems at the cost of increasing complexity in the analytic discourse between user and system. Techniques exist to increase the usability of interacting with analytic models, such as inferring data models from user interactions to steer the underlying models of the system via semantic interaction, shielding users from having to do so explicitly. Such approaches are often also referred to as mixed-initiative systems. Sensemaking researchers have called for development of tools that facilitate analytic sensemaking through a combination of human and automated activities. However, design guidelines do not exist for mixed-initiative visual analytic systems to support iterative sensemaking. In this paper, we present candidate design guidelines and introduce the Active Data Environment (ADE) prototype, a spatial workspace supporting the analytic process via task recommendations invoked by inferences about user interactions within the workspace. ADE recommends data and relationships based on a task model, enabling users to co-reason with the system about their data in a single, spatial workspace. This paper provides an illustrative use case, a technical description of ADE, and a discussion of the strengths and limitations of the approach.

AR-Mentor is a wearable real time Augmented Reality (AR) mentoring system that is configured to assist in maintenance and repair tasks of complex machinery, such as vehicles, appliances, and industrial machinery. The system combines a wearable Optical-See-Through (OST) display device with high precision 6-Degree-Of-Freedom (DOF) pose tracking and a virtual personal assistant (VPA) with natural language, verbal conversational interaction, providing guidance to the user in the form of visual, audio and locational cues. The system is designed to be heads-up and hands-free allowing the user to freely move about the maintenance or training environment and receive globally aligned and context aware visual and audio instructions (animations, symbolic icons, text, multimedia content, speech). The user can interact with the system, ask questions and get clarifications and specific guidance for the task at hand. A pilot application with AR-Mentor was successfully built to instruct a novice to perform an advanced 33-step maintenance task on a training vehicle. The initial live training tests demonstrate that AR-Mentor is able to help and serve as an assistant to an instructor, freeing him/her to cover more students and to focus on higher-order teaching.

As AI developers increasingly look to workflow technologies to perform complex integrations of individual software components, there is a growing need for the workflow systems to have expressive descriptions of those components. They must know more than just the types of a component’s inputs and outputs; instead, they need detailed characterizations that allow them to make fine-grained distinctions between candidate components and between candidate workflows. This paper describes ProCat, an implemented ontology-based catalog for components, conceptualized as processes, that captures and communicates this detailed information. ProCat is built on a layered representation that allows reasoning about processes at varying levels of abstraction, from qualitative constraints reflecting preconditions and effects, to quantitative predictions about output data and performance. ProCat employs Semantic Web technologies RDF, OWL, and SPARQL, and builds on Semantic Web services research. We describe ProCat’s approach to representing and answering queries about processes, discuss some early experiments evaluating the quantitative predictions, and report on our experience using ProCat in a system producing workflows for intelligence analysis.

As adaptive agents become more complex and take increasing autonomy in their user's lives, it becomes more important for users to trust and understand these agents. Little work has been done, however, to study what factors influence the level of trust users are willing to place in these agents. Without trust in the actions and results produced by these agents, their use and adoption as trusted assistants and partners will be severely limited. We present the results of a study among test users of CALO, one such complex adaptive agent system, to investigate themes surrounding trust and understandability. We identify and discuss eight major themes that significantly impact user trust in complex systems. We further provide guidelines for the design of trustable adaptive agents. Based on our analysis of these results, we conclude that the availability of explanation capabilities in these agents can address the majority of trust concerns identified by users.

As personal assistant software matures and assumes more autonomous control of user activities, it becomes more critical that this software can tell the user why it is doing what it is doing, and instill trust in the user that its task knowledge reflects standard practice and is being appropriately applied. Our research focuses broadly on providing infrastructure that may be used to increase trust in intelligent agents. In this paper, we will report on a study we designed to identify factors that influence trust in intelligent adaptive agents. We will then introduce our work on explaining adaptive task processing agents as motivated by the results of the trust study. We will introduce our task execution explanation component and provide examples in the context of a particular adaptive agent named CALO. Key features include (1) an architecture designed for re-use among different task execution systems; (2) a set of introspective predicates and a software wrapper that extracts explanation-relevant information from a task execution system; (3) a version of the Inference Web explainer for generating formal justifications of task processing and converting them to user-friendly explanations; and (4) a unified framework for explaining results from task execution, learning, and deductive reasoning.

A critical aspect of any explanation module is the set of user questions the system will be able to address. However, there has been relatively little work on listing and organizing the various categories of questions helpful to explanation. In this paper we address this problem by proposing a categorization of question types relevant to explaining task processing. For each question type, we also propose alternative explanation strategies for answering them. This categorization has helped to drive our work on an integrated cognitive explanation environment that has been used to explain the behavior of CALO, a software cognitive assistant that learns and organizes.

As personal assistant software matures and assumes more autonomous control of its users’ activities, it becomes more critical that this software can explain its task processing. It must be able to tell the user why it is doing what it is doing, and instill trust in the user that its task knowledge reflects standard practice and is being appropriately applied. We will describe the ICEE (Integrated Cognitive Explanation Environment) explanation system and its approach to explaining task reasoning. Key features include (1) an architecture designed for re-use among many different task execution systems; (2) a set of introspective predicates and a software wrapper that extract explanationrelevant information from a task execution system; (3) a version of the Inference Web explainer for generating formal justifications of task processing and converting them to userfriendly explanations; and (4) a unified framework for explanation in which the task explanation system is integrated with previous work on explaining deductive reasoning. Our work is focused on explaining belief-desire-intention (BDI) agent execution frameworks with the ability to learn. We demonstrate ICEE’s application within CALO, a state-of-the-art personal software assistant, to explain the task reasoning of one such

The Link Analysis Workbench (LAW) is a tool for detecting and monitoring situations of interest using inexact matching of graphical patterns. Here we describe some recent advances to LAW: incorporating hierarchy, cardinality, disjunction, and constraints in the pattern language and similarity metric, and a flexible, user-friendly interface for displaying matching data. These capabilities support analysts in rapidly exploring and understanding large, incomplete relational data sets.

Intelligence professionals work with incomplete and noisy data. Their information needs are often hard to express, and almost impossible to get right the first time. This paper describes the GEM pattern language for encoding analysts' information needs in graphical patterns, and its use in the Link Analysis Workbench (LAW) system to find inexact matches to those patterns in large relational data sets. The LAW user typically interacts with the system through a cycle in which the user (1) creates an initial GEM pattern corresponding to his information need, (2) uses the LAW matcher to retrieve a collection of matching episodes in the data, (3) revises the pattern based on the shortcomings of the matches, and (4) repeats the process until the revised pattern is returning the right data. The pattern language and the system are designed to facilitate the user in quickly traversing this cycle

To meet the intelligence community’s need for link analysis tools that work together, researchers are currently investigating ways of building workflows of these tools using an intelligent system architecture. A key challenge in building a dynamic link analysis workflow environment is representing the behavior of the individual link analysis algorithms being composed. In this paper, we outline techniques for modeling algorithms that allow a system architecture to reason about their behavior and performance, individually and in combination. The algorithm characterization model we propose is based on a layered approach, where the layers range from high-level qualitative descriptions of algorithms to detailed statistical descriptions of their effect on the data. Recent research and development in technology for intelligence analysis has produced a large number of tools, each of which addresses some aspect of the link analysis problem—the challenge of finding events, entities, and connections of interest in large relational data sets. Software developed in recent projects perform many diverse functions within link analysis, including detecting pre-defined patterns (Boner 2005; Coffman, Greenblatt, & Marcus 2004; Piochet al. 2004; Wolvertonet al. 2003), learning these patterns of interest (Holder et al. 2005), classifying individuals according to group membership (Adibi & Chalupsky 2005) or level of threat (Macskassy & Provost 2005), resolving aliases for individuals (Davis et al. 2005), identifying neighborhoods of interest within the data, and others. While these tools often perform complementary functions within the overall link analysis space, there has been limited success getting them to work together. One-time integration efforts have been time-consuming to engineer, and lack flexibility. To address this problem, a recent focus of research has been to link these tools together dynamically, through workflows composed by Grid software (Deelman et al. 2003), a blackboard system (Corkill 2003), or some other intelligent System Architecture (SA). One key challenge in building this kind of dynamic link analysis workflow environment is representing the behavior of the individual link analysis algorithms being composed. In this paper, we outline techniques for modeling algorithms that meet the requirements in the domain of link analysis. Copyright c © 2006, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. The algorithm model we propose is characterized by a layered approach, so that different System Architecture candidates can make use of some aspects of the model, even if they are not capable of reasoning about all of them. The layers range from high-level qualitative descriptions of algorithms to detailed statistical descriptions of their effect on the data. Below we outline the challenges in characterizing link analysis algorithms, describe our proposed algorithm characterization approach in more detail, and discuss related research in algorithm characterization and representing capabilities.

SRI’s LAW (Link Analysis Workbench) is a system that helps intelligence analysts detect occurrences of situations of interest by finding pattern instances in vast amounts of data using graph edit distance matching techniques. However to be completely successful it has to convey the results of the such findings to the users in a way that they can quickly grasp, not only to make use of it or to present it, but also to provide the feedback necessary to fine tune the discovery process using LAW. This paper presents what we think is required in order to achieve this goal, the solutions that were designed and implemented in the current LAW architecture and the approach that we envision in the next installments of LAW .