AbstractThis chapter examines expertise in complex and ill-structured domains from the perspective of cognitive flexibility theory (CFT). The emphasis is on adaptation under conditions of “ordinary novelty.” An approach to situational novelty via meta-features of an adaptive mindset that generalizes across cases in ways that content does not, and that fosters the skill of novel rearrangement of previously encountered case features in ways that are adaptive to new situations, is presented. The chapter describes CFT’s theoretical and empirical approach to combating rigidity and oversimplification, and to accelerating expertise in assembling “schemas of the moment” with learning based on the principled development and application of computer-supported case-based environments. Receiving special emphasis are many new modes of deliberate practice of adaptive flexibility. The chapter concludes with societal implications for today’s rapidly changing and increasingly complex world.

The growth of sophistication in machine capabilities must go hand in hand with growth of sophistication in human–machine interaction capabilities. To continue advancement as we build today’s intelligent machines, designers need formative tools for creating sociotechnical systems. In this article, we will briefly assess the appropriateness of “levels of automation” as a tool for designing human–machine systems. Additionally, we present coactive design and interdependence analysis as a viable alternative tool moving forward into more advanced and sophisticated human–machine systems.

We present a process theory describing how strategists form and stabilize mental models of causal relationships for the class of strategy problems that involve dynamic decision-making (DDM). Most of the problems strategists face are DDM problems that are characterized in terms of the following five dimensions of difficulty: (1) Interdependencies, (2) Time Delays, (3) Dynamics, (4) Nonlinearities, and (5) Stock and Flow Accumulation Processes. We describe how strategists exhibit a reductive bias (a descriptive, not a pejorative, term) by systematically making simplifying assumptions on one or more of these five dimensions when developing mental models of causal relationships. Once established, mental models become highly resistant to substantive modification as strategists engage a variety of cognitive maneuvers that protect their beliefs about focal problem features and causal relationships. These knowledge shields are more extensive and differentiated than simple confirmation biases or discounting, and emerge from goals to actively counter unsupporting information. Although simplified mental models are a necessary and often reasonable mechanism to cull complexity from problem deliberation, some dimensional reductions are harmful for the firm. Our process theory has important implications for numerous threads of strategy research, including: mental models and search, behavioral strategy, dynamic capabilities, and categories and competition.

In this article we describe how we apply the concept of coactive emergence as a phenomenon of complexity that has implications for the design of sensemaking support tools involving a combination of human analysts and software agents. We apply this concept in the design of work methods for distributed sensemaking in cybersecurity work. Sensemaking is a motivated, continuous effort to understand, anticipate, and act upon complex situations. We discuss selected results of a macrocognitive work analysis that informed our focus for design and development of support tools. In that analysis, we identified seven target topics that would be the focus of our research: engaging automation as a full partner, reducing the volume of uncorrelated events, continuous knowledge discovery, more effective visualizations, collaboration and sharing, minimizing tedious work, and architecting scalability and resilience. In addressing the first target topic, we show how coactive emergence inspires an agent-supported threat understanding process that is consistent with Klein’s Data/Frame theory of sensemaking. In subsequent sections, we describe our efforts to address the remaining six target topics as part of design and development of a cyber operations framework called Sol. Specifically, we describe the use of agents, policies, and visualization to enable coactive emergence for taskwork and teamwork. We also show how policy-governed agents working collaboratively with people can help in additional ways. We introduce the primary implementation frameworks that provide the core capabilities of our Sol cyber framework: the Luna Software Agent Framework and the KAoS Policy Services Framework. We describe results of initial studies addressing some of the issues raised in this article. Finally, we describe the status of Sol and plans for future development and evaluation.

In this paper, we describe a reconfigurable testbed for experiementation on joint activity in mixed human-agent-robot teamwork (HART). The testbed was originally inspired by the classic AI planning problem of Blocks World (BW) extended into what we call Blocks World for Teams (BW4T) [1] and now with more generality and power into RT4T, a Reconfigurable Testbed for Teams. By teams, we mean at least two, but usually more human , agent, or robot members. We describe the results of two experiments using BW4T, one showing the results of increasing autonomy without addressing interdependence and the other addressing soft interdependence as a performance factor. We introduce RT4T and a new teamwork measurement schema.

As counterpoint to the authors' previous discussions of the "seven deadly myths" of autonomous systems, here they present seven design principles to be understood and embraced for the virtues they engender.

Coactive Design is a new approach to address the increasingly sophisticated roles that people and robots play as the use of robots expands into new, complex domains. The approach is motivated by the desire for robots to perform less like teleoperated tools or independent automatons and more like interdependent teammates. In this article, we describe what it means to be interdependent, why this is important, and the design implications that follow from this perspective. We argue for a human-robot system model that supports interdependence through careful attention to requirements for observability, predictability, and directability. We present a Coactive Design method and show how it can be a useful approach for developers trying to understand how to translate high-level teamwork concepts into reusable control algorithms, interface elements, and behaviors that enable robots to fulfill their envisioned role as teammates. As an example of the coactive design approach, we present our results from the DARPA Virtual Robotics Challenge, a competition designed to spur development of advanced robots that can assist humans in recovering from natural and man-made disasters. Twenty-six teams from eight countries competed in three different tasks providing an excellent evaluation of the relative effectiveness of different approaches to human-machine system design.

In this paper we discuss the need for a command and control (C2) capability for moving target (MT) defenses. We describe some of the requirements and constraints associated with such a capability, and propose a human-agent teamwork approach for MTC2. We further discuss some specific concepts and technologies that could play an important role in the development of this capability, and conclude by describing some implementation details of a prototype being developed to demonstrate and study the proposed concepts for MTC2.

In this article, we describe how we augment human perception and cognition through Sol, an agent-based framework for distributed sensemaking. We describe how our visualization approach, based on IHMC's OZ flight display, has been leveraged and extended in our development of the Flow Capacitor, an analyst display for maintaining cyber situation awareness, and in the Parallel Coordinates 3D Observatory (PC3O or Observatory), a generalization of the Flow Capacitor that provides capabilities for developing and exploring lines of inquiry. We then introduce the primary implementation frameworks that provide the core capabilities of Sol: the Luna Software Agent Framework, the VIA Cross-Layer Communications Substrate, and the KAoS Policy Services Framework. We show how policy-governed agents can perform much of the tedious high-tempo tasks of analysts and facilitate collaboration. Much of the power of Sol lies in the concept of coactive emergence, whereby a comprehension of complex situations is achieved through the collaboration of analysts and agents working together in tandem. Not only can the approach embodied in Sol lead to a qualitative improvement in cyber situation awareness, but its approach is equally relevant to applications of distributed sensemaking for other kinds of complex high-tempo tasks.

The macrocognitive workplace is constantly changing, and a work system can never match its environment completely; there are always gaps in fitness because the work is itself a moving target. This article looks at a domain where the workplace is a moving target in three ways: cyberdefense. New technology and work methods are continually being introduced, domain constraints are not constant; the work itself is changing in terms of its new goals and requirements, and anything can be surprising. The article presents a possible sensemaking strategy and implications for the design of intelligent systems founded on human-machine interdependence, semantically rich policy governance, and having the goal of achieving resilience in the cognitive work.

There are several applications in which humans and agents jointly perform a task. If the task involves interdependence among the team members, coordination is required to achieve good team performance. This paper discusses the role of explanation in coordination in human-agent teams. Explanations about agent behavior for humans can improve coordination in human-agent teams for two reasons. First, with more knowledge about an agent’s actions and plans, humans can more easily adapt their own behavior to that of the agent. Second, with more insight in the reasons behind an agent’s behavior, humans will have more trust in the agents, and therefore more easily coordinate their actions. The paper also presents a study in the BW4T testbed that examines the effects of agents explaining their behavior on human-agent team performance. The results of this study show that explanations about agent behavior do not always lead to better team performance, but they do impact the user experience in a positive way.

There is a common belief that making systems more autonomous will improve the system and is therefore a desirable goal. Though small scale simple tasks can often benefit from automation, this does not necessarily generalize to more complex joint activity. When designing today's more sophisticated systems to work closely with humans, it is important not only to consider the machine's ability to work independently through autonomy, but also its ability to support interdependence with those involved in the joint activity. We posit that to truly improve systems and have them reach their full potential, designing systems that support interdependent activity between participants is the key. Our claim is that increasing autonomy, even in a simple and benign environment, does not always result in an improved system. We will show results from an experiment in which we demonstrate this phenomena and explain why increasing autonomy can sometimes negatively impact performance.

In this article, we explain our rationale for the development of Luna, a software agent framework. In particular, we focus on how we use capabilities for comprehensive policy-based governance to ensure that key requirements for security, declarative specification of taskwork, and built-in support for joint activity within mixed teams of humans and agents are satisfied. KAoS, IHMC's ontology-based policy services framework, enables the semantically-rich and extensible semantics and the operational power and flexibility needed to realize these capabilities within Luna. We show how Luna is specifically designed to allow developers and users to leverage different forms of policybased governance in an endless variety of ways.

In this article, we describe how we implement human-machine teamwork in Sol, a framework for cyber operations [3]. Specifically, we describe how the use of software agents (Luna), semantically rich policies (KAoS), and principles of visualization grounded in an understanding of human perception and cognition (OZ) can be used to support distributed sensemaking and effective response to cyber threats.

In this article we describe how we apply the concept of coactive emergence as a phenomenon of complexity that has implications for the design of sensemaking support tools involving a combination of human analysts and software agents. We apply this concept in the design of work methods for distributed sensemaking in cyber operations. Sensemaking is a motivated, continuous effort to understand, anticipate, and act upon complex situations. We discuss selected results of a macrocognitive work analysis that informed our focus for design and development of support tools. In that analysis, we identified seven target topics that would be the focus of our research: engaging automation as a full partner, reducing the volume of uncorrelated events, continuous knowledge discovery, more effective visualizations, collaboration and sharing, minimizing tedious work, and architecting scalability and resilience. In addressing the first target topic, we show how coactive emergence inspires an agent-supported threat understanding process that is consistent with Klein’s Data/Frame theory of sensemaking. In subsequent sections, we describe our efforts to address the remaining six target topics as part of design and development of a cyber operations framework called Sol. Specifically, we describe the use of agents, policies, and visualization to enable coactive emergence for taskwork and teamwork. We also show how policy-governed agents working collaboratively with people can help in additional ways. We introduce the primary implementation frameworks that provide the core capabilities of our Sol cyber framework: the Luna Software Agent Framework, and the KAoS Policy Services Framework. We describe areas for future development of Sol, including the incorporation of the VIA Cross-Layer Communications Substrate. Finally, we describe recent results and current plans for empirical studies addressing some of the issues raised in this article.

In this article, we outline the general concept of coactive emergence, an iterative process whereby joint sensemaking and decision-making activities are undertaken by analysts and software agents. Then we explain our rationale for the development of the Luna software agent framework. In particular, we focus on how we use capabilities for comprehensive policy-based governance to ensure that key requirements for security, declarative specification of taskwork, and built-in support for joint activity within mixed teams of humans and agents are satisfied.