We extend previous work on applying computational linguistics in understanding distributed sensemaking. In an experiment, teams of three respond to incidents in the C3Fire simulation under different levels of shared information, and radio communications were automatically transcribed (with an overall accuracy of around 80% for these recordings). The transcription was analyzed by computer to classify speech acts. We compared heuristics and regular expressions, supervised machine learning (using linear SVM), and unsupervised learning (using BERT). For this small corpus of utterances, SVM provides acceptable performance (around 79% accuracy) with minimal computational demand compared with the other approaches. In terms of team communication, differences in information conditions are identified (particularly in terms of speech acts relating to statements about “fire” and “rescue,” and statements about “reasoning and planning”). The study demonstrates the potential of automated analysis of team communications and indicates when teams might struggle with sensemaking.

A significant cost of system operation is the training required for the end users to achieve and maintain competence. The training need is partially determined by user interface properties that make it harder or easier to learn. This article explores improving system learnability through human-machine interface design. A design pattern of Progressive Disclosure through Layered Interfaces is proposed to improve learnability (time taken to learn) by presenting users with a subset of functionality. An experimental study supported tentative conclusions. Interesting differences between experiment participants suggest a customised approach, including Progressive Disclosure, to reduce the training burden.

In this paper, computational linguistics is applied to define and capture Speech Acts in distributed sensemaking in a military map-exercise. The exercise was performed by teams of three participants playing the role of Company Commanders (assisted by a team of confederates) using either text-only or a combination of voice and text communications, using either basic or elaborated reporting formats. Analysis of communications was performed in terms of network structure (i.e., who spoke with whom), topics (derived, using Latent Dirichlet Allocation, from all communications and from the speech of the Commanders), and speech acts reveals differences between conditions. It is proposed that the ‘topics’ uncovered through the analysis is analogous with the ‘frame’ in Data-Frame Model of sensemaking. In this way, it is possible to identify how frames are introduced, questioned and elaborated during communications in a team-based Command and Control exercise. Further, analyzing speech acts for different topics and under different conditions showed how different communication media or report formats can help or hinder team activity. It is proposed that such analysis could be viable for monitoring and supporting team communications, particularly during training exercises, but potentially as a route to providing automated decision and sensemaking support to teams.

Visual analytics tools integrate provenance recording to externalize analytic processes or user insights. Provenance can be captured on varying levels of detail, and in turn activities can be characterized from different granularities. However, current approaches do not support inferring activities that can only be characterized across multiple levels of provenance. We propose a task abstraction framework that consists of a three stage approach, composed of 1) initializing a provenance task hierarchy, 2) parsing the provenance hierarchy by using an abstraction mapping mechanism, and 3) leveraging the task hierarchy in an analytical tool. Furthermore, we identify implications to accommodate iterative refinement, context, variability, and uncertainty during all stages of the framework. We describe a use case which exemplifies our abstraction framework, demonstrating how context can influence the provenance hierarchy to support analysis. The article concludes with an agenda, raising and discussing challenges that need to be considered for successfully implementing such a framework.

During legal investigations, analysts typically create external representations of an investigated domain as resource for cognitive offloading, reflection and collaboration. For investigations involving very large numbers of documents as evidence, creating such representations can be slow and costly, but essential. We believe that software tools, including interactive visualisation and machine learning, can be transformative in this arena, but that design must be predicated on an understanding of how such tools might support and enhance investigator cognition and team-based collaboration. In this paper, we propose an approach to this problem by: (a) allowing users to visually externalise their evolving mental models of an investigation domain in the form of thematically organized Anchored Narratives; and (b) using such narratives as a (more or less) tacit interface to cooperative, mixed initiative machine learning. We elaborate our approach through a discussion of representational forms significant to legal investigations and discuss the idea of linking such representations to machine learning.

POLAR is a prototype data visualisation system developed for use by Intelligence Analysts conducting Patterns-of-Life (PoL) analysis. PoL analysis involves exploring data to identify behaviour patterns of individuals or groups. POLAR began with a series of requirements interviews with an ex-military analyst. The design process revealed a series of design alternatives that were explored. One of these related to whether, contrary to suggestions in the literature, animation might provide an advantage to the analyst. We report the design of the system and an evaluation study to assess the effect of animation on users. Whilst animation didn't result in significant improvements in performance, the use of animation with trails did result in significantly better usability assessments and higher user-engagement ratings. Post task interviews with analysts suggested where animation might offer a performance advantage.

Working Group Report in 'Provenance and Logging for Sense Making' report from Dagstuhl Seminar 18462: Provenance and Logging for Sense Making, Dagstuhl Reports, Volume 8, Issue 11

In the field of naturalistic decision making, the data–frame model (DFM) has proven to be a popular and useful way of thinking about sensemaking. DFM provides a parsimonious account of how ‘sensemakers’ interact with the data in their environment to make sense of what is happening. In this paper, however, we argue that it is useful to elaborate DFM in several ways. We begin by arguing for the idea of sensemaking as a quest for coherence, an idea that we see as consistent with the DFM. We then present some examples of sensemaking studies and use these to motivate a ‘distributed resources’ model of sensemaking. This model uses the notion of resources for action, as resources that can be flexibly drawn upon in both choosing courses of action and accounting for the actions of oneself and of others (as opposed to prescriptions or mechanisms that determine behaviour in any strict way). The model describes resources involved in sensemaking in terms of three domains: knowledge and beliefs, values and goals, and action. Knowledge and beliefs are concerned with how things are, values and goals are concerned with how things are desired to be and action provides the means for redressing the gap. Central to the model is the idea that these resources can be distributed across a cognitive work system including actors and representational media. Hence, the model aims to provide a framework for analysing sensemaking as distributed cognition.

This paper reports part of the results of an exploratory study that investigated how the use of external representations alters the process of sensemaking. The results show that there are significant correlations between the level of structuredness and the document triage process, in terms of its performance and its efficiency.

We report on findings from a 'state-of-practice' survey conducted with Interaction Design (IxD)/User Experience (UX) professionals called 'What's Hot in Interaction Design'. We focus on 20 items from the survey which elicited practitioners' usage of and attitudes towards personas. The survey items were derived from a review of academic and professional literature sources. The results show that practitioners think that personas have benefits, but come with associated resource demands and pitfalls, which we enumerate. We organize the results in terms of strength of opinion and discuss implications for methods, tools and curricula.

Presents a listing of VSAT conference reviewers.

This paper presents the preliminary results of our initial, descriptive, practical, hybrid argumentation model, designed for the use by criminal intelligence analysts (from now on referred to as analysts) working with sophisticated visual analytical software in uncertain sense-making environments. Analysts are required to create exhibits (as evidence) for a court of law or as input for decision-making in intelligence-led policing. These exhibits are required to be accurate, relevant and unbiased. Eight experienced criminal intelligence analysts from West Midlands police and the Belgium police evaluated a low-fidelity prototype resembling the first-order argumentation concepts of our initial argumentation model. The evaluation was to assess the applicability and practicality of the first-order argumentation concepts within our model. The preliminary results presented in this paper indicate that most of the first-order argumentation concepts are both applicable and practical and that the participants would use such concepts to construct their rationale from the onset of an analytical activity, if it were included as part of a software application.

The Criminal Intelligence Analyst's role is to create exhibits which are relevant, accurate and unbiased. Exhibits can be used as input to assist decision-making in intelligence-led policing. It may also be used as evidence in a court of law. The aim of this study was to determine how Criminal Intelligence Analysts recognise and manage significant information as a method to determine what is relevant for their attention and for the creation of exhibits. This in turn may provide guidance on how to design and incorporate loose and flexible argumentation schemas into sense-making software. The objective is to be informed on how to design software, which affords Criminal Intelligence Analysts with the ability to effortlessly determine the relevance of information, which subsequently could assist with the process of assessing and defending the quality of exhibits.

This paper reports the result of a study that we conducted to develop an instrument for measuring sensemaking and to understand how sensemakers conceptualise the sensemaking that they are doing. To address these aims we conducted a review of how sensemaking is described in the literature and derived a series of features. These were used to construct a questionnaire, which we deployed in an experimental study in which participants performed a sensemaking task. The results help to validate the questionnaire and provide insights into how users think about the experience of sensemaking.

As an explanation of sensemaking, data-frame theory has proven to be popular, influential and useful. Despite its strengths however, we propose some weaknesses in the way that the concept of a ‘frame’ could be interpreted. The weaknesses relate to a need to clearly contrast what we refer to as ‘generic’ vs. ‘situation-specific’ belief structures and the idea that multiple generic belief structures may be utilized in the construction of embedded situation-specific beliefs. Neither weakness is insurmountable, and we propose a model of sensemaking based on the idea of spreading activation through associative networks as a concept that provides a solution to this. We explore the application of this idea using the notion of activation to differentiate generic from situation specific beliefs.

The reconstruction of analysts' reasoning processes (reasoning provenance) during complex sensemaking tasks can support reflection and decision making. One potential approach to such reconstruction is to automatically infer reasoning from low-level user interaction logs. We explore a novel method for doing this using machine learning. Two user studies were conducted in which participants performed similar intelligence analysis tasks. In one study, participants used a standard web browser and word processor; in the other, they used a system called INVISQUE (Interactive Visual Search and Query Environment). Interaction logs were manually coded for cognitive actions based on captured think-aloud protocol and posttask interviews based on Klein, Phillips, Rall, and Pelusos's data/frame model of sensemaking as a conceptual framework. This analysis was then used to train an interaction frame mapper, which employed multiple machine learning models to learn relationships between the interaction logs and the codings. Our results show that, for one study at least, classification accuracy was significantly better than chance and compared reasonably to a reported manual provenance reconstruction method. We discuss our results in terms of variations in feature sets from the two studies and what this means for the development of the method for provenance capture and the evaluation of sensemaking systems.

Sensemaking has been described as a process involving information structuring. However, there are few detailed accounts of how this manifests in practice, particularly in relation to the creation and use of external representations such as data visualisations, and how such structuring aids sensemaking. To explore these questions in depth, we present an interview study of police crime analysts from which a model of their analysis process is developed. We describe the model focusing on the notion of 'think-steps', which for the analysts acted as a primary structuring concept. We describe how 'think-steps' propagate throughout the analysis process captured in the model. For the analysts, 'think-steps' are extensible templates that decompose a case into elements, provide a way of storing and visually structuring data, support generation of requests for information, focus research, simulate a case, and shape reporting. We reflect on the implications that our findings might have for design, including the possibility of a repertoire of evolving, sharable and reusable templates for sensemaking within a community of practice.

To analyze large amounts of data, visual analysis tools offer filter mechanisms for drilling down into multi-dimensional information spaces, or slicing and dicing them according to given criteria. This paper introduces an analysis approach for navigating multi-dimensional process instance execution logs based on business process models. By visually selecting parts of a business process model, a set of available log entries is filtered to include only those entries that result from execution instances of the selected process branches. Using this approach allows to exploratively navigate through process execution logs and analyze them according to the causal-temporal relationships encoded in the underlying business process model. The business process models used by the approach can either be created using model editors, or be statistically derived using process mining techniques. We exemplify our approach with a prototypical implementation.

This paper reports an empirical exploration of how different configurations of collaboration technology affect peoples' ability to construct and maintain common ground while conducting collaborative intelligence analysis work. Prior studies of collaboration technology have typically focused on simpler conversational tasks, or ones that involve physical manipulation, rather than the complex sensemaking and inference involved in intelligence work. The study explores the effects of video communication and shared visual workspace (SVW) on the negotiation of common ground by distributed teams collaborating in real time on intelligence analysis tasks. The experimental study uses a 2×2 factorial, between-subjects design involving two independent variables: presence or absence of Video and SVW. Two-member teams were randomly assigned to one of the four experimental media conditions and worked to complete several intelligence analysis tasks involving multiple, complex intelligence artefacts. Teams with access to the shared visual workspace could view their teammates' eWhiteboards. Our results demonstrate a significant effect for the shared visual workspace: the effort of conversational grounding is reduced in the cases where SVW is available. However, there were no main effects for video and no interaction between the two variables. Also, we found that the \"conversational grounding effort\" required tended to decrease over the course of the task.