Webstrates are web substrates, a practical realization of shareable dynamic media under which distributability, shareability, and malleability are fundamental software principles. Webstrates blur the distinction between application and document in a way that enables users to share, repurpose, and refit software across a variety of domains, but its reliance on a central server constrains its use; it is at odds with personal and collective control of data; and limits applications to the web. We extend the fundamental principles to include interoperability and sovereignty over data and propose MyWebstrates, an implementation of Webstrates on top of a new, lower-level substrate for synchronization built around local-frst software principles. MyWebstrates registers itself in the user's browser and function as a piece of local software that can selectively synchronise data over sync servers or peer-to-peer connections. We show how MyWebstrates extends Webstrates to enable offline collaborative use, interoperate between Webstrates on non-web technologies such as Unity, and maintain personal and collective sovereignty over data. We demonstrate how this enables new types of applications of Webstrates and discuss limitations of this approach and new challenges that it reveals.

Artificial intelligence (AI) has the potential to bring significant benefits to highly regulated industries such as healthcare or banking. Adoption, however, remains low. AI’s entry into complex socio-techno-legal systems raises issues of transparency, specifically for regulators. However, the perspective of supervisors, regulators who monitor compliance with applicable financial regulations, has rarely been studied. This paper focuses on understanding the needs of supervisors in anti-money laundering (AML) to better inform the design of AI justifications and explanations in highly regulated fields. Through scenario-based workshops with 13 supervisors and 6 banking professionals, we outline the auditing practices and socio-technical context of the supervisor. By combining the workshops’ insights with an analysis of compliance requirements, we identify the AML obligations that conflict with AI opacity. We then formulate seven needs that supervisors have for model justifiability. We discuss the role of explanations as reliable evidence on which to base justifications.

Robo-advisors are democratizing access to life-insurance by enabling fully online underwriting. In Europe, financial legislation requires that the reasons for recommending a life insurance plan be explained according to the characteristics of the client, in order to empower the client to make a "fully informed decision". In this study conducted in France, we seek to understand whether legal requirements for feature-based explanations actually help users in their decision-making. We conduct a qualitative study to characterize the explainability needs formulated by non-expert users and by regulators expert in customer protection. We then run a large-scale quantitative study using Robex, a simplified robo-advisor built using ecological interface design that delivers recommendations with explanations in different hybrid textual and visual formats: either "dialogic"—more textual—or "graphical"—more visual. We find that providing feature-based explanations does not improve appropriate reliance or understanding compared to not providing any explanation. In addition, dialogic explanations increase users' trust in the recommendations of the robo-advisor, sometimes to the users' detriment. This real-world scenario illustrates how XAI can address information asymmetry in complex areas such as finance. This work has implications for other critical, AI-based recommender systems, where the General Data Protection Regulation (GDPR) may require similar provisions for feature-based explanations.

Explainability (XAI) has matured in recent years to provide more human-centered explanations of AI-based decision systems. While static explanations remain predominant, interactive XAI has gathered momentum to support the human cognitive process of explaining. However, the evidence regarding the benefits of interactive explanations is unclear. In this paper, we map existing findings by conducting a detailed scoping review of 48 empirical studies in which interactive explanations are evaluated with human users. We also create a classification of interactive techniques specific to XAI and group the resulting categories according to their role in the cognitive process of explanation: "selective", "mutable" or "dialogic". We identify the effects of interactivity on several user-based metrics. We find that interactive explanations improve perceived usefulness and performance of the human+AI team but take longer. We highlight conflicting results regarding cognitive load and overconfidence. Lastly, we describe underexplored areas including measuring curiosity or learning or perturbing outcomes.

Computational media describes a vision of software, which, in contrast to application-centric software, is (1) malleable, so users can modify existing functionality, (2) computable, so users can run custom code, (3) distributable, so users can open documents across different devices, and (4) shareable, so users can easily share and collaborate on documents. Over the last ten years, the Webstrates and Codestrates projects aimed at realizing this vision of computational media. Webstrates is a server application that synchronizes the DOM of websites. Codestrates builds on top of Webstrates and adds an authoring environment, which blurs the use and development of applications. Grounded in a chronology of the development of Webstrates and Codestrates, we present eight tensions that we needed to balance during their development. We use these tensions as an analytical lens in three case studies and a game challenge in which participants created games using Codestrates. We discuss the results of the game challenge based on these tensions and present key takeaways for six of them. Finally, we present six lessons learned from our endeavor to realize the vision of computational media, demonstrating the balancing act of weighing the vision against the pragmatics of implementing a working system.

The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to complex AI systems. Although it is usually considered an essentially technical field, effort has been made recently to better understand users' human explanation methods and cognitive constraints. Despite these advances, the community lacks a general vision of what and how cognitive biases affect explainability systems. To address this gap, we present a heuristic map which matches human cognitive biases with explainability techniques from the XAI literature, structured around XAI-aided decision-making. We identify four main ways cognitive biases affect or are affected by XAI systems: 1) cognitive biases affect how XAI methods are designed, 2) they can distort how XAI techniques are evaluated in user studies, 3) some cognitive biases can be successfully mitigated by XAI techniques, and, on the contrary, 4) some cognitive biases can be exacerbated by XAI techniques. We construct this heuristic map through the systematic review of 37 papers-drawn from a corpus of 285-that reveal cognitive biases in XAI systems, including the explainability method and the user and task types in which they arise. We use the findings from our review to structure directions for future XAI systems to better align with people's cognitive processes.

The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to complex AI systems. Although it is usually considered an essentially technical field, effort has been made recently to better understand users' human explanation methods and cognitive constraints. Despite these advances, the community lacks a general vision of what and how cognitive biases affect explainability systems. To address this gap, we present a heuristic map which matches human cognitive biases with explainability techniques from the XAI literature, structured around XAI-aided decision-making. We identify four main ways cognitive biases affect or are affected by XAI systems: 1) cognitive biases affect how XAI methods are designed, 2) they can distort how XAI techniques are evaluated in user studies, 3) some cognitive biases can be successfully mitigated by XAI techniques, and, on the contrary, 4) some cognitive biases can be exacerbated by XAI techniques. We construct this heuristic map through the systematic review of 37 papers-drawn from a corpus of 285-that reveal cognitive biases in XAI systems, including the explainability method and the user and task types in which they arise. We use the findings from our review to structure directions for future XAI systems to better align with people's cognitive processes.

AbstractAffinity diagramming is widely applied to analyze qualitative data such as interview transcripts. It involves multiple analytic processes and is often performed collaboratively. Drawing on interviews with three practitioners and upon our own experience, we show how practitioners combine multiple analytic processes and adopt different artifacts to help them analyze their data. Current tools, however, fail to adequately support mixing analytic processes, devices, and collaboration styles. We present a vision and prototype ADQDA, a cross-device, collaborative affinity diagramming tool for qualitative data analysis, implemented using distributed web technologies. We show how this approach enables analysts to appropriate available pertinent digital devices as they fluidly migrate between analytic phases or adopt different methods and representations, all while preserving consistent analysis artifacts. We validate this approach through a set of application scenarios that explore how it enables new ways of analyzing qualitative data that better align with identified analytic practices.

We present a framework for defining the level of explainability based on technical, legal and economic considerations. Our approach involves three logical steps: First, define the main con-textual factors, such as who is the audience of the explanation, the operational context, the level of harm that the system could cause, and the legal/regulatory framework. This step will help characterize the operational and legal needs for explanation, and the corresponding social benefits. Second, examine the technical tools available, including post-hoc approaches (input perturbation, saliency maps...) and hybrid AI approaches. Third, as function of the first two steps, choose the right levels of global and local explanation outputs, taking into the account the costs involved. We identify seven kinds of costs and emphasize that explanations are socially useful only when total social benefits exceed costs.

The recent enthusiasm for artificial intelligence (AI) is due principally to advances in deep learning. Deep learning methods are remarkably accurate, but also opaque, which limits their potential use in safety-critical applications. To achieve trust and accountability, designers and operators of machine learning algorithms must be able to explain the inner workings, the results and the causes of failures of algorithms to users, regulators, and citizens. The originality of this paper is to combine technical, legal and economic aspects of explainability to develop a framework for defining the "right" level of explain-ability in a given context. We propose three logical steps: First, define the main contextual factors, such as who the audience of the explanation is, the operational context, the level of harm that the system could cause, and the legal/regulatory framework. This step will help characterize the operational and legal needs for explanation, and the corresponding social benefits. Second, examine the technical tools available, including post hoc approaches (input perturbation, saliency maps...) and hybrid AI approaches. Third, as function of the first two steps, choose the right levels of global and local explanation outputs, taking into the account the costs involved. We identify seven kinds of costs and emphasize that explanations are socially useful only when total social benefits exceed costs.

Data workers are people who perform data analysis activities as a part of their daily work but do not formally identify as data scientists. They come from various domains and often need to explore diverse sets of hypotheses and theories, a variety of data sources, algorithms, methods, tools, and visual designs. Taken together, we call these alternatives. To better understand and characterize the role of alternatives in their analyses, we conducted semi-structured interviews with 12 data workers with different types of expertise. We conducted four types of analyses to understand 1) why data workers explore alternatives; 2) the different notions of alternatives and how they fit into the sensemaking process; 3) the high-level processes around alternatives; and 4) their strategies to generate, explore, and manage those alternatives. We find that participants' diverse levels of domain and computational expertise, experience with different tools, and collaboration within their broader context play an important role in how they explore these alternatives. These findings call out the need for more attention towards a deeper understanding of alternatives and the need for better tools to facilitate the exploration, interpretation, and management of alternatives. Drawing upon these analyses and findings, we present a framework based on participants' 1) degree of attention, 2) abstraction level, and 3) analytic processes. We show how this framework can help understand how data workers consider such alternatives in their analyses and how tool designers might create tools to better support them.

Data workers are non-professional data scientists who engage in data analysis activities as part of their daily work. In this position paper, we draw on our past experience in studying their data analysis processes and workflows, and the tools we built to support sensemaking. We describe our background as computer scientists and our multidisciplinary approach. Finally, we conclude with open questions and research directions, and argue for more research into the challenges faced by data workers.

Data workers are non-professional data scientists who engage in data analysis activities as part of their daily work. In this position paper, we share past and on-going work to understand data workers’ sense-making practices. We use multidisciplinary approaches to explore their human-tool partnerships. We introduce our current research on the role of alternatives in data analysis activities. Finally, we conclude with open questions and research directions.

Webstrates presents an alternative take on the future of web use and development. In Webstrates, changes to the Document Object Model (DOM) of webpages (called webstrates) are persisted across reloads and synchronized to other clients of the same webstrate. This includes changes to embedded JavaScript and CSS. With Webstrates we demonstrate one possible direction for the future web that we hope can inspire and foster discussion at the ProWeb 2017 workshop. We will gladly demonstrate Webstrates at the workshop and illustrate best programming practices for it. Further, we would like to discuss open issues (e.g. programming language support, collaborative programming, fine-grained permis- sions, handling large datasets, and other advances in the future of web programming) from which we believe future versions of Webstrates and its community will greatly benefit.

Uncertainty plays an important and complex role in data analysis, where the goal is to find pertinent patterns, build robust models, and support decision making. While these endeavours are often associated with professional data scientists, many domain experts engage in such activities with varying skill levels. To understand how these domain experts (or "data workers") analyse uncertain data we conducted a qualitative user study with 12 participants from a variety of domains. In this paper, we describe their various coping strategies to understand, minmise, exploit or even ignore this uncertainty. The choice of the coping strategy is influenced by accepted domain practices, but appears to depend on the types and sources of uncertainty and whether participants have access to support tools. Based on these findings, we propose a new process model of how data workers analyse various types of uncertain data and conclude with design considerations for uncertainty-aware data analytics.

We introduce the grab ‘n’ drop toolglass, an extension of the toolglass bi-manual interaction technique. It enables users to create and configure their own toolglasses from existing user interfaces that were not designed for toolglasses. Users compose their own toolglass interactions at runtime from an application’s user interface elements, bringing interaction closer to the objects of interest in a workspace. Through a proof-of-concept implementation for Mac OS X, we show how grab ‘n’ drop capabilities could be added to existing applications at the toolkit level, without modifying application source code or UI design. Finally, we evaluate the power and flexibility of this approach by applying it to a variety of applications. We further identify limitations and risks associated with this approach and propose changes to existing toolkits to foster such user-reconfigurable interaction.

We introduce Codestrates, a literate computing approach to developing interactive software. Codestrates blurs the distinction between the use and development of applications. It builds on the literate computing approach, commonly found in interactive notebooks such as Jupyter notebook. Literate computing weaves together prose and live computation in the same document. However, literate computing in interactive notebooks are limited to computation and it is challenging to extend their user interface, reprogram their functionality, or develop stand-alone applications. Codestrates builds literate computing capabilities on top of Webstrates and demonstrates how it can be used for (i) collaborative interactive notebooks, (ii) extending its functionality from within itself, and (iii) developing reprogrammable applications.

Existing user interface toolkits are based on a single user interacting with a single machine with a relatively fixed set of input devices. Today's interactive systems, however, can involve multiple users interacting with a heterogeneous set of input, computational, and output capabilities across a dynamic set of different devices. The abstractions that help programmers create interactive software for one kind of system do not necessarily scale to these new kinds of environments. New toolkits designed around these environments, however, need to be able to bridge existing software and libraries or recreate them from scratch. In this position paper, we examine these new constraints and needs. We look at three strategies for software toolkits that help to bridge existing toolkit models to these new interaction paradigms.