The ability of LLM-based agents to role-play may be especially helpful for gathering multiple perspectives in problem-solving situations. To understand how workers would respond to getting perspectives from agents, we conducted an interview study with 18 knowledge workers using a generative AI-powered multi-agent design probe. Participants described their current practices of perspective seeking from people, and outlined challenges with this practice including not knowing the right person to speak to and anxiety around reaching out. Participants reported finding value in the probe when it produced responses that aligned with their current solutions, and wanted sources or explanations for solutions they were unfamiliar with. They especially highlighted the need to be challenged by the probe, and were worried that the availability of such tools would reduce their interactions with colleagues. We discuss implications for designing for challenge, and locate agents as preparation tools for interacting with colleagues.

Writing is a well-established practice to support ideation and creativity. While Large Language Models (LLMs) have become ubiquitous in providing different kinds of writing assistance to different writers, LLM-powered writing systems often fall short in capturing the nuanced personalization and control necessary for effective support and creative exploration. To address these challenges, we introduce GhostWriter, an AI-enhanced writing design probe that enables users to exercise enhanced agency and personalization. GhostWriter leverages LLMs to implicitly learn the user's intended writing style for seamless personalization, while exposing explicit teaching moments for style refinement and reflection. We study 18 participants who use GhostWriter for editing and creative tasks, observing that it helps users craft personalized text and empowers them by providing multiple ways to steer system output. Based on this study, we present insights on people's relationships with AI-assisted writing and offer design recommendations to promote user agency in similar co-creative systems.

Large language models (LLMs) have become ubiquitous in providing different forms of writing assistance to different writers. However, LLM-powered writing systems often fall short in capturing the nuanced personalization and control needed to effectively support users – particularly for those who lack experience with prompt engineering. To address these challenges, we introduce GhostWriter, an AI-enhanced design probe that enables users to exercise enhanced agency and personalization during writing. GhostWriter leverages LLMs to implicitly learn the user's intended writing style for seamless personalization, while exposing explicit teaching moments for style refinement and reflection. We study 18 participants who use GhostWriter on two distinct writing tasks, observing that it helps users craft personalized text generations and empowers them by providing multiple ways to control the system's writing style. Based on this study, we present insights on how specific design choices can promote greater user agency in AI-assisted writing and discuss people's evolving relationships with such systems. We conclude by offering design recommendations for future work.

We present Enterprise Alexandria, a new system for automatically constructing a knowledge base with high-precision and typed entities from private enterprise data such as emails, documents and intranet pages. Built as an extension of Alexandria [Winn et al., 2019], the key novelty of Enterprise Alexandria is the ability in processing both the textual information and the structured metadata available in each document in an online learning fashion, making use of any manual curations that have happened in the interim. This task is performed entirely eyes-off to respect the privacy of the user and the restricted access their documents. The knowledge discovery process uses a probabilistic program defining the process of generating the data item from a set of unknown typed entities. Using probabilistic inference, Enterprise Alexandria can jointly discover a large set of entities with custom types specific to the organization. Experiments on three real-world datasets show that the system outperforms alternative methods with the ability to work effectively at large scale.

A system is provided that allows a user to visualize data. A dataset that includes a plurality of data items arranged in a two-dimensional (2D) format is received. A request to visualize the dataset in three dimensions is then received. A three-dimensional (3D) visualization of the dataset is then generated based on this request. This 3D visualization adds a third dimension to the 2D arrangement of data items, where the extent of the third dimension is commensurate with the type and value of the data items. This 3D visualization includes an initial depiction of the dataset which is generated from a point of view that is specified by this request. The initial depiction of the dataset is then displayed on a display device of the system in lieu of the dataset itself.

This forum provides a space to engage with the challenges of designing for intelligent algorithmic experiences. We invite articles that tackle the tensions between research and practice when integrating AI and UX design. We welcome interdisciplinary debate, artful critique, forward-looking research, case studies of AI in practice, and speculative design explorations. --- Juho Kim and Henriette Cramer, Editors

Current Machine Learning (ML) models can make predictions that are as good as or better than those made by people. The rapid adoption of this technology puts it at the forefront of systems that impact the lives of many, yet the consequences of this adoption are not fully understood. Therefore, work at the intersection of people's needs and ML systems is more relevant than ever. This area of work, dubbed Human-Centered Machine Learning (HCML), re-thinks ML research and systems in terms of human goals. HCML gathers an interdisciplinary group of HCI and ML practitioners, each bringing their unique, yet related perspectives. This one-day workshop is a successor of Gillies et al. 2016 CHI Workshop and focuses on recent advancements and emerging areas in HCML. We aim to discuss different perspectives on these areas and articulate a coordinated research agenda for the XXI century.

“Things We’ve Learnt About..” is a publication from the Human Experience & Design team that summarises our work around a specific theme in a way that we hope is interesting, insightful and inspirational. And most importantly, succinct.

‘HCI in the wild’ was meant to be a call to get HCI investigations out of the lab into the mêlée of real life. This is of course a commendable suggestion, though begs questions about what kinds of methods and topics are suited for exploring in this mêlée as against in the lab. Claims by some experimentalists that they seek ecological validity in lab studies are largely missing the point since the thing that studies in the wild seek are essentially only those things that occur outside the lab—and hence are not things that can be replicated, modelled, or emulated. But in any case, some of those who have taken up the call for studies in the wild have taken this rather too literally—they have sought wild places, places where HCI researchers have not gone before. Needless to say this being HCI, the places in question are not often that wild, woods near Brighton, for example, street life in south Cambridge. What they ignore as they venture into these settings is the mêlée of office life, the place where the bulk of computer systems are located and the place in which, oddly enough, increasingly little HCI research gets done.

Things We’ve Learnt About..” is a publication from the Human Experience & Design team that summarises our work around a specific theme in a way that we hope is interesting, insightful and inspirational. And most importantly, succinct.

“Things We’ve Learnt About..” is a publication from the Human Experience & Design team that summarises our work around a specific theme in a way that we hope is interesting, insightful and inspirational. And most importantly, succinct.

This article describes and reflects on the processes of designing two devices, Timecard and Fenestra, that both aim to propose new ideas for creating technologies that support rituals of honoring deceased loved ones. The discussion provides insight into how their respective designs were crafted to provide a range of interactions and to interweave with domestic practices, artifacts, and spaces; the article also describes the projects' similar strategies to supporting relationships with the deceased. Reflections then are offered about the design of future technologies aimed at supporting the processes both of adapting to the loss of loved ones and of honoring their continued evolving place in the lives of the living after they are gone.

Sophisticated ubiquitous sensing systems are being used to measure motor ability in clinical settings. Intended to augment clinical decision-making, the interpretability of the machine-learning measurements underneath becomes critical to their use. We explore how visualization can support the interpretability of machine-learning measures through the case of Assess MS, a system to support the clinical assessment of Multiple Sclerosis. A substantial design challenge is to make visible the algorithm's decision-making process in a way that allows clinicians to integrate the algorithm's result into their own decision process. To this end, we present a series of design iterations that probe the challenges in supporting interpretability in a real-world system. The key contribution of this article is to illustrate that simply making visible the algorithmic decision-making process is not helpful in supporting clinicians in their own decision-making process. It disregards that people and algorithms make decisions in different ways. Instead, we propose that visualisation can provide context to algorithmic decision-making, rendering observable a range of internal workings of the algorithm from data quality issues to the web of relationships generated in the machine-learning process.

Hybrid systems between biology and computation to study living organisms have demonstrated potential in promoting children's science experience and better understanding of their actions on the environment. However, these systems offer limited interactions between the user and the biological subject caused by inflexible equipment and missing possibilities to interfere with the biological subject through an interface. We present GrowKit, a digital/physical construction kit for living organisms that enables children to personalize their own experiments in biology. Our findings suggest that the comprehensive scaffolding offered by storytelling cards, experimental building blocks and remote lab software allows young learners to explore a broad range of biological ideas and conduct personally meaningful experiments, and promotes engagement and curiosity in children. We present GrowKit, a digital/physical construction toolkit for biology that provides young learners with playful STEM experience of designing, making, and conducting experiments.

Apparatus is described which has a memory configured to receive captured sensor data depicting at least one hand of a user operating the control system. The apparatus has a tracker configured to compute, from the captured sensor data, values of pose parameters of a three dimensional (3D) model of the hand, the pose parameters comprising position and orientation of each of a plurality of joints of the hand. A physics engine stores data about at least one virtual entity. The physics engine is configured to compute an interaction between the virtual entity and the 3D model of the hand based at least on the values of the pose parameters and data about the 3D model of the hand. A feedback engine is configured to trigger feedback to the user about the computed interaction, the feedback being any one or more of visual feedback, auditory feedback, haptic feedback.

With the proliferation of personal and social computing there is an increased interest in the field of human-computer interaction to support people's memory practises. Yet, there is only a limited understanding of the role of artefacts in the social dynamics in memory. With memory dialogue, we introduce a methodology for exploring artefact-based memory sharing. Participants created physical or digital memory artefacts, exchanged them, and reflected on the process. Our qualitative findings show how this method can help uncover the complexity of shared memory. Participants largely chose bonding experiences and created artefacts as conversation starters about differences in their memories.

Since the Bauhaus, industry and product design education have been intrinsically linked. A century later, industry collaborations still form a major component in product design education. In 2016, the opportunity arose for around 50 undergraduate students studying both Product and Digital Interaction Design (BSc) at the University of Dundee to take part in the 2016 Microsoft Research Design Expo. This global student competition challenged the students to explore ‘Symbiosis and the Conversational User Interface (CUI)’. Over the course of an 11-week semester, the students were divided into nine inter-disciplinary teams with one team later selected to disseminate their project at the annual Microsoft Faculty Summit conference in Seattle, USA. Through semi-structured interviews and focus groups with a sample of students, tutors and industry advisors, data was gathered to determine the influence of this collaboration on the total learning experience of all participants. In this paper, we briefly describe the background and context of the project before presenting a sample of student work. The paper then goes on to consider; the tension between competition and collaboration both within a team setting and the wider studio dynamic; how students interpret and incorporate input from both academic tutors and industry advisors; and the role prototypes play in the communication of ideas and concepts during the early stages of the design process. Reflecting on the major relationships and behaviours that all participants need to display, the paper concludes with a series of recommendations that we believe are essential for the design and delivery of future collaborative projects between industry and academia.

Fully articulated hand tracking promises to enable fundamentally new interactions with virtual and augmented worlds, but the limited accuracy and efficiency of current systems has prevented widespread adoption. Today's dominant paradigm uses machine learning for initialization and recovery followed by iterative model-fitting optimization to achieve a detailed pose fit. We follow this paradigm, but make several changes to the model-fitting, namely using: (1) a more discriminative objective function; (2) a smooth-surface model that provides gradients for non-linear optimization; and (3) joint optimization over both the model pose and the correspondences between observed data points and the model surface. While each of these changes may actually increase the cost per fitting iteration, we find a compensating decrease in the number of iterations. Further, the wide basin of convergence means that fewer starting points are needed for successful model fitting. Our system runs in real-time on CPU only, which frees up the commonly over-burdened GPU for experience designers. The hand tracker is efficient enough to run on low-power devices such as tablets. We can track up to several meters from the camera to provide a large working volume for interaction, even using the noisy data from current-generation depth cameras. Quantitative assessments on standard datasets show that the new approach exceeds the state of the art in accuracy. Qualitative results take the form of live recordings of a range of interactive experiences enabled by this new approach.

Within the domain of wearables, our paper explores opportunities for self-expression, learning about the body, and interactions with others that are enabled through the physical and interactive properties of a new kind of digital display. Recent advances in the manufacturing of thin, bendable electronics and a novel architecture for arranging pixels permit the fabrication of a 'display material' that challenges conventional perceptions of this medium. Envisioning digital displays as a material means that their design is no longer limited to rectangular screens, but can be scaled in size; cut, folded or molded; or combined with other materials and devices to create compelling interactions. In this paper, we explore some of this potential in the context of configuring body-worn items such as clothing. As a more concrete use example, we further present and discuss some initial ideas for aesthetic, expressive and functional configurations of body casts as a specific kind of body cover.