Theatres and concert halls play a crucial role within the performing arts, where managerial and administrative staff are essential to bringing live performances to audiences. Existing AI research has focused on artistic creation, but less attention has been paid to the purposeful design of AI systems that support organisational practices. This paper addresses this gap by identifying the needs, challenges and opportunities for AI integration into everyday workflows, forming the basis for design principles to guide the architecturing, training, and deployment of AI systems that empower staff, rather than replace them. This is explored through a co-design workshop with theatre marketing and communication professionals. Through reflections of the themes explored in the workshop and by following the guiding principles, this paper presents examples of implementation of AI systems that could be adopted, offering concrete directions for developing AI that benefits the cultural sector.

This article represents our editorial contribution to the exploration of immersive storytelling, broadly conceived in this Special Issue as a set of creative and design practices, audience experiences, a field of study and a toolkit of rapidly developing digital technologies. Herein we set out our motivation for collaborating as an interdisciplinary team seeking to interrogate and expand on celebratory industry-led discourses that (over-)emphasise the transformative potential of digital innovations for immersive storytelling without properly situating these as part of longer trajectories of creative and audience practice. We provide an account of this background context that purposefully shifts the focus from technologies of the moment – Virtual Reality, the Metaverse, Artificial Intelligence – to the underpinning storytelling that can give rise to, as well as make use of, these technologies to facilitate immersion. Having introduced the various perspectives on immersive storytelling presented in the articles collected in this Special Issue, we go on to draw out key themes, topics and approaches from the body of work as a whole. Our analysis encompasses definitional questions, interdisciplinary perspectives and the sharing of expertise. It highlights the role of users as the nexus of technology and narrative, including emotional and sensory interactions, which contribute to their agency within immersive experiences. The discussion of immersive storytelling moves beyond a focus on VR to consider the wider context, including the overlap between real-world locations and narrative content of experiences, and the importance of setting audience expectations. Because, taken together, the articles include multiple projects and case studies involving designing and making immersive experiences, it is apparent that this is an expanded, rather than normalised, design space in which inclusion and exclusion are important considerations. From these findings we put forward a future agenda for the field of immersive storytelling, which revolves around issues of accessibility, ethics and audience research.

Although information about addressing the challenge of climate change is widely available, many people struggle in making sense of the actions they can take in the context of their everyday lives. We present Climate Club, a group-based role-playing card game designed to help people make sense of the climate actions they can take in relation to the barriers they face. Gameplay involves helping relatable fictional characters in actualising climate actions, by providing alternative solutions that work within their constraints. We describe the game and how it was shaped through iterative playtesting using the tandem transformational game design process. By reflecting on this process and a qualitative study conducted with university students exploring the game's features, capabilities and limitations, we reveal design insights that can be useful for researchers, designers and educators to design and evaluate games, and other purposeful HCI artefacts, for climate action.

General anxiety is a common mental health condition across the world, with many barriers preventing people from accessing treatments. A growing body of evidence suggests that serious games offer appealing and effective solutions for anxiety. Despite their potential, an understanding of game design qualities and contextual game mechanics in game-based interventions are significantly under-explored, especially for general anxiety. Alternative and emerging game genres may offer new intervention approaches previously overlooked in digital mental health research. We use story-led exploration games, particularly the environmental storytelling walking simulator game genre, as an analytical template to understand new game design strategies and features that can potentially enable engaging and efficacious serious games as interventions for general anxiety. Using semi-structured interviews and thematic analysis, we summarize input from expert clinical participants who identified and evaluated several game design qualities towards developing walking simulators as potential serious game interventions.

This paper explores how novel possibilities for personalisation and interactivity in video media, posed by developments in responsive video technology, can be applied to support patients as they make complex decisions about their health and care. We present the stakeholder-centred design of Exploring Your Options, a narrative-based responsive video developed to support young female patients in understanding and negotiating the fertility preservation options available to them following a cancer diagnosis. By reflecting on findings resulting from the process of designing and analysing Exploring Your Options with the involvement of cancer-experienced young people, health professionals and related stakeholders, we reveal opportunities for designing health information videos leveraging personalisation and interactivity to meet both informational and emotional support needs of patients, and highlight insights and considerations that can guide designers seeking to leverage these opportunities in ways that are sensitive and appropriate to the demands of complex healthcare contexts.

This chapter explains Augmented Reality (AR) and introduces innovative uses of AR in primary education: science is used as an example. One of the main advantages of using AR applications and platforms is the ability to display relevant digital information to support pupils' learning in real time and through specific contexts. The authors have found, through their research, that augmented reality offers exciting opportunities for pupils to digitally interact in ways traditional teaching methods cannot deliver. The chapter is a summary of ideas from Fearn's review of published research plus initial findings from his subsequent research.

Competitive video game playing, an activity called esports, is increasingly popular to the point that there are now many professional competitions held for a variety of games. These competitions are broadcast in a professional manner similar to traditional sports broadcasting. Esports games are generally fast paced, and due to the virtual nature of these games, camera positioning can be limited. Therefore, knowing ahead of time where to position cameras, and what to focus a broadcast and associated commentary on, is a key challenge in esports reporting. This gives rise to moment -to -moment prediction within esports matches which can empower broadcasters to better observe and process esports matches. In this work we focus on this moment -to -moment prediction and in particular present techniques for predicting if a player will die within a set number of seconds for the esports title Dota 2 . A player death is one of the most consequential events in Dota 2 . We train our model on 'telemetry' data gathered directly from the game itself, and position this work as a novel extension of our previous work on the challenge. We use an enhanced dataset covering 9,822 Dota 2 matches. Since the publication of our previous work, new dataset parsing techniques developed by the WEAVR project enable the model to track more features, namely player status effects, and more importantly, to operate in real time. Additionally, we explore two new enhancements to the original model: one data -based extension and one architectural. Firstly we employ learnt embeddings for categorical features, e.g. which in game character a player has selected, and secondly we explicitly model the temporal element of our telemetry data using recurrent neural networks. We find that these extensions and additional features all aid the predictive power of the model achieving an F1 score of 0.54 compared to 0.17 for our previous model (on the new data). We improve this further by experimenting with the length of the time -series in the input data and find using 15 time steps further improves the F1 score to 0.62. This compares to F1 of 0.1 for a standard RNN on the same task. Additionally a deeper analysis of the Time to Die model is carried out to assess its suitability as a broadcast aid.

It has previously been argued that Augmented Reality (AR) has the potential to provide relevant digital information to support pupils learning in real-time through engaging formats. However, the use of AR in the classroom remains uncommon for mainstream adoption. This study uses a service design methodology to focus on a more holistic approach toward AR as an educational service by investigating the challenges and conditions of integrating AR learning experiences into a primary curriculum. The research paper aims to provide an overview of how primary science (Key stage 2) in the UK is currently being delivered and where both barriers and opportunities occur for adoption to integrate more meaningful AR experiences that add real value to primary science education. An exploration stage (phase one) includes a questionnaire, contextual interviews, classroom observations, and focus groups. A thematic analysis was conducted on gathered data, identifying emerging patterns around teachers’ needs. The discussion section empathises with teachers’ needs and unearths key factors surrounding the implications of using augmented reality for primary science education.

This paper presents the findings of the Within the Walls of York Gaol, an interdisciplinary and collaborative practice-based research project which was established to examine the intersection between emerging forms of immersive media, game design, and contemporary archaeological exhibition design. The project was a collaboration between researchers and practitioners at York Museums Trust, The University of York, and the University of Glasgow and used co-design methods to design and produce two immersive installations in the prison cells at York Castle Museum in collaboration with community groups. The project sought to re-think the role of the museum within regional creative economies and to argue for the importance of the regional museum as a hub for digital design and innovation. Through innovative forms of collaborative research practice the project was able to develop new understandings of the ways in which collaborative approaches to the design and development of interactive digital media can be used to build skilled communities of practice around regional museums. The project took a critical stance to the use of immersive media technologies and led to the development of innovative and highly distinctive digital immersive installations which emerged in response to the specific opportunities and design requirements of museums and their audiences. This paper will set out the findings of this project as well as identify new creative directions and new forms of collaborative design practice which can be adopted and adapted for use across the museum sector.

This article presents how assemblage theory, as taken from Deleuze and Guattari, can be used to understand the intensive approaches of startup accelerators in supporting startup companies. Through a study of a startup accelerator in Jakarta, Indonesia, we present three snapshots to exemplify manifestations of what we argue as the accelerator’s “ seed accelerator” form of content and “ seed funding” form of expression as well as their reciprocal presupposition to demonstrate the multiplicity of assemblage as the organizational principles of the accelerator. Employing the tenets of formalization and territorialization from assemblage theory to analyze the results, this article shows that the “ seed accelerator” form of content is manifested by way of how the accelerator’s bodies of its human elements, activities, events, and infrastructure relate and interconnect throughout the accelerator’s 12-week program towards its end point, that is, fulfilling the stakes for the Final Demo-Day, while, on the other hand, the “ seed funding” form of expression is manifested by way of the usage of terms related to fund-raising, expressions of worry, and the expectations of the hub management and the VC in preparing the startups for the next level of funding. Moreover, we argue that the formalized function of the accelerator assemblage is to intensively seed scalable startups. This assemblage analysis thus offers an interrelational perspective regarding startup accelerators, and demonstrates the value of formalization and territorialization in assemblage theory to understand the programming arrangements in a startup accelerator.

Esport is a large and still growing industry with vast audiences. Multiplayer Online Battle Arenas (MOBAs), a sub-genre of esports, possess a very complex environment, which often leads to experts missing important coverage while broadcasting live competitions. One common game event that holds significant importance for broadcasting is referred to as a team fight engagement. Professional player's own knowledge and understanding of the game may provide a solution to this problem. This paper suggests a model that predicts and detects ongoing team fights in a live scenario. This approach outlines a novel technique of deriving representations of a complex game environment by relying on player knowledge. This is done by analysing the positions of the in-game characters and their associated cameras, utilising this data to train a neural network. The proposed model is able to both assist in the production of live esport coverage as well as provide a live, expert-derived, analysis of the game without the need of relying on outside sources.

In team sports, it can be a non-trivial task to compare players and understand how much they contribute and in what way, especially when those players are fulfilling significantly different roles on a team. What metric do you use to compare a Pitcher to a Batter? A Point Guard to a Center? Understanding what degree players are performing and how they compare to one another allows for greater depth in storytelling for the broadcasting team. It also provides audiences with overall performances across team positions.

Work in AI-based explanation systems has uncovered an interesting contradiction: people prefer and learn best from why explanations but expert esports commentators primarily answer what questions when explaining complex behavior in real-time strategy games. Three possible explanations for this contradiction are: 1.) broadcast audiences are well-informed and do not need why explanations; 2.) consuming why explanations in real-time is too cognitively demanding for audiences; or 3.) producing live why explanations is too difficult for commentators. We answer this open question by investigating the effects of explanation types and presentation modalities on audience recall and cognitive load in the context of an esports broadcast. We recruit 111 Dota 2 players and split them into three groups: the first group views a Dota 2 broadcast, the second group has the addition of an interactive map that provides what explanations, and the final group receives the interactive map with detailed why explanations. We find that participants who receive short interactive text prompts that provide what explanations outperform the no explanation group on a multiple-choice recall task. We also find that participants who receive detailed why explanations submit reports of cognitive load that are higher than the no explanation group. Our evidence supports the conclusion that informed audiences benefit from explanations but do not have the cognitive resources to process why answers in real-time. It also supports the conclusion that stacked explanation interventions across different modalities, like audio, interactivity, and text, can aid real-time comprehension when attention resources are limited. Together, our results indicate that interactive multimedia interfaces can be leveraged to quickly guide attention and provide low-cost explanations to improve intelligibility when time is too scarce for cognitively demanding why explanations.

The metagame is an overloaded term with no unified definition despite its importance and its common occurrence across different fields such as game design and behavioural economics. In our research we provide a unified and compact definition of the term metagame and metagaming by firstly highlighting their historical evolution. Although the original definition of metagame meant multiple things such as the environment surrounding the game, it has come to mean a perceived optimal or dominant playing strategy that is usually popular within an esport at that specific point in time. Metagaming as a verb is distinct and refers to a number of ways, external to the game's environment, a player can affect the outcome of a game. We focus on how these terms crystallised in the world of digital entertainment (esports) by providing multiple examples of metagames and metagaming in competitive settings. We additionally highlight the benefits of metagame shifts from the point of view of game developers. Finally, we provide a theoretical framework on the life cycles of metagames, as well general guidelines for understanding the current metagame of League of Legends and Dota 2. We conclude that by understanding the highly fluctuating metagame(s) of an esport at specific points in time, researchers will gain a better historical context of that game’s space. This in turn will give them insight into the decision making of professional esports players, game developers and tournament organisers. Additionally, this work will help researchers create better analytics tools and machine learning algorithms.

Unlike traditional physical sports, Esport games are played using wholly digital platforms. As a consequence, there exists rich data (in-game, audio and video) about the events that take place in matches. These data offer viable linguistic resources for generating comprehensible text descriptions of matches, which could, be used as the basis of novel text-based spectator experiences. We present a study that investigates if users perceive text generated by the NLG system as an accurate recap of highlight moments. We also explore how the text generated supported viewer understanding of highlight moments in two scenarios: i) text as an alternative way to spectate a match, instead of viewing the main broadcast; and ii) text as an additional information resource to be consumed while viewing the main broadcast. Our study provided insights on the implications of the presentation strategies for use of text in recapping highlight moments to Dota 2 spectators.

This paper introduces a generic framework for OBM storytelling. Aiming to function as a complete end-to-end reference for authoring OBM narrative content - from conception to realization - it proposes an integrated model that includes the three essential levels: conceptual, technological and aesthetic. At the conceptual level, we introduce a set of abstractions which provide a unified reference for thinking, describing and analysing interactive narrative structures of OBM content. Their recursive nature make our model stand out in terms of its power of expression. These abstractions have direct one-to-one operational counterparts implemented in our production-independent authoring toolkit – Cutting Room. This ensures that any specific story designs conceived within the proposed conceptual model are directly realisable as OBM productions. This isomorphic relationship between the abstract concepts and their operationalisation is another distinguishing aspect of our overall proposition. We have validated the model at the aesthetic level through the production of the interactive film What is Love?, experienced by over 900 people at the media art festival Mediale 2018 in York, UK, and evaluated through a dedicated questionnaire by 94 of them. As the foundations of OBM storytelling have not yet been established, we trust this paper constitutes a significant milestone in its development.

Multiplayer strategy games are examples of imperfect information games, where information about the game state can be retrieved through in-game mechanics. One such mechanic is vision. Within esports titles of this genre, such as League of Legends (LoL) and Dota 2, players often gather map information through the use of friendly units called wards. In LoL, one of the most popular esports title worldwide, warding has hitherto been evaluated only using a heuristic called vision score, provided by Riot, the game’s developer. In this paper, we examine the accuracy at LoL’s vision score at predicting the overall game-winner within the context supported by the game. We have ported LoL’s vision score to Dota 2, a similarly popular esports title, and compared its performance against a novel warding model. We have compared both models not only at predicting the overall winner, but also the current state of the game and their ability to predict and reflect short term game advantage and events. We found our model significantly outperformed LoL’s vision score. Additionally, we trained and evaluated a model for predicting the value of wards in real-time through the use of a Neural Network.

Esports (competitive videogames) have grown into a global phenomenon with over 450m viewers and a 1.5bn USD market. Esports broadcasts follow a similar structure to traditional sports. However, due to their virtual nature, a large and detailed amount data is available about in-game actions not currently accessible in traditional sport. This provides an opportunity to incorporate novel insights about complex aspects of gameplay into the audience experience – enabling more in-depth coverage for experienced viewers, and increased accessibility for newcomers. Previous research has only explored a limited range of ways data could be incorporated into esports viewing (e.g. data visualizations post-match) and only a few studies have investigated how the presentation of statistics impacts spectators’ experiences and viewing behaviors. We present Weavr, a companion app that allows audiences to consume data-driven insights during and around esports broadcasts. We report on deployments at two major tournaments, that provide ecologically valid findings about how the app’s features were experienced by audiences and their impact on viewing behavior. We discuss implications for the design of second-screen apps for live esports events, and for traditional sports as similar data becomes available for them via improved tracking technologies.

Brooke Leave Home is a personalized film designed to engage a non-expert audience with open data about the support young adults receive when leaving the care system in England. The film draws upon a range of video-based data storytelling techniques to present each viewer with a personalized perspective on the topic based on data from their own local area. We present the film's design and describe how its storytelling techniques were developed to support viewers in understanding, and fostering empathic connections with, the data sources featured and the implications they have for care leavers. We also present a study with 47 viewers, which explores how these techniques were experienced and how effective they were in aiding engagement with the data included and its meaning.