This paper explores how technology can replicate equine-assisted services without requiring physical interaction with horses. Human-horse interaction is utilised in equine-assisted therapy to improve mental wellbeing; however, access to such interventions is limited due to the high horses’ cost and maintenance. Horses possess advanced emotional intelligence, interpreting human gestures and non-verbal cues to respond adaptively, often mirroring human emotions. By modelling these capabilities in a virtual environment using Artificial Intelligence (AI) and Machine Learning techniques, users can access the replicated services while interacting with virtual horses. A system is designed to detect participants’ emotional states reflected in their motion via motion analysis and integrate these cues into an interactive virtual reality (VR) experience to foster calmness and improve communication skills. An experiment was conducted to train an emotion-recognition model for replicating horse-human interaction in VR. Results demonstrate the system’s accuracy in detecting emotional states and the findings suggest the potential of VR-based equine-assisted services to replicate the real ones to enhance accessibility and engagement with such services.

In natural dialogues and interactive scenarios, real-time motion generation for virtual characters faces challenges: traditional systems relying on precomputed motion libraries exhibit latency, rigid body language, and inadequate multimodal synchronization, limiting emotional engagement and immersion. We propose a hierarchical acceleration framework integrating multimodal neural networks to address these issues. The framework employs knowledge distillation to transfer motion generation expertise from a teacher model into a lightweight diffusion backbone. Our approach enables 8-step full-body motion inference and 1-step facial synthesis via diffusion-GAN fine-tuning. The distillation loss combines kinematic feature matching and output distribution alignment. Building upon this acceleration framework, we implement a dual-stream network architecture that decouples facial expression and motion generation for cross-modal coupling, directly addressing the multimodal synchronization challenges identified earlier. Experimental results demonstrate that our framework achieves real-time performance with high naturalness and performs better than existing emotional expressiveness and multimodal synchronization methods. The system supports applications including virtual social interactions, gamified entertainment, and educational training, contributing to advancing virtual character technology for enhanced human-computer interactions.

Studying human behaviour and mind in realistic environments has been a challenge in various research fields. We formulate the foundational elements of a methodological framework for the multi-modal research of social interaction in virtual reality (VR) based on two experiments involving human participants interacting with virtual humans. The framework comprises commonly used VR headsets and neurophysiological sensors to measure and evaluate participants’ behaviours and brain activities. We provide guidelines related to methodological solutions, data collection and analysis pipelines.

This paper provides an overview of a fully implemented Virtual Reality (VR) application which aims to support the assessment of the most common subtypes of Obsessive Compulsive Disorder (OCD): cleaning and checking behaviours. The VR application consists of a tool for the therapist to select differing levels of tasks for the participants to fulfil within an interactive 3D virtual kitchen. Participants’ assessment will be taking place at tasks completion level whilst their behaviour will be analysed through continuous recording of physiological measures and self-reporting questionnaires.

Online game playing of youth in China, especially their problematic online gaming (POG), has become one of the social issues that affects large numbers of people and their families. However, studies about the impact of player’s motivation on problematic playing are sparse and lack systematic approaches. Our current study is aimed at investigating the relationship between gaming motivations and POG. This paper presents the results of a large-scale survey conducted in China with 1557 participants, of whom 1358 (87.2%) were male. A multiple regression analysis with 10 game motivations as predictors has been performed to explore which factors have effects on game addiction. It is shown that the best predictors of game addiction are the escapism motivation, followed by the competition motivation and then the advancement motivation. The mediation effect of demographic variables on the relationships between player’s motivations and game addiction is further examined using the casual steps, and a significant mediating effect of age on game addiction is revealed. The POG differences across gender and age were also examined. The findings enable a better understanding of the underlying mechanics of POG and to minimize the risks and maximise the positive impact of games on society.

Our research studies early face-body perception of socially anxious individuals during social interaction with a virtual agent (VA) in VR using EEG. VAs’ expressiveness is manipulated during social interactions through their facial animations portraying realistic positive/negative/neutral expressions. Facial expressions of the VAs are recorded using real-time facial animation performance capture and pre-validated. Wearing an HMD, tactile-based VR controllers and a mobile EEG system, participants will interact with individual VAs in a virtual office setting (an employee meeting their employer with a handshake), a scenario known to generate anxiety. Behavioural, physiological, and EEG data will be analysed to reveal the effect of emotional valence on early face-body perception during social interactions. This study provides a framework for synchronised multimodal data recording and analysis.

Recent advances in visualisation technologies have opened up new possibilities for human-agent communication. For systems where agents use automated planning, visualisation of agent intentions, i.e., agent planned actions, can assist human understanding and decision making (e.g., deciding when human control is required or when it can be delegated to an agent). We are working in an application area, shipbuilding, where branched plans are often essential, due to the typical uncertainty experienced. Our focus is how best to communicate, using visualisation, the key information content of branched plans. It is important that such visualisations communicate the complexity and variety of the possible agent intentions i.e., executions, captured in a branched plan, whilst also connecting to the practitioner's understanding of the problem. Thus we utilise an approach to generate the complete branched plan, to be able to provide a full picture of its complexity, and a mechanism to select a subset of diverse traces that characterise the possible agent intentions. We have developed an interface which uses 3D visualisation to communicate details of these characterising execution traces. Using this interface, we conducted a study evaluating the impact of different modes of presentation on user understanding. Our results support our expectation that visualisation of branched plan characterising execution traces increases user understanding of agent intention and plan execution possibilities.

Virtual agents interact with each other through dialogues in various types of narratives (e.g. narrative films). In this paper, we propose an approach on the basis of DialoGPT pre-trained language model, which explores the impact of dialogue generation with different levels of agents' personalities derived from narrative films based on the Big-Five model, as well as with three different embedding methods. From the experimental results using automatic metrics and human judgments, we investigate and analyze the impact of different settings on narrative dialogue generation. Also, we demonstrate that our approach is able to generate dialogues with increased variety that correctly reflect the corresponding target personality.

Gambling has been on the rise over the past years and understanding different patterns of the human behavior while gambling involves the identification of the emotions experienced while gambling, as well as how these change during a gambling activity. This work attempts to address these components towards the creation of a computational model of gambling experience. Specifically, we created a gambling game (roulette) and evaluated the interaction of participants with the game by assessing their emotional responses using the video modality. This work provides the basis for developing a multimodal interface that can help capturing the gambling experience. Within our research we attempt to answer the following research questions: (a) which are the emotions experienced by someone gambling and (b) how do the emotions detected change before and after an event.

Social anxiety disorder has been widely recognised as one of the most commonly diagnosed mental disorders. Individuals with social anxiety disorder experience difficulties during social interactions that are essential in the regular functioning of daily routines; perpetually motivating research into the aetiology, maintenance and treatment methods. Traditionally, social and clinical neuroscience studies incorporated protocols testing one participant at a time. However, it has been recently suggested that such protocols are unable to directly assess social interaction performance, which can be revealed by testing multiple individuals simultaneously. The principle of two-person neuroscience highlights the interpersonal aspect of social interactions that observes behaviour and brain activity from both (or all) constituents of the interaction, rather than analysing on an individual level or an individual observation of a social situation. Therefore, two-person neuroscience could be a promising direction for assessment and intervention of the social anxiety disorder. In this paper, we propose a novel paradigm which integrates two-person neuroscience in a neurofeedback protocol. Neurofeedback and interbrain synchrony, a branch of two-person neuroscience, are discussed in their own capacities for their relationship with social anxiety disorder and relevance to the paradigm. The newly proposed paradigm sets out to assess the social interaction performance using interbrain synchrony between interacting individuals, and to employ a multi-user neurofeedback protocol for intervention of the social anxiety.

Football (or Soccer) 1 matches are often analysed from 2D video footage that lacks any spatial details of the players on the pitch. On the other hand, the real-time positions of the players derived from recorded tracking data are not easily understandable without an immersive and effective visualisation of the players located on a pitch with the ability of the system to review actions over a timeline. To address some of these drawbacks, we propose a novel "Believable Agent Behaviour" (BAB) engine, based on virtual reality equipped with artificial intelligence for the generation of realistic virtual agent animations to enact the data-driven events from football matches. This system implements an immersive, interactive, believable, and automated end-to-end simulation to analyse and replay a football match using immersive 3D graphics.

The technology supporting Interactive Digital Narrative (IDN) is of particular significance to cultural heritage research. IDN technology provides a means of engagement in cultural heritage sites, a medium for culturally significant stories, and culturally significant story-centric games. While previous work in this space has numerous examples of user experience (UX) evaluations of the interactive narrative works themselves, there is significantly less in terms of evaluation of technology for authoring IDN, creating a UX research space in this area that is focused on the audience and not authors. We propose to balance this focus by considering the UX of authoring tools more closely. In this work, we undertake a review of the state of the art of authoring tools for IDN, such as story-centric games, and report on a rigorous UX evaluation of representative technologies (n = 21). We also address the challenges of UX research for these tools through an original evaluation methodology where authors complete a story composed of representative story features. Our study leads us to conclude seven UX principles for IDN authoring tools that explore both how authors use tools to create story-focused games and how the interface for these tools impacts the creative process.

Recent advances in visualisation technologies have opened up new possibilities for human-agent communication. For systems where agents use automated planning, visualisation of agent planned actions can play an important role in allowing human users to understand agent intent and to help decide when control can be delegated to the agent or when they need to be involved. We are interested in application areas where branched plans are required, due to the typical uncertainty experienced. Our focus is how best to communicate, using visualisation, the key information content of a branched plan. It is important that such visualisations communicate the complexity and variety of the possible executions captured in a branched plan, whilst also connecting to the practitioner's understanding of the problem. Thus we have developed an approach that: generates the complete branched plan, to be able to provide a full picture of its complexity; a mechanism to select a subset of diverse traces that characterise the possible executions; and an interface that uses 3D visualisation to communicate details of these characterising execution traces to practitioners. Using this interface, we conducted a study evaluating the impact of different modes of presentation on user understanding. Our results support our expectation that visualisation of characterising branched plan execution traces increases user understanding of agent intention and range of plan execution possibilities.

In this paper we present details of a virtual tour and game for VR headset that are designed to investigate an interactive and engaging approach of applying VR to student recruitment for an undergraduate course. The VR tour employs a floating menu to navigate through a set of 360° panoramic photographs of the teaching environment and uses hotspot interaction to display further information about the course. The VR game is a fast-paced shooting game. The course information is embedded on cubes that the player needs to focus on and destroy. The game experience is expected to generate an engaging way to promote the course. This work in progress outlines the concept and development of the prototype, and discusses the next stages of testing in order to evaluate the effectiveness of applying VR to undergraduate student recruitment.

Prefrontal cortex (PFC) asymmetry is an important marker in affective neuroscience and has attracted significant interest, having been associated with studies of motivation, eating behavior, empathy, risk propensity, and clinical depression. The data presented in this paper are the result of three different experiments using PFC asymmetry neurofeedback (NF) as a Brain-Computer Interface (BCI) paradigm, rather than a therapeutic mechanism aiming at long-term effects, using functional near-infrared spectroscopy (fNIRS) which is known to be particularly well-suited to the study of PFC asymmetry and is less sensitive to artifacts. From an experimental perspective the BCI context brings more emphasis on individual subjects' baselines, successful and sustained activation during epochs, and minimal training. The subject pool is also drawn from the general population, with less bias toward specific behavioral patterns, and no inclusion of any patient data. We accompany our datasets with a detailed description of data formats, experiment and protocol designs, as well as analysis of the individualized metrics for definitions of success scores based on baseline thresholds as well as reference tasks. The work presented in this paper is the result of several experiments in the domain of BCI where participants are interacting with continuous visual feedback following a real-time NF paradigm, arising from our long-standing research in the field of affective computing. We offer the community access to our fNIRS datasets from these experiments. We specifically provide data drawn from our empirical studies in the field of affective interactions with computer-generated narratives as well as interfacing with algorithms, such as heuristic search, which all provide a mechanism to improve the ability of the participants to engage in active BCI due to their realistic visual feedback. Beyond providing details of the methodologies used where participants received real-time NF of left-asymmetric increase in activation in their dorsolateral prefrontal cortex (DLPFC), we re-establish the need for carefully designing protocols to ensure the benefits of NF paradigm in BCI are enhanced by the ability of the real-time visual feedback to adapt to the individual responses of the participants. Individualized feedback is paramount to the success of NF in BCIs.

In this paper, we present an approach for generating dialogues for characters within the context of computational narratives using personality-based features for deep neural networks. The approach integrates the requirements of both narrative genres and personality traits for the definition of character-based stylistic models. The modelling of characters' features from existing datasets of complete stories permits the generation of personality-rich character dialogues. We present early results from an evaluation based on a sample of characters' personality traits across different narrative genres, demonstrating variability in the resulting dialogues.

We describe a method and a proof of concept which allow the generation of rich and engaging dialogues between virtual characters from a formalised plot description. The structure of the dialogue generated borrows from inferential pragmatics, following the Geneva Model of discourse analysis, in order to provide realistic interaction between characters in the narrative. At a higher level, this discourse is organised following heuristics borrowed from narratology theory in order to elicit emotions linked to dramatic tension and thus favour narrative engagement. Besides enriching narrative generation systems embedded within simulation applications, our work also has the potential to be adapted to support engaging interactive dialogues between users and virtual conversational agents in narrative systems.

Interactive digital narrative research presents a diverse range of authoring tools [1, 4, 8, 12, 14]. Although our field often publishes the technology, it less often publishes a refined UX design pipeline for those tools’ authoring experience. This is despite the UX of these tools long being identified as a key challenge [14] and UX design pipelines being an active area of research in adjacent technologies such as the games that sometimes deliver our stories [3, 10, 11]. We present a three-stage design pipeline targeting the creation of interactive narrative authoring tools that is informed by existing design pipelines that consider the user and their experience at all stages. We then detail our own application of this pipeline to the design of a new authoring tool, reporting on the methodologies, analyses, and findings of each step.

This paper presents a modern, open-source game engine/framework for the visual novel genre of interactive narrative. It takes the insights from the engines of visual novel games and the products made with them to produce a free engine that contains all the features and components required of a standard visual novel, and demonstrates its capabilities with a demo artefact. Visual novels provide authors with a powerful way of presenting their fiction and narratives, yet they are often considered less viable due to the costs required against the profit in sales, or because of their technical requirements to use. The M22 engine aims to address both these issues.