Immersive exhibitions increasingly combine large-scale projections with mixed reality (MR) head-mounted displays (HMDs), but how users perceive transitions across these heterogeneous visualization environments remains underexplored. We investigate cross-reality (CR) object- and scene-level transitions in a hybrid immersive art installation spanning projection, augmented reality (AR), and virtual reality (VR). In a within-subjects study (N=24), we compared a calibrated condition with a bundled inconsistency condition introducing noticeable differences in spatial alignment, visual appearance, and cross-device latency. This contrast was used not to establish perceptual thresholds or hardware targets, but as a diagnostic probe to make transition disruptions more perceptible and discussable. The inconsistency condition reduced presence and increased workload. Interviews showed that spatial misalignment disrupted action-outcome predictability, appearance mismatch reduced aesthetic coherence, and latency weakened perceived responsiveness. Effects also varied by asset type: rigid static assets exposed inconsistencies more clearly, animated skeletal assets shifted attention toward motion, and particle effects often masked small discrepancies. These findings clarify how users perceive projection-MR transitions and support asset-aware design strategies for coherent hybrid immersive art. We also release HUICRSync, an open-source Unreal Engine plugin for prototyping projection-MR CR experiences.

Narrative puzzles in video games are puzzles that require exploration, and logical and creative thinking to solve, that form part of the progression of a narrative. They are useful for advancing, structuring, and pacing a game’s plot while also adding some variation to gameplay. Story Puzzle Heuristics for Interactive Narrative eXperiences 2 (SPHINX 2) is a procedural narrative puzzle generation system developed for implementation in open world video games. The design of the system and its implementation in the video game Curse of the SPHINX are presented and analysed in an exploratory user study. Inductive thematic analysis of semi-structured interview transcripts, and data collected using the miniPXI indicates that: (a) participants used several different methods to solve SPHINX 2 puzzles; (b) SPHINX 2 is mostly effective in communicating puzzles to players; (c) SPHINX 2 puzzles are somewhat fun and engaging; (d) the open world setting of the game influenced engagement with puzzles; (e) SPHINX 2 can be improved by increasing puzzles’ narrative context, and improving confusing elements like variable puzzle complexity.

Natureculture heritage sites are dynamic spaces where ecological, cultural, and historical forces converge, often creating tensions between preservation, accessibility, and interpretation. This study explores how locative technologies can navigate these tensions by embedding digital interaction within the material and affective fabric of place, enabling context-sensitive storytelling and surfacing silenced narratives. We investigate visitor experiences at three contrasting sites — Madeira's Levadas (Portugal), the Avebury Landscape (UK), and the site of the Battle of the Boyne (Ireland) — to examine how visitors engage with more-than-human histories and temporalities, thus identifying six distinct design spaces within three overarching typologies of natureculture heritage: Ideal Nature , Ideal Time , and Ideal History . Based on how these novel typologies inform the operation of locative technologies as ethical mediators between human and nonhuman agencies, we subsequently present the design of seven bespoke locative experiences across the three sites, diversely exploring communication technologies in support of novel heritage encounters. This paper advances HCI debates on posthuman design for naturecultures by illustrating how locative media concepts can be informed by the particular context and challenges of the heritage site, and how its design can consider situated, relational, and ethically attentive engagement with multilayered heritage.

Representing uncertain history through Interactive Digital Narratives offers interactors the opportunity to challenge the grand narrative through engagements with the historical record. A gap in the literature exists in the evaluation of IDNs’ ability to teach interactors how to navigate historical uncertainty, while a demand for diegetic procedural knowledge learning exists. We propose an evaluation framework that helps determine the effectiveness of an IDN to deliver knowledge in the context of historical uncertainty and seek to evaluate not only the acquisition of declarative knowledge but also procedural knowledge learning needed to handle such conflicting perspectives. This evaluation framework is supported by a VR IDN implementation of a prehistoric complex, presenting alternative interpretations based on published research upon which the interactor is invited to answer questions about the historical site. Validation through cultural heritage experts supports the IDN’s ability to empower interactors to question the grand narrative and take an active role in the interpretation of the site.

Mobile digital media technologies, such as smartphones, have great potential for in situ storytelling in cultural heritage sites. By geo-locating narrative content across a site, it can be experienced simultaneously with the physical site, creating a situated narrative of historical relevance. However, looking at and interacting with mobile devices tend to capture the user’s attention and all too easily distract from the site. For this reason, it is important to strike a balance between immersion (into the digital content) and presence (in the physical site). In this paper, we describe an approach to locative storytelling for cultural heritage, which is based on the exclusive use of sound and music for all ludonarrative aspects of the experience and based on the concept of psychogeography, i.e., how places make us feel. Our case study is the Battle of the Boyne in Ireland, the site of the eponymous battle that took place on 1 July 1690 (O. S.) between the forces of the deposed King James II and those of King William III. The battle is considered one of the most significant events in Irish and English history and is part of a wider struggle for power across 17th century Europe. We present our narrative design, sound and music design, game design, which together are captured in our ludonarrative engine in which a combination of historical narrative, music narrative and spatial audio navigation are closely intertwined with game mechanics, creating an immersive experience for cultural, educational and intergenerational engagement.

With the emergence of advanced multimedia technology, such as VR/ MR and research into music consciousness and cognitive musicology, new opportunities arise for encouraging associative thinking and imagery in classical music, both for professional performers, and also, music lovers. In this paper, we report on a case study where the experimental group (N = 22) experienced three mixed reality scenes in which the room was augmented with virtual objects from nature (e.g., a tree and falling leaves) and/or the fantastic (e.g., a mermaid) designed to resonate with the music. The control group (N = 21) listened to the same music as the experimental group using audio only. Participants in both groups were asked to rate their emotions and experiences via several subscales of the Phenomenology of Consciousness Inventory, and describe associations, memories or thoughts that came to mind. Both groups reported being significantly absorbed by the experience and that the experience induced a state of relaxation. Listening to the music evoked a more inner-directed state of attention, while the MR group reported strong engagement with the multisensory MR environment. Familiarity with MR/VR increased the enjoyment of the MR experience. The audio group also described a wide range of associative memories and images, which bear resemblance to the MR visualizations in several ways. We conclude that MR art/music experiences have potential uses for (1) attracting new audiences to classical music; (2) encouraging professional performers and/or music lovers in associative thinking and imagery; and (3) possibly serving as a new form of immersive art music therapy.

For decades, digital technologies have been used to offer engaging experiences in cultural heritage sites, and these experiences have shown useful as a means to convey the histories associated with the sites. While some experiences are information-driven (e.g., tour guides), others are story-driven and therefore need a narrative justification for why the past and the present collide and in which way. This paper explores how narrative conventions from genre fiction specifically, ghost stories, science fiction, historical fiction and fantasy can frame narrative content in cultural heritage sites. We discuss these specific literary genres and their use in locative games to achieve the temporal collisions necessary for framing story-driven locative cultural heritage experiences. We identify the nature of the main temporal connections used by the different genres (including issues of narrative control) and offered a tentative mapping of the temporal connections onto an actual cultural heritage site.

Although Virtual Reality technology offers a variety of experiences, Mixed Reality (MR) is a relatively new field, offering the ability to merge real and digital environments in sophisticated ways. Perception is at the core of this integration and our experience of them: As humans, we perceive reality with our senses; we feel rain on our skin, we see the drops, we hear the sound, we can taste and smell the freshness of the air in real life. When creating and experiencing digitally enhanced realities, we still rely on the senses, but practically all VR/MR use only the audio-visual source, because other sensory modalities (e.g., tactile and olfactory) are not generally supported by the platforms. In this paper, we are concerned with the following questions: How can we reach other senses through a MR audio-visual environment? Can we create dreamlike experiences and feel music and art as our reality? Music and art therapies are well-established practices, so can MR have the potential to provide mindfulness? The pilot study presented in this paper offers insights into the nature of cross-modal associations and synesthesia interaction of senses theory as a framework for multisensory design. The benefits of multisensory stimulation are not fully explored in the field of neuroscience, but there are statements linking synesthesia (multisensory or cross-modal perception) with creativity and increased mental ability. Our results with a pilot study of our DreaMR series of experiences indicate a much greater ability for multisensory processing with MR in the general public than expected.

Provoked by numerous art-less walks during the pandemic, our Open Air Gallery AR project, is a smartphone-based locative media platform to display digital galleries and music albums with gamified engagement. This short work-in-progress paper presents one such "gallery game," a new type of interactive multimedia for virtual exhibition in nature or a park. Geri Hahn Art: Living Out Loud AR (2024) features the art of an artist-synesthete who uses synesthesia as her creative inspiration. The app includes music by Svetlana Rudenko (Piano, Logic Pro) and Mads Haahr (Synths), composed to transfer the art expression into music language using a unique multisensory design approach where art is translated into music, and location is chosen co-create the experience. Recorded talks with the artist give insights into her life in art, her inspiration and craft of expression. The gallery game genre allows access for a wide audience without the limitations of a physical exhibition venue (e.g., availability, costs for rent). Our preliminary results show a high level of interest in the genre as well as promising social benefits of cooperative play.

This paper presents a study of the effect of the level of perceptual realism of the game world's dimensionality., graphics., and physics on learning outcomes and motivation. An evaluation of two versions of an educational video game that differ in the level of realism was performed. Descriptive and inferential analysis results show that both versions are effective learning tools and equally motivating., suggesting that 3D educational games do not cause a higher cognitive load than 2D educational games.

Providing enough information to allow the VR player to self-identify is an important factor in their immersion into a virtual world. The sensory information being provided to the player’s eyes and ears through the headset must support the suspension of disbelief and telepresence into the virtual world. Given sound’s easily realized potential for diegetic ambiguity and its influence on presence as immersion, the aim of this paper is to explore the role of disembodied voice in VR interactive narratives and its effect on presence through self-identification. We start by analyzing the shift of focalization in the film Saving Private Ryan, the trans-diegetic narration in the game StarCraft II, and the address of the player in the game Call of Duty. We then contribute with close readings of two VR productions, The Last Goodbye and The Book of Distance, in terms of the use of second-person voice and interactivity as well as an analysis of the resulting levels of self-identification. A further contribution is a Twine interactive narrative that demonstrates some of the key concepts discussed in the paper. We conclude that future empirical work should explore the impact of second-person voice and interaction on the resultant self-identification and immersion.

This work-in-progress paper presents Synaesthesia Gallery AR : Journey Through the Senses , an educational game-like experience designed to bring awareness of synaesthesia, to learn and understand creativity of cross-modal perception. Synaesthesia is a perceptual trait characterised by cross-modal wiring of the brain experienced by a 4–6% of the population. Synaesthesia research has a long history (as far back as 1690), and sometimes synaesthesia was confused for hallucinations. The AR app documents creative synaesthesia experiences of individuals who have contributed significantly to synaesthesia research, including international artists, such as Carol Steen (president of USA Synaesthesia Association), and scientists, such as Jamie Ward (Professor at the University of Sussex, UK). Ramachandran and Hubbard have called synaesthesia “a window into perception, thought and language” that can help explain how the brain works. The app experience takes a form of an interactive digital gallery for the outdoors, a locative Augmented Reality (AR) art gallery with music by Liszt, Chopin, Schuman, Scriabin and others painted by artists-synaesthetes and music composed on art. The work is the result of a seven-year collaboration between artists-synaesthetes and pianist/researcher Dr Svetlana Rudenko. The app provides pleasing aesthetic experience as well as learning, to general population, neuroscience and psychology students, synaesthesia associations and music art lovers.

This paper introduces the Context Learning Game (CLG) framework, a game design framework developed for educational video games in formal education settings. The CLG framework is structured into three stages: context, learning, and game. The context stage establishes general game aspects such as the objectives, target audience, and available resources. The learning stage defines formal learning components, including learning theory, content, learning objectives, and learning activities. Finally, the game stage encompasses the formal game elements, development, and testing. The stages are hierarchical, with the context forming the foundation for the learning stage, and both serving as the basis for the game stage. The framework aims to integrate concepts from educational video game design, entertainment game design, and educational design while providing a simple iterative workflow. To illustrate the framework?s potential, a successful case study of an educational game?s design, development, and testing process is presented.

Narrative puzzles feature prominently in story-based open world video games where they form part of the progression of a narrative and require exploration and logical and creative thinking to solve. Open world games use navigation aids to help players solve narrative puzzles located across the vast worlds they provide. Narrative puzzles offer designers one of the most interactive methods of conveying a game’s narrative, particularly in the action-adventure focused genre of open-world games. In this paper we discuss the interaction between narrative puzzles and navigation aids in open world games, highlighting in case studies of three different games how the amount of information a navigation aid provides impacts the intricacy and expansiveness of puzzles. Finally, we discuss the design implications of these interactions and suggest potential future analytical uses of the discussed framework.

This paper presents a comprehensive evaluation of an educational video game, DS-Hacker 3D, that incorporates analogies to enhance the learning of conceptual knowledge in computer science, specifically Binary Search Tree (BST) concepts. The study addresses the challenges students face in understanding complex computer science topics and the limited availability of well-evaluated educational video games in the field. DS-Hacker 3D targets undergraduate students and follows a constructivist learning approach, establishing connections between new information and familiar knowledge through analogies. The evaluation includes validated assessment tools to measure learning outcomes and intrinsic motivation. The results demonstrate the effectiveness of the educational video game in facilitating the acquisition of BST conceptual knowledge and promoting intrinsic motivation. The study contributes to the development of educational video games for teaching computer science concepts.

This late breaking work paper presents Alice Dali Mixed Reality (MR) , a cutting-edge, first-ever concept of an interactive experience using music, art and narrative in a new and interactive fashion for individual use and as a live performance. We approach the project like a surrealist painting: an experience that resides halfway between the visible and the invisible. MR technology is uniquely suited for this purpose because it positions the audience not within a virtual world (like VR technology does) but in a liminal space that blends the real with the un-real, the conscious with the unconscious, the physical world with the world of dreams. We performed three interactive Alice Dali MR scenes for the Meta Quest Pro headset, projected to the screen for audience view in a live performance. The original mixed reality audio-visual experience and interaction in the MR app involves orchestration of original piano tracks, activated by the magic conductor’s wand when touching one of the rivers in Chapter 2 “Pool of Tears,” by petting animals in Chapter 11 “Who Stole the tarts?” and growing roses in Chapter 12 “Alice’s Evidence.” We present our motivation for the work, the overall experience design and brief reflections on future work, including usage areas such as family entertainment, intergenerational play, the music industry in the genre of concert with visuals, and music art therapy in rehabilitation programs.

This paper presents a serious game designed to help train child helpline counsellors in the detection and response to contacts by children risking conflict-related human trafficking and child sexual abuse and exploitation. It presents trainee counsellors with three stories following young people from Ukraine: Andriy, Olga and Ilya, who contact a child helpline to seek counselling for what are likely situations of trafficking or child sexual exploitation. In each story, the trainee takes on the role of counsellor and interacts with the child character through a branching narrative. The serious game is developed by Haunted Planet Studios and Child Helpline International, in partnership with Terre des Hommes Netherlands as part of their Emergency Ukraine Response, and is currently being tested as part of a larger training curriculum.

Cosmic horror, or Lovecraftian horror, is an important subgenre in horror fiction, which is concerned with the horror of the unknowable and incomprehensible. In cosmic horror, the thin veil of human sense-making through which we ordinarily perceive the world is inevitably destroyed through a confrontation with a deep and terrible truth about the universe. For the story protagonists, the encounter with the true nature of things practically always results in madness or death, or at least denial of the events that took place. While cosmic horror originates in literature, significant works exist also in films, graphic novels and games. However, while many games include elements from cosmic horror, the themes and genre conventions of cosmic horror mean that it is far from trivial for games to engage genuinely with the genre. This paper explores the potential for games to capture the feelings of experience of a Lovecraft story authentically via their mechanics and design. We give an overview of the genre conventions of literary cosmic horror and identify six narrative design “challenges” where the genre conventions of narrative-driven games appear to be directly incompatible with those of literary cosmic horror. For each challenge, we discuss the depth and nature of the seemingly irreconcilable differences and use examples from narrative horror games (Lovecraftian and otherwise) to identify potential solutions. Can games and cosmic horror really be mixed? And, if so, how?

This demo paper presents two prototypes of an educational video game for learning Binary Search Tree data structure in higher education environments. The prototypes were used to evaluate the effectiveness of video games to teach abstract non-intuitive conceptual knowledge and the effect of educational video games’ perceptual realism on learning gains and motivation. The paper provides a detailed description of the game, focusing on the game’s learning aspects, such as the learning theory, pedagogical approach, learning objectives, and learning activities. It also emphasizes how the game elements reflect the learning aspects to facilitate learning complex conceptual knowledge.

Uncertain cultural heritage presents a dilemma in its narrative representation. History seeks to push a grand narrative, at the expense of less convenient narratives. Critical historiographic approaches favor the consideration of multiple narratives as they focus on the mediation of history rather than arriving at a single truth. Virtual Reality Interactive Narratives, such as The Book of Distance , exemplify how uncertainty can be represented through re-enactment. In this late breaking work, we provide a close reading of The Book of Distance through the lens of critical historiography and suggest improvements for future similar experiences.