Osteoporosis affects >200 million people worldwide, with physical therapy being a key non-pharmaceutical intervention to reduce fall and fracture risk. To address engagement challenges with traditional therapy, virtual rehabilitation through exergames has emerged as a promising solution. Our clinical trial developed 4-bespoke augmented reality (AR) exergames to reduce fall risk in people with osteoporosis. The exergames mapped to physical therapy exercises tailored for osteoporosis management and included sit-to-stand, squat, arm-raises and step-up exercises. A total of 41 females aged 60–86 years (divided into a control and an intervention group) attended bi-weekly, 20-min sessions for 6-weeks. The intervention group used a Microsoft HoloLens 2 and Azure Kinect. The exergames evolved weekly, offering real-time visual and auditory feedback, providing cues, motivation and posture corrections. This chapter presents findings on participant tolerance for AR and enjoyment of the exergames. In total, 65

The capacity to create realistic virtual humans has progressed significantly, and such characters can be found in many applications across entertainment, education and health. As an essential element of interactive virtual humans, speech-driven 3D gesture generation still depends heavily on perceptual evaluation, yet studies often vary avatar appearance and facial presentation when judging the generated motions. Prior work suggests these visual choices can bias motion judgments, but controlled evidence remains limited. We address this gap with controlled evaluations of co-speech gestures across motion sources, spanning seven representative avatar renderings used in contemporary research and application pipelines. Our results show that avatar and face presentation systematically shift perceptual judgments, and we provide recommendations for benchmarking gesture synthesis as well as for deploying virtual humans in human-facing applications.

Radiance field-based rendering methods have attracted significant interest from the computer vision and computer graphics communities. They enable high-fidelity rendering with complex real-world lighting effects, but at the cost of high rendering time. 3D Gaussian Splatting solves this issue with a rasterisation-based approach for real-time rendering, enabling applications such as autonomous driving, robotics, virtual reality, and extended reality. However, current 3DGS implementations are difficult to integrate into traditional mesh-based rendering pipelines, which is a common use case for interactive applications and artistic exploration. To address this limitation, this software solution uses Nvidia's interprocess communication (IPC) APIs to easily integrate into implementations and allow the results to be viewed in external clients such as Unity, Blender, Unreal Engine, and OpenGL viewers. The code is available at https://github.com/RockyXu66/splatbus.

We propose a novel framework for decomposing arbitrarily posed humans into animatable multi-layered 3D human avatars, separating the body and garments. Conventional single-layer reconstruction methods lock clothing to one identity, while prior multi-layer approaches struggle with occluded regions. We overcome both limitations by encoding each layer as a set of 2D Gaussians for accurate geometry and photorealistic rendering, and inpainting hidden regions with a pretrained 2D diffusion model via score-distillation sampling (SDS). Our three-stage training strategy first reconstructs the coarse canonical garment via single-layer reconstruction, followed by multi-layer training to jointly recover the inner-layer body and outer-layer garment details. Experiments on two 3D human benchmark datasets (4D-Dress, Thuman2.0) show that our approach achieves better rendering quality and layer decomposition and recomposition than the previous state-of-the-art, enabling realistic virtual try-on under novel viewpoints and poses, and advancing practical creation of high-fidelity 3D human assets for immersive applications. Our code is available at https://github.com/RockyXu66/LayerGS

Osteoporosis is a major bone disease, affecting more than 200 million people globally. Physical exercise is a powerful non-pharmaceutical fracture prevention strategy for people with osteoporosis or those at risk of falls. However, the participation in and adherence to an exercise regimen by older adults is often low due to a lack of motivation, a fear of falling, safety and/or cost. Despite the potential of Augmented Reality (AR) exergames to enhance engagement, motivation, and accessibility in rehabilitation, there is a notable lack of clinical research on their use for osteoporosis management. This study protocol details a partially randomised controlled clinical trial investigating the feasibility and effectiveness of AR-exergames for osteoporosis rehabilitation. We designed four exergames using AR and a body-tracking camera, providing real-time feedback. Based on power analysis, a total of 50 participants were expected after satisfying the inclusion criteria. Forty-eight women (aged 60-86) were enrolled and assigned to the control/intervention training groups. Participants assessment at baseline and at the end of the 6-weeks training included muscle strength, flexibility, balance and pain. This clinical trial is designed to evaluate whether novel AR exergame-based training has significantly greater effects on physical (i.e., muscle strength, balance, flexibility) and affective (i.e., pain) outcomes compared to traditional training programmes. Findings from this trial provide critical insights into the feasibility and effectiveness of immersive, technology-enhanced rehabilitation for osteoporosis management. Key lessons highlight the importance of diverse recruitment strategies, flexible yet structured scheduling, and efficient resource allocation to improve trial efficiency and participant engagement in future studies.

We present a preliminary experiment, investigating professional hand-drawn animators' perception of how good one frame is as drawing reference for another. 10 professional hand-drawn animators rated the drawing reference quality of 54 hand-drawn frame pairs, each differing by character pose and region rotation, reflection, and distortion transformations. Our results indicate that animators perceive frames differing by rotation/reflection as better drawing reference than frames differing by distortion.

We present CrowdSplat, a novel approach that leverages 3D Gaussian Splatting for real-time, high-quality crowd rendering. Our method utilizes 3D Gaussian functions to represent animated human characters in diverse poses and outfits, which are extracted from monocular videos. We integrate Level of Detail (LoD) rendering to optimize computational efficiency and quality. The CrowdSplat framework consists of two stages: (1) avatar reconstruction and (2) crowd synthesis. The framework is also optimized for GPU memory usage to enhance scalability. Quantitative and qualitative evaluations show that CrowdSplat achieves good levels of rendering quality, memory efficiency, and computational performance. Through these experiments, we demonstrate that CrowdSplat is a viable solution for dynamic, realistic crowd simulation in real-time applications.

Efficient and realistic crowd rendering is an important element of many real-time graphics applications such as Virtual Reality (VR) and games. To this end, Levels of Detail (LOD) avatar representations such as polygonal meshes, image-based impostors, and point clouds have been proposed and evaluated. More recently, 3D Gaussian Splatting has been explored as a potential method for real-time crowd rendering. In this paper, we present a two-alternative forced choice (2AFC) experiment that aims to determine the perceived quality of 3D Gaussian avatars. Three factors were explored: Motion, LOD (i.e., #Gaussians), and the avatar height in Pixels (corresponding to the viewing distance). Participants viewed pairs of animated 3D Gaussian avatars and were tasked with choosing the most detailed one. Our findings can inform the optimization of LOD strategies in Gaussian-based crowd rendering, thereby helping to achieve efficient rendering while maintaining visual quality in real-time applications.

volume Holographic Optical Elements (VHOE) are being developed for use in image combiners and waveguide couplers within the near eye display(NED). VHOEs have the potential to replace conventional geometric optics and can perform multiple functions in a lightweight thin layer, depending on the recorded hologram design. However, due to high dispersion and the angular and spectral selectivity of VHOEs, bespoke solutions are needed for every new system design and it is important to be able to fabricate at a given recording wavelength (with available lasers) regardless of the intended operating wavelength of the VHOE. To alleviate the problem, we have proposed a Bragg angle compensation model, enabling recording at wavelengths distinct from the reconstructing/operating wavelength. Here the model is used to fabricate stacked VHOEs for an occlusion-capable NED design that utilizes both Uv and visible light sources. The experimental results show that the VHOE has a precision of +/- 1 degrees from the desired diffraction angle when recorded in 532nm and reconstructed in 405nm and the experimental Bragg curve is measured up to 90% Diffraction Efficiency. The modulation transfer function of the VHOE is measured at 16 cycles per degree.

We present work-in-progress on the development of a framework for immersive authoring of dynamic Mixed Reality scenes, employing the World-in-Miniature technique to reduce workload, using a tangible user interface for intuitive object placement, and incorporating spatio-temporal bounding volumes to avoid clipping with dynamic content. We developed a prototype on the Meta Quest 3 using its Passthrough camera access, and present the design and implementation details for such a system, along with insights that motivate promising directions for future work.

We survey the use of computational topology in hand-drawn animation technology over the past 15 years. We discuss three main subfields of hand-drawn animation technology research: frame deformation, frame feature correspondence, and volumetric modeling of hand-drawn characters. We explore the various topological spaces and operators applied to each subfield, detailing the artistic and computational advantages and limitations of each topological approach. Throughout our discussion, we provide insights from leading hand-drawn animation professionals — Glen Keane (Disney) and Mark Mullery (Cartoon Saloon) — to enrich our assessments of topological approaches’ suitability in industry settings.

In this article, we report on research exploring tangible interfaces for authoring cutaway visualizations in Mixed Reality (MR). The proposed approach is to allow users to create the outline of a virtual cutaway section by physically tracing their fingers directly on the surface of a real object, to reveal virtual information within. Building on a prototype of such system, developed for the Microsoft HoloLens 2, we conducted user studies to compare the performance and feedback of participants using a tangible system in contrast to the completion of analogous tasks using conventional mid-air gestures. The experiments revealed that the tangible interface supported more accurate tracing, and participants consistently rated it as more intuitive and effective. These findings highlight the promise of tangible approaches and provide compelling direction for future work.

The rapid expansion of immersive interaction and visualization has led to the emergence of volumetric videos. However, analyzing information within such content is still in its early stages, primarily because well-established techniques for 2D videos and 3D animations are not directly applicable. This challenge stems from the limited availability of open-source volumetric content and the inherent difficulties in capturing volumetric videos. These obstacles have resulted in a notable gap in the fields of summarization, retrieval, and streaming of volumetric videos. This paper delves into the summarization of human volumetric videos by employing traditional shape descriptors, and pseudo-labels. The primary goal is to discern specific key frames within the sequence that distinctly stand out from their neighbouring frames, which are further labeled with a textual description of pose information per frame and a semantic action annotation of the sequence. Furthermore, the extracted metadata could be used in future applications, to enable users to search through large collections of volumetric video data.

Volume visualization plays a crucial role in both academia and industry, as volumetric data is extensively utilized in fields such as medicine, geosciences, and engineering. Addressing the complexities of volume rendering, neural rendering has emerged as a potential solution, facilitating the production of high-quality volume rendered images. In this paper, we propose TwineNet, a neural network architecture specifically designed for volume rendering. TwineNet combines features extracted from volume data, transfer functions, and viewpoints by utilizing twining skip connections across multiple feature layers. Building upon the TwineNet architecture, we introduce two neural networks, VolTFNet and PosTFNet, which leverage convolutional encoder–decoders and multilayer perceptrons to synthesize volume rendered images with novel transfer functions and viewpoints. Our experimental findings demonstrate the superiority of our models compared to state-of-the-art approaches in generating high-quality volume rendered images with novel transfer functions and viewpoints. This research contributes to advancing the field of volume rendering and showcases the potential of neural rendering techniques in scientific visualization.

This study aims to identify effective ways to design virtual rehabilitation to obtain physical improvement (e.g. balance and gait) and support engagement (i.e. motivation) for people with osteoporosis or other musculoskeletal disorders. Osteoporosis is a systemic skeletal disorder and is among the most prevalent diseases globally, affecting 0.5 billion adults. Despite the fact that the number of people with osteoporosis is similar to, or greater than those diagnosed with cardiovascular disease and dementia, osteoporosis does not receive the same recognition. Worldwide, osteoporosis causes 8.9 million fractures annually; it is associated with substantial pain, suffering, disability and increased mortality. The importance of physical therapy as a rehabilitation strategy to avoid osteoporosis fracture cannot be over-emphasised. However, the main rehabilitation challenges relate to engagement and participation. The use of virtual rehabilitation to address such challenges in the delivery of physical improvement is gaining in popularity. As there currently is a paucity of literature applying virtual rehabilitation to patients with osteoporosis, the authors broadened the search parameters to include articles relating to the virtual rehabilitation of other skeletal disorders (e.g. Ankylosing spondylitis, spinal cord injury, motor rehabilitation, etc.). This systematic review initially identified 130 titles, from which 23 articles (involving 539 participants) met all eligibility and selection criteria. Four groups of devices supporting virtual rehabilitation were identified: a head-mounted display, a balance board, a camera and more specific devices. Each device supported physical improvement (i.e. balance, muscle strength and gait) post-training. This review has shown that: (a) each device allowed improvement with different degrees of immersion, (b) the technology choice is dependent on the care need and (c) virtual rehabilitation can be equivalent to and enhance conventional therapy and potentially increase the patient’s engagement with physical therapy.

In this paper, we discuss work-in-progress research on using tangible interfaces for intuitively authoring and visualizing internal 3D structures in Mixed Reality (MR). Virtually embedding internal structures is an approach that is commonly used for explanatory and instructional visualizations in domains such as anatomy, engineering, and geosciences. However such embedded-object visualizations often suffer from spatial ambiguity, and can be difficult to create without technical proficiency with 3D modeling tools or graphical programming. To address these issues, we propose an approach that leverages tangible interaction in an immersive authoring system, where users curate a mixed-reality visualization from within the mixed reality experience itself. Tangible interaction, allowing the user to physically touch, hold, and feel physical elements of the MR interface, eases the modeling process and enhances the sense of the relative 3D spatial orientations and positions of objects manipulated by the user. Specifically, we provide an intuitive user interface for virtually embedding objects inside captured real-world objects, and customizing cutaways of the real object to reveal the underlying internal objects.

Via virtual reality (VR) and in 3D immersive virtual environments (IVEs), viewers can observe their surroundings in any direction. This unrestricted view challenges storytelling, as directors cannot control the viewer’s perspective. To overcome this, we propose to develop a system that enhances narrative focus by rendering only high-quality story-critical elements within an IVE. This approach ensures that viewers are guided to the intended view, maintaining the director’s vision while preserving the immersive experience of VR.

In this poster, we present work-in-progress research towards improving the spatial perception of 3D objects on Optical See-through Mixed Reality displays, addressing, in particular, visualizations comprising virtual objects embedded inside physical real-world objects. This use-case is common in visualization applications in medicine, engineering and geosciences, but is an added challenge in mixed reality (MR), where multiple modalities of data (real and virtual objects) essentially occupy the same 3D space leading to inevitable depth ambiguity. In this poster, we review several strategies for ameliorating this problem, discuss insights from preliminary implementations of such approaches and propose avenues for future work.

In this paper, we discuss work-in-progress research on tangible interfaces for interactive cutaway visualizations in Mixed Reality (MR). We present an approach that allows users to flexibly and intuitively define virtual cutaway geometry by directly interacting with real-world objects. Using hand movements the user physically traces the shape of the cutaway on an object, without the need for specialized input devices. Tangible interaction allows more accurate interaction and makes it easier for users to plan the placement and shape of required cutaways that fit the object. We developed a prototype demonstrator on the Microsoft HoloLens 2 and present the design and implementation details for such a system and discuss insights, gained from preliminary testing, that motivate compelling directions for future work.