BACKGROUND:Although family caregivers play a critical role in care delivery, research has shown that they face significant physical, emotional, and informational challenges. One promising avenue to address some of caregivers' unmet needs is via the design of digital technologies that support caregivers' complex portfolio of responsibilities. Augmented reality (AR) applications, specifically, offer new affordances to aid caregivers as they perform care tasks in the home. OBJECTIVE:This study explored how AR might assist family caregivers with the delivery of home-based cancer care. The specific objectives were to shed light on challenges caregivers face where AR might help, investigate opportunities for AR to support caregivers, and understand the risks of AR exacerbating caregiver burdens. METHODS:We conducted a qualitative video elicitation study with clinicians and caregivers. We created 3 video elicitations that offer ways in which AR might support caregivers as they perform often high-stakes, unfamiliar, and anxiety-inducing tasks in postsurgical cancer care: wound care, drain care, and rehabilitative exercise. The elicitations show functional AR applications built using Unity Technologies software and Microsoft Hololens2. Using elicitations enabled us to avoid rediscovering known usability issues with current AR technologies, allowing us to focus on high-level, substantive feedback on potential future roles for AR in caregiving. Moreover, it enabled nonintrusive exploration of the inherently sensitive in-home cancer care context. RESULTS:We recruited 22 participants for our study: 15 clinicians (eg, oncologists and nurses) and 7 family caregivers. Our findings shed light on clinicians' and caregivers' perceptions of current information and communication challenges caregivers face as they perform important physical care tasks as part of cancer treatment plans. Most significant was the need to provide better and ongoing support for execution of caregiving tasks in situ, when and where the tasks need to be performed. Such support needs to be tailored to the specific needs of the patient, to the stress-impaired capacities of the caregiver, and to the time-constrained communication availability of clinicians. We uncover opportunities for AR technologies to potentially increase caregiver confidence and reduce anxiety by supporting the capture and review of images and videos and by improving communication with clinicians. However, our findings also suggest ways in which, if not deployed carefully, AR technologies might exacerbate caregivers' already significant burdens. CONCLUSIONS:These findings can inform both the design of future AR devices, software, and applications and the design of caregiver support interventions based on already available technology and processes. Our study suggests that AR technologies and the affordances they provide (eg, tailored support, enhanced monitoring and task accuracy, and improved communications) should be considered as a part of an integrated care journey involving multiple stakeholders, changing information needs, and different communication channels that blend in-person and internet-based synchronous and asynchronous care, illness, and recovery.

Rerun is a software system to support post-facto analysis of driving simulation research. It is built in Unity 3D, and captures virtual driving behavior so that it can be played back. A unique feature of Rerun is that the playback can be rendered from any perspective in the virtual space. This is useful in multi-person interaction studies because researchers can examine scenarios from each participant’s perspective, or even from an outside observer’s perspective. This enables fine-grained understanding of implicit and explicit signalling between participants, enabling research to reconstruct what factors are pertinent to driving communication.

In this work, we address an important problem of optical see through (OST) augmented reality: non-negative image synthesis. Most of the image generation methods fail under this condition, since they assume full control over each pixel and cannot create darker pixels by adding light. In order to solve the non-negative image generation problem in AR image synthesis, prior works have attempted to utilize optical illusion to simulate human vision but fail to preserve lightness constancy well under situations such as high dynamic range. In our paper, we instead propose a method that is able to preserve lightness constancy at a local level, thus capturing high frequency details. Compared with existing work, our method shows strong performance in image-to-image translation tasks, particularly in scenarios such as large scale images, high resolution images, and high dynamic range image transfer.

In this short paper we explore the opportunities and challenges of designing XR technologies to support the collaborative work between family caregivers and clinicians as they attend to the physical care needs of patients in the home setting.

In applications such as optical see-through and projector augmented reality, producing images amounts to solving non-negative image generation, where one can only add light to an existing image. Most image generation methods, however, are ill-suited to this problem setting, as they make the assumption that one can assign arbitrary color to each pixel. In fact, naive application of existing methods fails even in simple domains such as MNIST digits, since one cannot create darker pixels by adding light. We know, however, that the human visual system can be fooled by optical illusions involving certain spatial configurations of brightness and contrast. Our key insight is that one can leverage this behavior to produce high quality images with negligible artifacts. For example, we can create the illusion of darker patches by brightening surrounding pixels. We propose a novel optimization procedure to produce images that satisfy both semantic and non-negativity constraints. Our approach can incorporate existing state-of-the-art methods, and exhibits strong performance in a variety of tasks including image-to-image translation and style transfer.

In virtual reality (VR), continuous movement often leads to users experiencing vection, or cybersickness. To circumvent vection, users are typically given the choice to use teleportation. However, teleportation leads to less spatial awareness compared to continuous movement. One approach to combating cybersickness in continuous movement is to restrict the user’s field of view (FOV). This is typically done by occluding the user’s peripheral vision with a vignette. We developed a FOV restricting continuous locomotion system that does not occlude the user’s FOV and instead uses a 3D portal to display the continuous movement in a limited area in the user’s FOV. We found that this system reduces nausea and disorientation compared to continuous locomotion. However, our system did not significantly increase spatial awareness compared to teleportation.

Real-time tracking and visual feedback offer interactive AR-assisted capture systems as a convenient and low-cost alternative to specialized sensor rigs and robotic gantries. We present a simple strategy for decoupling localization and visual feedback in these applications from the primary sensor being used to capture the scene. Our strategy is to use an AR HMD and 6-DOF controller for tracking and feedback, synchronized with a separate primary sensor for capturing the scene. This approach allows for convenient real-time localization of sensors that cannot do their own localization (e.g., microphones). In this poster paper, we present a prototype implementation of this strategy and investigate the accuracy of decoupled tracking by mounting a high resolution camera as the primary sensor, and comparing decoupled runtime pose estimates to the pose estimates of a high-resolution offline structure from motion.

In this project, we explore the design of a system for helping users capture surface reflectance functions with a headmounted augmented reality (AR) device and a hand-held controller. We supplement a standard 6-DOF controller with a mountable light source that we track during capture. Users begin by using the controller to select a surface region for capture. We then record images through the head-mounted camera while guiding the user’s control of the hand-held light source with real-time feedback through the AR display. Our system provides a simple and efficient way to capture information about surface reflectance in the wild.

Facial expressions are an important part of human communication. At the same time, faces also reveal aspects of our identities – ethnicity, gender, or even sexual orientation – that can surface biases held by our counterparts and lead to downstream inequalities in our social interactions. However, we do not have the option to entirely change what we look like, or at least hide the identities that our faces might expose. As Optical See-Through (OST) Augmented Reality (AR) headsets possess the advantages of near-eyes display and better depth alignment between virtual renderings and the environment, we decided on an OST AR approach for the solution. In this paper, we present a system designed for OST AR headsets that occludes the subject’s facial features with an emotion-presenting emoji model in 3D space.

Facial expressions are an important part of human communication. However, children with autism spectrum disorders (ASD) are often suffering from difficulties of understanding non-verbal cues and form appropriate responses. Traditional approaches including labeling formatted photographs of human facial expressions from a third person's perspective could help them learn and improve such skills. Yet such training systems are often in lack of real time feedback. As Optical See-Through (OST) Augmented Reality (AR) headsets possess the advantages of near-eyes display and better depth alignment between virtual renderings and the environment, we decided on an OST AR approach for the system. In this paper, we present a system designed for OST AR headsets that occludes the subject's facial expressions with an emotion-presenting 3D emoji model. We hope this system could help us understand how children with ASD perceive emotions through standard emotion presenting systems and help them enhance their skills of understanding facial expressions.