Augmented reality (AR) display characteristics have the potential to either enhance or impair users' spatial abilities and performance. While previous work included studies of spatial performance with various display factors, evidence for objective performance differences is limited due to compensatory behaviors employed by users and overall behavioral differences. In general, it is challenging to document the effects of display factors on task performance as they depend on users' task behaviors, which in turn depend on users' reliance and trust in the technology, which are also affected by the display factors. In this paper, we present two within-subjects experiments (each N = 20) in which we aim to elucidate some of the interrelations between two AR display factors (field of view and visual contrast) with objective task performance and subjective assessments of reliance and trust, while controlling for different behaviors. Participants performed a $360^\circ$360∘ search-and-selection task in a unique hybrid setup, in which we simulated a controlled task environment by having participants stand inside an immersive CAVE-like space while at the same time wearing a head-worn display that overlaid AR tags over the simulated environment. Specifically, we evaluated three fields of view ($43^\circ$43∘, $93^\circ$93∘, and $143^\circ$143∘) and three visual contrasts (0.05, 0.25, and 0.5). We controlled for four different behaviors: AR-Only (only relying on AR), AR-First (prioritizing AR over real world), Real-First (prioritizing real world over AR), and Real-Only (only relying on real world). By controlling for these behaviors, we were able to show objective and subjective benefits of larger fields of view and visual contrast. We illustrate how the controlled behaviors relate to users' subjective reliance and trust in an AR system, and why it is important for researchers and practitioners to understand these subjective and behavioral aspects.

Real-world augmented reality (AR) systems are increasingly network connected in order to succeed in operational spaces dependent on cutting edge sensor and computing technologies, all connected together via the network. This dependency inevitably introduces new sources of error to the AR system through instability in network performance. These errors create familiar problems, such as registration errors, on scales unfamiliar to modern AR systems. The effects of these problems on user performance are difficult to investigate as user performance involves many interrelated factors, some of which are difficult to measure. In this work, we control for user behavior to reduce the complexity of investigating the effects of real-world scale latency and signal dropouts on users’ objective task performance and subjective assessments of the AR system. Behavior was controlled by instructing participants to adopt specific behaviors based on different levels of trust and reliance on AR tags and environmental visual cues. We found significant negative effects of both network error sources on subjective assessments of both the AR system and task, as well as objective task completion time and errors committed. Participants’ subjective assessments and objective task also varied unexpectedly with increasing network error levels, suggesting that human perception of the effects of network errors are not well calibrated to the actual effects on performance. We also found significant interaction between behavior and network errors on task performance, drawing implications not just for AR system design, but also for AR system operation guidelines.

Augmented Reality (AR) head-worn display (HWD) technologies for warfighters have seen various advances over the last decade that make them attractive for simulations, training, and operations. In particular, optical see-through (OST) AR displays are becoming more used on the battlefield as they do not reduce warfighters' visual acuity of the real world. Unfortunately, these displays are still limited in terms of the field of view (FOV) and luminance of the display, the latter of which competes with the luminance in the warfighter's environment. The objective of this work is to evaluate how these two AR HWD factors impact participants' spatial task performance and perception. Specifically, this paper presents an experiment, performed inside a novel hybrid experimental space in which both the FOV and the luminance contrast of the HWD were varied compared to the simulated environment. Participants performed a spatial task involving simulated humans arranged in the 360-degree space around them, augmented with red or blue team member tags on the AR display. The results show that a FOV of 45 degrees or wider as well as a luminance contrast of 0.1 or higher were required for participants to reach a task performance that matched or exceeded that which could be reached without the use of AR. Implications for warfighter AR HWD systems are discussed, especially as they pertain to a warfighter's willingness to trust an AR HWD system and how that trust impacts performance.

The information presented by augmented reality (AR) systems may not appear exactly like information perceived in the "real" world. Shortfalls in AR technology give rise to anomalies like tracker errors, lack of opacity compared to the background, and reduced field of view (FOV) compared to the human visual system's field of view. These anomalies can make users feel like the information presented in an AR training system is not believable. This lack of belief can lead to negative training, where trainees adjust how they train due to flaws in the training system and are therefore less prepared for actual battlefield situations. We have completed an experiment to investigate trust, reliance, and human task performance in an augmented reality three-dimensional experimental scenario. Specifically, we used a methodology in which simulated real (complex) entities were supplemented by abstract (basic) cues presented as overlays in an AR headworn display (HWD) for a visual search and awareness task. We simulated shortfalls of different AR display systems to determine which of the properties most affect training efficacy. Results from our experiment will feed directly into the design of training systems that use AR displays and will help increase the efficacy of training.

This research paper explores the impact of augmented reality (AR) tracking characteristics, specifically an AR head-worn display’s tracking registration accuracy and precision, on users’ spatial abilities and subjective perceptions of trust in and reliance on the technology. Our study aims to clarify the relationships between user performance and the different behaviors users may employ based on varying degrees of trust in and reliance on AR. Our controlled experimental setup used a 360° field-of-regard search-and-selection task and combines the immersive aspects of a CAVE-like environment with AR overlays viewed with a head-worn display.We investigated three levels of simulated AR tracking errors in terms of both accuracy and precision (+0°, +1°, +2°). We controlled for four user task behaviors that correspond to different levels of trust in and reliance on an AR system: AR-Only (only relying on AR), AR-First (prioritizing AR over real world), Real-Only (only relying on real world), and Real-First (prioritizing real world over AR). By controlling for these behaviors, our results showed that even small amounts of AR tracking errors had noticeable effects on users’ task performance, especially if they relied completely on the AR cues (AR-Only). Our results link AR tracking characteristics with user behavior, highlighting the importance of understanding these elements to improve AR technology and user satisfaction.

Various reasons exist why humans desire to magnify portions of our visually perceived surroundings, e.g., because they are too far away or too small to see with the naked eye. Different technologies are used to facilitate magnification, from telescopes to microscopes using monocular or binocular designs. In particular, modern digital cameras capable of optical and/or digital zoom are very flexible as their high-resolution imagery can be presented to users in real-time with displays and interfaces allowing control over the magnification. In this paper, we present a novel design space of intuitive augmented reality (AR) magnifications where an AR head-mounted display is used for the presentation of real-time magnified camera imagery. We present a user study evaluating and comparing different visual presentation methods and AR interaction techniques. Our results show different advantages for unimanual, bimanual, and situated AR magnification window interfaces, near versus far vergence distances for the image presentation, and five different user interfaces for specifying the scaling factor of the imagery.

The expression of human emotion is integral to social interaction, and in virtual reality it is increasingly common to develop virtual avatars that attempt to convey emotions by mimicking these visual and aural cues, i.e. the facial and vocal expressions. However, errors in (or the absence of) facial tracking can result in the rendering of incorrect facial expressions on these virtual avatars. For example, a virtual avatar may speak with a happy or unhappy vocal inflection while their facial expression remains otherwise neutral. In circumstances where there is conflict between the avatar's facial and vocal expressions, it is possible that users will incorrectly interpret the avatar's emotion, which may have unintended consequences in terms of social influence or in terms of the outcome of the interaction. In this paper, we present a human-subjects study (N = 22) aimed at understanding the impact of conflicting facial and vocal emotional expressions. Specifically we explored three levels of emotional valence (unhappy, neutral, and happy) expressed in both visual (facial) and aural (vocal) forms. We also investigate three levels of head scales (down-scaled, accurate, and up-scaled) to evaluate whether head scale affects user interpretation of the conveyed emotion. We find significant effects of different multimodal expressions on happiness and trust perception, while no significant effect was observed for head scales. Evidence from our results suggest that facial expressions have a stronger impact than vocal expressions. Additionally, as the difference between the two expressions increase, the less predictable the multimodal expression becomes. For example, for the happy-looking and happy-sounding multimodal expression, we expect and see high happiness rating and high trust, however if one of the two expressions change, this mismatch makes the expression less predictable. We discuss the relationships, implications, and guidelines for social applications that aim to leverage multimodal social cues.

When medical caregivers transfer patients to another person’s care (a patient handoff), it is essential they effectively communicate the patient’s condition to ensure the best possible health outcomes. Emergency situations caused by mass casualty events (e.g., natural disasters) introduce additional difficulties to handoff procedures such as environmental noise. We created a projected mixed reality simulation of a handoff scenario involving a medical evacuation by air and tested how low, medium, and high levels of helicopter noise affected participants’ handoff experience, handoff performance, and behaviors. Through a human-subjects experimental design study (N = 21), we found that the addition of noise increased participants’ subjective stress and task load, decreased their self-assessed and actual performance, and caused participants to speak louder. Participants also stood closer to the virtual human sending the handoff information when listening to the handoff than they stood to the receiver when relaying the handoff information. We discuss implications for the design of handoff training simulations and avenues for future handoff communication research.

Patient handoffs are a common, yet frequently error prone occurrence, particularly in complex or challenging battlefield situations. Specific protocols exist to help simplify and reinforce conveying of necessary information during a combat-casualty handoff, and training can both reinforce correct behavior and protocol usage while providing relatively safe initial exposure to many of the complexities and variabilities of real handoff situations, before a patient’s life is at stake. Here we discuss a variety of mixed reality capabilities and training contexts that can manipulate many of these handoff complexities in a controlled manner. We finally discuss some future human-subject user study design considerations, including aspects of handoff training, evaluation or improvement of a specific handoff protocol, and how the same technology could be leveraged for operational use.

Previous research on distance estimation in virtual reality (VR) has well established that even for geometrically accurate virtual objects and environments users tend to systematically mis-estimate distances. This has implications for Social VR, where it introduces variables in personal space and proxemics behavior that change social behaviors compared to the real world. One yet unexplored factor is related to the trend that avatars' embodied cues in Social VR are often scaled, e.g., by making one's head bigger or one's voice louder, to make social cues more pronounced over longer distances. In this paper we investigate how the perception of avatar distance is changed based on two means for scaling embodied social cues: visual head scale and verbal volume scale. We conducted a human-subject study employing a mixed factorial design with two Social VR avatar representations (full-body, head-only) as a between factor as well as three visual head scales and three verbal volume scales (up-scaled, accurate, down-scaled) as within factors. For three distances from social to far-public space, we found that visual head scale had a significant effect on distance judgments and should be tuned for Social VR, while conflicting verbal volume scales did not, indicating that voices can be scaled in Social VR without immediate repercussions on spatial estimates. We discuss the interactions between the factors and implications for Social VR.

Augmented reality (AR) technologies provide a shared platform for users to collaborate in a physical context involving both real and virtual content. To enhance the quality of interaction between AR users, researchers have proposed augmenting users' interpersonal space with embodied cues such as their gaze direction. While beneficial in achieving improved interpersonal spatial communication, suchshared gaze environmentssuffer from multiple types of errors related to eye tracking and networking, that can reduce objective performance and subjective experience. In this paper, we present a human-subjects study to understand the impact ofaccuracy,precision,latency, anddropoutbased errors on users' performance when using shared gaze cues to identify a target among a crowd of people. We simulated varying amounts of errors and the target distances and measured participants' objective performance through their response time and error rate, and their subjective experience and cognitive load through questionnaires. We found significant differences suggesting that the simulated error levels had stronger effects on participants' performance than target distance with accuracy and latency having a high impact on participants' error rate. We also observed that participants assessed their own performance as lower than it objectively was. We discuss implications for practical shared gaze applications and we present a multi-user prototype system.

Introduction We introduce a new type of patient simulator referred to as the Physical-Virtual Patient Simulator (PVPS). The PVPS combines the tangible characteristics of a human-shaped physical form with the flexibility and richness of a virtual patient. The PVPS can exhibit a range of multisensory cues, including visual cues (eg, capillary refill, facial expressions, appearance changes), auditory cues (eg, verbal responses, heart sounds), and tactile cues (eg, localized temperature, pulse). Methods We describe the implementation of the technology, technical testing with healthcare experts, and an institutional review board–approved pilot experiment involving 22 nurse practitioner students interacting with a simulated child in 2 scenarios: sepsis and child abuse. The nurse practitioners were asked qualitative questions about ease of use and the cues they noticed. Results Participants found it easy to interact with the PVPS and had mixed but encouraging responses regarding realism. In the sepsis scenario, participants reported the following cues leading to their diagnoses: temperature, voice, mottled skin, attitude and facial expressions, breathing and cough, vitals and oxygen saturation, and appearance of the mouth and tongue. For the child abuse scenario, they reported the skin appearance on the arms and abdomen, perceived attitude, facial expressions, and inconsistent stories. Conclusions We are encouraged by the initial results and user feedback regarding the perceived realism of visual (eg, mottling), audio (eg, breathing sounds), and tactile (eg, temperature) cues displayed by the PVPS, and ease of interaction with the simulator.

It is made from wood, projectors, and a computer running multiprojector blending software. Developed in 2008–2009 in response to an agency interest in future work environments for intelligence analysts, the immersive curved ADesk supports a variety of tasks, such as traditional office or development work, design, and even immersive telepresence. The authors discuss the A-Desk (“Analyst Desk”), a user-enveloping display system consisting of a bowl-shaped, two-dimensionally curved display and work surface. The A-Desk concept and working prototypes were developed after consultations with the U.S. National Geospatial-Intelligence Agency (NGA) and with support from the U.S. Intelligence Advanced Research Projects Activity (IARPA) (Future Analyst Workspace (A-Desk), Principal Investigators Henry Fuchs and Greg Welch, U.S. Air Force Office of Scientific Research, Agency Number FA8750-08-2-0209, IARPA Analyst Workspace for Exploitation (A-SpaceX) Program, Program Manager Dr. Jeffrey G. Morrison). The ADesk aims to address these agencies’ need for innovative display technologies for use by analysts whose complex daily tasks include visualizing intelligence and surveillance data from multiple sources comprising different modalities, developing and managing hypotheses about past and future activities, scenarios, or events, and communicating such information to, as well as discussing and processing it with, collaborators.

INTRODUCTION:We introduce a new type of patient simulator referred to as the Physical-Virtual Patient Simulator (PVPS). The PVPS combines the tangible characteristics of a human-shaped physical form with the flexibility and richness of a virtual patient. The PVPS can exhibit a range of multisensory cues, including visual cues (eg, capillary refill, facial expressions, appearance changes), auditory cues (eg, verbal responses, heart sounds), and tactile cues (eg, localized temperature, pulse).METHODS:We describe the implementation of the technology, technical testing with healthcare experts, and an institutional review board-approved pilot experiment involving 22 nurse practitioner students interacting with a simulated child in 2 scenarios: sepsis and child abuse. The nurse practitioners were asked qualitative questions about ease of use and the cues they noticed.RESULTS:Participants found it easy to interact with the PVPS and had mixed but encouraging responses regarding realism. In the sepsis scenario, participants reported the following cues leading to their diagnoses: temperature, voice, mottled skin, attitude and facial expressions, breathing and cough, vitals and oxygen saturation, and appearance of the mouth and tongue. For the child abuse scenario, they reported the skin appearance on the arms and abdomen, perceived attitude, facial expressions, and inconsistent stories.CONCLUSIONS:We are encouraged by the initial results and user feedback regarding the perceived realism of visual (eg, mottling), audio (eg, breathing sounds), and tactile (eg, temperature) cues displayed by the PVPS, and ease of interaction with the simulator.

Virtual reality (VR) technologies provide a shared platform for collaboration among users in a spatial context. To enhance the quality of social signals during interaction between users, researchers and practitioners started augmenting users’ interpersonal space with different types of virtual embodied social cues. A prominent example is commonly referred to as the "Big Head" technique, in which the head scales of virtual interlocutors are slightly increased to leverage more of the display’s visual space to convey facial social cues. While beneficial in improving interpersonal social communication, the benefits and thresholds of human perception of facial cues and comfort in such Big Head environments are not well understood, limiting their usefulness and subjective experience.In this paper, we present a human-subject study that we conducted to understand the impact of an increased or decreased head scale in social VR on participants’ ability to perceive facial expressions as well as their sense of comfort and feeling of "uncanniness." We explored two head scaling methods and compared them with respect to perceptual thresholds and user preferences. We further show that the distance to interlocutors has an important effect on the results. We discuss implications and guidelines for practical applications that aim to leverage VR-enhanced social cues.

Embodied virtual agents serving as patient simulators are widely used in medical training scenarios, ranging from physical patients to virtual patients presented via virtual and augmented reality technologies. Physical-virtual patients are a hybrid solution that combines the benefits of dynamic visuals integrated into a human-shaped physical form that can also present other cues, such as pulse, breathing sounds, and temperature. Sometimes in simulation the visuals and shape do not match. We carried out a human-participant study employing graduate nursing students in pediatric patient simulations comprising conditions associated with matching/non-matching of the visuals and shape.