Adaptive steered molecular dynamics (ASMD) is a computational biophysics method in which an external force is applied to a selected set of atoms or a specific reaction coordinate to induce a particular molecular motion. Virtual reality (VR) based methods for protein-ligand docking are beneficial for visualizing on-the-fly interactive molecular dynamics and performing promising docking trajectories. In this paper, we propose a novel method to guide ASMD with optimal trajectories collected from human experiences using interactive molecular dynamics in virtual reality (iMD-VR). We also explain the benefits of using VR as a tool for expediting the process of ligand binding, outlining an experimental protocol that enables iMD-VR users to guide Amprenavir into and out of the binding pockets of HIV-1 protease and recreate their respective crystallographic binding poses within 5 minutes. Later, we discuss our analysis of the results from iMD-VR-assisted ASMD simulation and assess its performance compared to a standard ASMD simulation. From the accuracy point of view, our proposed method calculates higher Potential Mean Force (PMF) values consistently relative to a standard ASMD simulation with an almost twofold increase in all the experiments. Finally, we describe the novelty of the research and discuss results showcasing a faster and more effective convergence of the ligand to the protein's binding site as compared to a standard molecular dynamics simulation, proving the effectiveness of VR in the field of drug discovery. Future work includes the development of an artificial intelligence algorithm capable of predicting optimal binding trajectories for many protein-ligand pairs, as well as the required force needed to steer the ligand to follow the said trajectory.

Organized by experts from across academia, industry, the Federal Labs, and SPIE, this meeting will highlight emerging capabilities in immersive technologies and degraded visual environments as critical enablers to future multi-domain operations.

Electroencephalography (EEG)-based brain–computer interfaces (BCIs) for motor rehabilitation aim to “close the loop” between attempted motor commands and sensory feedback by providing supplemental information when individuals successfully achieve specific brain patterns. Existing EEG-based BCIs use various displays to provide feedback, ranging from displays considered more immersive (e.g., head-mounted display virtual reality (HMD-VR)) to displays considered less immersive (e.g., computer screens). However, it is not clear whether more immersive displays improve neurofeedback performance and whether there are individual performance differences in HMD-VR versus screen-based neurofeedback. In this pilot study, we compared neurofeedback performance in HMD-VR versus a computer screen in 12 healthy individuals and examined whether individual differences on two measures (i.e., presence, embodiment) were related to neurofeedback performance in either environment. We found that, while participants’ performance on the BCI was similar between display conditions, the participants’ reported levels of embodiment were significantly different. Specifically, participants experienced higher levels of embodiment in HMD-VR compared to a computer screen. We further found that reported levels of embodiment positively correlated with neurofeedback performance only in HMD-VR. Overall, these preliminary results suggest that embodiment may relate to better performance on EEG-based BCIs and that HMD-VR may increase embodiment compared to computer screens.

This study explores two hypotheses about human-agent teaming: 1. Real-time coordination among a large set of autonomous robots can be achieved using predefined "plays" which define how to execute a task, and "audibles" which modify the play on the fly. 2. A spokesperson agent can serve as a representative for a group of robots, relaying information between the robots and human teammates. These hypotheses are tested in a simulated game environment: a human participant leads a search-and-rescue operation to evacuate a town threatened by an approaching wildfire, with the object of saving as many lives as possible. The participant communicates verbally with a virtual agent controlling a team of ten aerial robots and one ground vehicle, while observing a live map display with real-time location of the fire and identified survivors. Since full automation is not currently possible, two human controllers control the agent's speech and actions, and input parameters to the robots, which then operate autonomously until the parameters are changed. Designated plays include monitoring the spread of fire, searching for survivors, broadcasting warnings, guiding residents to safety, and sending the rescue vehicle. A successful evacuation of all the residents requires personal intervention in some cases (e.g., stubborn residents) while delegating other responsibilities to the spokesperson agent and robots, all in a rapidly changing scene. The study records the participants' verbal and nonverbal behavior in order to identify strategies people use when communicating with robotic swarms, and to collect data for eventual automation.

Head mounted displays (HMD) are becoming ubiquitous. Simulator sickness has been an issue since the first simulators and HMDs were created. As computational power and display capabilities increase, so does their utilization in technologies such as HMDs. However, this does not mean that the issues that once plagued these systems are now obsolete. In fact, evidence suggests that these issues have become more prevalent. Whether the system is Augmented Reality (AR), Virtual Reality (VR), or Mixed Reality (MR) the issues associated with simulator sickness or cybersickness have become more widespread. The reasons are uncertain, but probably multiple. One possible reason is the concept of vection, which is the illusion of movement to the participant where there is none physically. Vection plays a vital role in immersion and presence, however; it is also integral in simulator sickness. Another potential reason is the availability of HMDs. Traditionally a tool used in military training or laboratory settings, HMDs have now become a consumer item. This work reviews the current state of HMD issues such as simulator sickness or cybersickness. It reviews the similarities and differences of the sickness states that are commonly found with HMDs. Also, terms such as presence and immersion are delineated so they are used appropriately. The current theories on simulator sickness and cybersickness are reviewed. Further, the measurement and mitigation strategies currently being employed to reduce sickness are reviewed. Lastly, suggestions for more accurate measurement are recommended.

Virtual reality (VR) has been widely utilized for training and education purposes because of pedagogical, safety, and economic benefits. The investigation of moral judgment is a particularly interesting VR application, related to training For this study, we designed a within-subject experiment manipulating the role of study participants in a Trolley Dilemma scenario: either victim or driver. We conducted a pilot study with four participants and describe preliminary results and implications in this poster.

Search and rescue missions involving robots face multiple challenges. The ratio of operators to robots is frequently one to one or higher, operators tasked with robots must contend with cognitive overload for long periods, and the robots themselves may be discomfiting to located survivors. To improve on the current state, we propose a swarm of robots equipped with natural language abilities and guided by a central virtual “spokesperson” able to access “plays”. The spokesperson may assist the operator with tasking the robots in their exploration of a zone, which allows the operator to maintain a safe distance. The use of multiple robots enables rescue personnel to cover a larger swath of ground, and the natural language component allows the robots to communicate with survivors located on site. This capability frees the operator to handle situations requiring personal attention, and overall can accelerate the location and assistance of survivors. In order to develop this system, we are creating a virtual reality simulation, in order to conduct a study and analysis of how humans communicate with these swarms of robots. The data collected from this experiment will inform how to best design emergency response swarm robots that are effectively able to communicate with the humans around them.

Be it discussed as cybersickness, immersive sickness, simulator sickness, or virtual reality sickness, the ill effects of visuo-vestibular mismatch in immersive environments are of great concern for the wider adoption of virtual reality and related technologies. In this position paper, we discuss a unified research approach that may address motion sickness and identify critical research topics.

Virtual reality, augmented reality, and other forms of virtual environments have the potential to dramatically change how individuals work, learn, and interact with each other. A key objective of human factors research and practice is to determine how these environments should be designed to maximize performance efficiency, ensure health and safety, and circumvent potential human virtual environment interaction problems. This session will demonstrate some of the distinct and diverse uses of virtual reality, mixed reality, and virtual environments in an alternative format. The session will begin with each demonstrator providing a brief overview of their virtual environment and describing how it has been used to address a particular problem or research need. Following the description portion of the session, all demonstrations will be set-up around the room, and session attendees will be encouraged to directly interact with the environment and ask demonstrators questions about their research and inquire about the effectiveness of using their virtual environment for research, training, and evaluation purposes. The overall objective of this alternative session is to provoke ideas among the attendees for how virtual reality, mixed reality, and virtual environments can help address their research, training, education or business needs.

It is known that observers tend to misperceive distances during spatial judgment tasks in virtual reality. Virtual environments restrict field of view as compared to real environments. We explore whether horizontal field of view restriction affects real or perceived ocular convergence. We also explore effects that the size and symmetry of field of view may have. We find that convergence is not impacted by altering field of view, but the subjective median plane is affected. We also find that distance is better estimated in wider fields of view, and that lateral bias in estimation is correlated to the symmetry of the field.

Spatial perception in virtual environments has been a topic of intense research. Arguably, the majority of this work has focused on distance perception. However, orientation perception is also an important factor. In this paper, we systematically investigate allocentric orientation judgments in both real and virtual contexts over the course of four experiments. A pattern of sinusoidal judgment errors known to exist in 2D perspective displays is found to persist in immersive virtual environments. This pattern also manifests itself in a real world setting using two differing judgment methods. The findings suggest the presence of a radial anisotropy that persists across viewing contexts. Additionally, there is some evidence to suggest that observers have multiple strategies for processing orientations but further investigation is needed to fully describe this phenomenon. We also offer design suggestions for 3D user interfaces where users may perform orientation judgments.

We explored how users perceive virtual characters that performed the role of a counseling interviewer, while presenting different levels of social class, as well as single or multi-tasking behavior. To investigate this subject, we designed a 2x2 experiment (tasking type and social class of the virtual counseling interviewer). In the experiment, participants experienced the counseling interview interactions over video conferencing on a smartphone. We measured user responses to and perceptions of the virtual human interviewer. The results demonstrate that the tasking types and social class of the virtual counselor affected user responses to and perceptions of the virtual counselor. The results offer insight into the design and development of effective, realistic, and believable virtual human counselors. Furthermore, the results also address current social questions about how smartphones might mediate social interactions, including human-agent interactions.

The emergence of low cost virtual and augmented reality systems has encouraged the development of immersive training applications for medical, military, and many other fields. Many of the training scenarios for these various fields may require the presentation of realistic interactions with virtual humans. It is thus vital to determine the critical factors of fidelity required in those interactions to elicit naturalistic behavior on the part of trainees. Negative training may occur if trainees are inadvertently influenced to react in ways that are unexpected and unnatural, hindering proper learning and transfer of skills and knowledge back into real world contexts. In this research, we examined whether haptic priming (presenting an illusion of virtual human touch at the beginning of the virtual experience) and different locomotion techniques (either joystick or physical walking) might affect proxemic behavior in human users. The results of our study suggest that locomotion techniques can alter proxemic behavior in significant ways. Haptic priming did not appear to impact proxemic behavior, but did increase rapport and other subjective social measures. The results suggest that designers and developers of immersive training systems should carefully consider the impact of even simple design and fidelity choices on trainee reactions in social interactions.

A research agenda for intelligent systems that will result in fundamental new capabilities for understanding the Earth system.

Recent advances in 3D scanning, reconstruction, and animation techniques have made it possible to rapidly create photorealistic avatars based on real people. While it is now possible to create personalized avatars automatically with consumer-level technology, their visual fidelity still falls far short of 3D avatars created with professional cameras and manual artist effort. To evaluate the importance of investing resources in the creation of high-quality personalized avatars, we conducted an experiment to investigate the effects of varying their visual texture fidelity, specifically focusing on identity recognition of specific individuals. We designed two virtual reality experimental scenarios: (1) selecting a specific avatar from a virtual lineup and (2) searching for an avatar in a virtual crowd. Our results showed that visual fidelity had a significant impact on participants' abilities to identify specific avatars from a lineup wearing a head-mounted display. We also investigated gender effects for both the participants and the confederates from which the avatars were created.

Projected displays can present life-sized imagery of a virtual human character that can be seen by multiple observers. However, typical projected displays can only render that virtual human from a single viewpoint, regardless of whether head tracking is employed. This results in the virtual human being rendered from an incorrect perspective for most individuals in a group of observers. This could result in perceptual miscues, such as the “Mona Lisa” effect, causing the virtual human to appear as if it is simultaneously gazing and pointing at all observers in the room regardless of their location. This may be detrimental to training scenarios in which all trainees must accurately assess where the virtual human is looking or pointing a weapon. In this paper, we discuss our investigations into the presentation of eye gaze using REFLCT, a previously introduced head mounted projective display. REFLCT uses head tracked, head mounted projectors and retroreflective screens to present personalized, perspective correct imagery to multiple users without the occlusion of a traditional head mounted display. We examined how head mounted projection for enhanced presentation of eye gaze might facilitate or otherwise affect social interactions during a multi-person guessing game of “Twenty Questions.”

There are few effective treatments for rehabilitation of severe motor impairment after stroke. We developed a novel closed-loop neurofeedback system called REINVENT to promote motor recovery in this population. REINVENT (Rehabilitation Environment using the Integration of Neuromuscular-based Virtual Enhancements for Neural Training) harnesses recent advances in neuroscience, wearable sensors, and virtual technology and integrates low-cost electroencephalography (EEG) and electromyography (EMG) sensors with feedback in a head-mounted virtual reality display (VR) to provide neurofeedback when an individual's neuromuscular signals indicate movement attempt, even in the absence of actual movement. Here we describe the REINVENT prototype and provide evidence of the feasibility and safety of using REINVENT with older adults.

We explored the social influence of humor in a virtual human counselor's self-disclosure while also varying the ethnicity of the virtual counselor. In a 2x3 experiment (humor and ethnicity of the virtual human counselor), participants experienced counseling interview interactions via Skype on a smartphone. We measured user responses to and perceptions of the virtual human counselor. The results demonstrate that humor positively affects user responses to and perceptions of a virtual counselor. The results further suggest that matching styles of humor with a virtual counselor's ethnicity influences user responses and perceptions. The results offer insight into the effective design and development of realistic and believable virtual human counselors. Furthermore, they illuminate the potential use of humor to enhance self-disclosure in human-agent interactions.