We describe the design and development of an Extended Reality Advanced Trauma Life Support (ATLS) simulator that incorporates several ATLS scenarios. ATLS is a training program developed by the American College of Surgeons for teaching medical practitioners a systematic approach to treating trauma patients. The ATLS simulator is based on case-level data, which helps create reusable medical training scenarios. The simulation consists of three components, namely, incident history, initial assessment and resuscitation, and a secondary survey. It provides several scenarios for medical practitioners to perform the tasks from the ATLS checklist and practice diagnosing patients. The simulator can also predict the requirement of an ICU room, ventilator and the length of stay for a given trauma patient based on the type and severity of their injury. With our ATLS simulator we aim to provide medical practitioners a comprehensive training module for practicing emergency trauma response.

The COVID-19 pandemic has had a tremendous impact on businesses, educational institutions, and other organizations that require in-person gatherings. Physical gatherings such as conferences, classes, and other social activities have been greatly reduced in favor of virtual meetings on Zoom, Webex or similar video-conferencing platforms. However, video-conferencing is quite limited in its ability to create meeting spaces that capture the authentic feel of a real-world meeting. Without the aid of body language cues, meeting participants have a harder time paying attention and keeping themselves engaged in virtual meetings. Video-conferencing, as it currently stands, falls short of providing a familiar environment that fosters personal connection between meeting participants. This paper explores an alternative approach to virtual meetings through the use of extended reality (XR) and embodied interactions. We present an application that leverages the full-body tracking capabilities of the Azure Kinect and the immersive affordances of XR to create more vibrant and engaging remote meeting environments.

The outbreak of COVID-19 has put various restrictions on human lifestyle. At the beginning of the outbreak, almost all public spaces were closed to minimize the spread of this virus. Even as public spaces open up, they have several restrictions. Such restrictions include limited occupancy in common rooms to ensure social distancing and this can lead to increased occupancy costs inside buildings. The strategy of "Design as a cure" has been long used by architects and urban planners to minimize the spread of infectious diseases in urban environments. Re-configuring the space layout and optimizing the heating, ventilation, and air condition (HVAC) operations were some immediate solutions proposed by building designers to minimize the risk of COVID-19 infection in buildings. This paper explores the use of smart re-configurable spaces (SReS) to improve the efficiency of indoor space utilization while maintaining a safe indoor environment. We used an existing smart building design framework to design SReS for a common area/lounge in one of the cadet resident halls at Virginia Tech. User requirements were measured by conducting an interview with the residential coordinator and focus groups among the cadets. The concept of generative design was used in Revit 2021 to design various layouts of the lounge. Towards the end, we create a layout for maximum occupancy and suggest various re-configuration strategies. Future work includes modeling and evaluating the human-building interaction of SReS in virtual reality (VR).

The COVID-19 pandemic has greatly accelerated the digitization of services. As physical spaces become harder to access, there is a growing shift towards the use of virtual spaces for remote work, education and entertainment. In 2020, brick-and-mortar spaces like museums, art exhibits and galleries were especially affected by a lack of visitors. Shifting to a virtual medium would allow these entities to reach out and retain visitors more effectively. However, the use of virtual spaces to support these kinds of services is still quite under-explored. Presence, engagement and a real connection are difficult to establish through virtual exhibits. To explore these challenges, we partnered with the Liberation War Museum in Bangladesh and created a web-based 3D virtual museum to represent three of their historical galleries. Each virtual gallery has a different presentation modality - "self-guided", "avatar-guided" and "game-based". We sought to explain which of these artifact presentation modes led to the best performance in learnability, usability and engagement by conducting a user study. Our findings and user feedback are presented in this paper. We hope that these findings will be useful for designing virtual experiences that can allow users to learn and engage with virtual artifacts as effectively as they would with real-world ones.

Smart environments, comprised of networked embedded devices, improve the lives of their users by providing them with a variety of assistive services that traditional built environments are incapable of supporting. However, as the number of connected devices in smart environments continue to increase, so does the level of complexity involved in interacting with these environments. Traditional human-computer interaction techniques are not always well-suited for smart environments and this poses some unique usability challenges. To facilitate interactions within such technology-rich smart environments, new models and interaction interfaces need to be developed. In this paper we propose a multi-modal approach to smart environment interaction and explore two novel interaction interfaces: gesture-based interface and mixed-reality-based interface. We also conducted a user study to compare the learnability, efficiency and memorability of these new interfaces to two more commonly used interfaces: voice-based interface and a smartphone GUI-based interface. Our user study experiment involved four light control tasks that subjects were asked to complete using the four interaction interfaces. Study subjects found different interaction techniques to be more suitable for different tasks based on the type, complexity and context of the task. Our analysis of the study results and subject feedback suggest that a multi-modal approach is preferable to a uni-modal approach for interacting with smart environments. We suggest that novel interaction techniques be further explored in order to develop efficient multi-modal approaches along with the widely used techniques.

Mixed Reality (MR) technologies provide users with an environment that incorporates virtual objects and metadata into their physical surroundings. This opens up exciting new possibilities for applications in various domains including: education, training, healthcare, and Computer Supported Cooperative Work (CSCW). Recognizing physical objects in the environment provides contextual clues that can improve user task performance and influence effective cognitive load for complex tasks. However, this facet of MR is still relatively nascent because of hardware limitations. Presently, MR headsets can only detect physical surfaces; they can not distinguish between physical artifacts that a user might need in their work environment. Machine Learning (ML) based object recognition solutions can help overcome this limitation. However, current commercially available MR devices have insufficient computational capabilities and are unable to support the state-of-the-art ML based object recognition solutions. To address these challenges, we describe a novel approach and the corresponding framework by introducing a hardware level separation between the two tasks of object recognition and rendering virtual components. Our approach leverages ML based object recognition, MR, and a portable IoT edge computing platform to support contextual awareness. We describe the implementation of this approach and provide a case study. The prototype implementation uses an edge device to process and recognize objects to get information about the user’s physical environment and provides the user with situational awareness through the MR device. Our case study uses a Christmas tree decoration activity to demonstrate the ability of the proposed framework to support assembly-type tasks. The case study establishes the applicability of the proposed approach to a variety of problem domains. Object recognition and context aware capabilities, combined with virtual indicators and instructions, provide a solution that increases usability, helps reduce human error, and improves the overall task performance.

Smart Built Environments (SBEs) empowered by the Internet of Things (IoT) dramatically augment the capabilities of traditional built environments by imbuing everyday objects with computational and communication capabilities. SBEs primarily consist of three types of components: architectural elements, embedded technology (smart objects) and enhanced interaction modalities. As smart objects hold the ability to change the state of the environment, inefficient design of smart configurations can lead to potentially harmful conditions affecting the safety and security of the inhabitants. The interaction scenarios and space use pattern of SBEs are also notably different from traditional built environments. But, to the best of our knowledge, there has been limited work on developing a consolidated design framework addressing the three interdependent SBE elements and evaluating the safety and security of the IoT application environment. We propose an SBE design framework based on the traditional architectural design process. The framework combines the technological aspects of SBEs with the traditional architectural design process while leveraging Building Information Modeling (BIM) and participatory design. We describe a Mixed Reality(MR)-based reference framework implementation that is particularly helpful for representing, visualizing and modeling the vast amount of data, digital components and novel SBE interaction scenarios.

The recent advances in mixed reality (MR) technologies provide a great opportunity to support deployment and use of MR applications for training and education. Users can interact with virtual objects that can help them be more engaged and acquire more information compared to the more traditional approaches. MR devices, such as the Microsoft HoloLens device, use spatial mapping to place virtual objects in the surrounding space and support embodied interaction with those objects. However, some applications may require an extended range of embodied interactions that are beyond the capabilities of the MR device. For instance, interaction with virtual objects using arms, legs, and body almost the same way we interact with physical objects. We describe an approach to extend the functionality of Microsoft HoloLens to support an extended range of embodied interactions in an MR space by using the Microsoft Kinect V2 sensor device. Based on that approach, we developed a system that maps the captured skeletal data from the Kinect device to the HoloLens device coordinate system. We have measured the overall delay of the developed system to evaluate its effect on application responsiveness. The described system is currently being used for the development of a HoloLens application for nurse aide certification in the Commonwealth of Virginia.

In the era of emerging Smart Built Environments (SBEs), a smart house, unlike regular houses with static "components", consists of numerous interconnected and often actuated devices, capable of executing tasks independent of user supervision. Living in such a SBE, where for example, the furniture can rearrange itself, and the doors open and close of their own volition, may be difficult and unpredictable. Furthermore, cyber-security attacks and intrusion could allow attackers to assume control of the SBE, damage its components and to potentially harm its inhabitants. Such novel characteristics of SBEs present developers with several unique challenges with regards to implementing the needed safety and security measures and protocols that go along with them. With such environments, therefore, there is a need for a system that is capable of monitoring user activities in real-time, identifying the safety and security hazards to users in their immediate local context, warning users of these hazards, and perhaps even taking preventative and mitigative action against the hazards that it identified. In this paper, we survey some of these challenges and explore the design and implementation of a system designed around the safety and security of SBE inhabitants. We propose an approach to modeling SBE safety that combines the three laws of robotics and the swarm behavior model. We also present a preliminary prototype and discuss a case study.

The advances in mixed reality (MR) technologies provide an opportunity to support the deployment and use of MR for training and education. We describe an approach that extends the functionality of the Microsoft HoloLens device to support a wider range of embodied interactions by making use of the Microsoft Kinect V2 device. The embodied interactions can support novel interaction scenarios, especially within the context of training and skills development, thereby removing or reducing the need for training equipment.