This paper discusses a Juxtopia® Open-Wear research collaboration with the Maryland Fire & Rescue Institute (MFRI) to evaluate how an artificial intelligent (AI) wearable augmented reality (AR) intervention quantifiably improves hands-free clinical training proficiency of firefighter United States (U.S.) Emergency Medical Technicians (EMT) and prepares them for administering effective clinical skills (e.g., Tourniquet) in response to hazardous materials (hazmat) incidences. The AI-AR system, Juxtopia® Context-Aware Mobile Mixed Reality Assistive Device (CAMMRAD) Prepare E-Training system was evaluated in a two day controlled study including firefighter EMTs subjects who participated in simulated training exercise that mimicked their real-world operations. Results of the study indicates a need for AI-AR training to continually improve EMT clinical skill proficiency.

A Juxtopia research team collaborated with the Maryland Fire & Rescue Institute (MFRI) to test how the Juxtopia® wearable augmented reality (AR) intervention may better deliver a hands-on clinical training intervention to firefighter Emergency Medical Technicians (EMT) and prepare them for effective response to HAZMAT (hazardous materials) incidences. During a controlled study, human subjects participated in a minimal risk research study (i.e., both as victims or caregivers) in which firefighter EMTs participated in a simulated training exercise that mimicked their real-world operations. This study consisted of two testing days. Day one included 10 participants that completed a full day of training and familiarity with wearable augmented reality (AR) goggles and a Juxtopia® Virtual Tutor (JVT) software application, which is derived from the Juxtopia® Intelligent Virtual Instructor (JiVi) platform. Day two included the completion of four psychomotor clinical skills performed by EMTs. It also included the application of moulage injuries for victims, using the Juxtopia® CAMMRAD PREPARE system to administer clinical skills on patient actors. This paper discusses the research methods and results.

One cause of preventable death is a lack of proper skills for providing critical care. The conventional course taught to non-medical individuals involves instructions of advanced emergency procedures routinely limited to a verbal block of instructions in a standardized presentation (for example, an instructional video).In the present study, we evaluate the benefits of using an OST-HMD for training of caregivers in an emergency medical environment. A rich user interface was implemented that provides 3D visual aids including images, text and tracked 3D overlays corresponding to each task that needs to be performed. A user study with 20 participants is conducted which involves training of two tasks where each subject performs one task with the HMD and the other with standard training. Two evaluations were performed, with the first immediately after the training followed by a second one three weeks later. Our results indicate that using a mixed reality HMD is more engaging, improves the time-on-task, and increases the confidence level of users in providing emergency and critical care.

One major cause of preventable death is a lack of proper skills for providing critical care. Conventional training for advanced emergency medical procedures is often limited to a verbal block of instructions and/or an instructional video. In this study, we evaluate the benefits of using an optical see-through head-mounted display (OST-HMD) for training of caregivers in an emergency medical environment. A rich user interface was implemented that provides 3D visual aids including images, text and tracked 3D overlays for each task. A user study with 20 participants was conducted for two medical tasks, where each subject received conventional training for one task and HMD training for the other task. Our results indicate that using a mixed reality HMD is more engaging, improves the time-on-task, and increases the confidence level of users.

Researchers from Morgan State University (i.e., Dr. Kadir Aslan) and Juxtopia (i.e., Dr. Jayfus Doswell) constructed a prototype crystallization device (PCD) based on the MA-MAEC technique for the rapid crystallization of peptides and proteins within minutes. In contrast, traditional PCDs typically take up to several weeks to complete using conventional crystallization techniques. Researchers tested the resulting PCD device on the Amyloid precursor protein (APP) and its components as biological macromolecules of interest. APP is most commonly studied as the precursor molecule whose proteolysis generates beta amyloid (Aβ42) peptide whose amyloid fibrillar form is the primary component of amyloid plaques found in the brains of Alzheimer's disease patients. Purified APP and synthetic peptides [Aβ42 (42 amino acid residues), copper binding domain (CuBD, 63 amino acid residues) and growth factor-like domain (GFLD, 105 amino acid residues) of APP] were crystallized using microwave heating on the innovative PCD crystallization platforms to demonstrate the proof-of-principle use of the iCrystal system based on multi-parameter assessment: 1) Time of crystallization and microwave power; 2) Size of crystals; and 3) Quality of crystals. The control experiments included: 1) No microwave heating; 2) No SNFs and no microwave heating to assess the effectiveness of the MA-MAEC technique for the rapid crystallization of APP and its relevant components. The iCrystal PCD time was significantly less than traditional PCD with a higher quality crystal. The crystallization experiments for APP and its relevant peptides were repeated separately a minimum of 50 different 21-well PMMA platform to assess the reproducibility of the technique. In this regard, the variation in the size, crystallization time, and quality of APP crystals remain similar. The researched MA-MAEC technique, within the iCrystal system, demonstrated a potential to revolutionize the way peptides and proteins are crystallized as it relates to human diseases, which will afford for rapid development of treatments.

Wearable Augmented Reality (AR) combines research in AR, mobile/ubiquitous computing, and human ergonomics in which a video or optical see-through head mounted display (HMD) facilitates multi-modal delivery of contextually relevant and computer generated visual and auditory data over a physical, real-world environment. Wearable AR has the capability of delivering on-demand assistance and training across a variety of domains. A primary challenge presented by such advanced HCI technologies is the development of scientifically-grounded methods for identifying appropriate information presentation, user input, and feedback modalities in order to optimize performance and mitigate cognitive overload. A proposed framework and research methodology are described to support instantiation of physiologically-driven, adaptive AR to assess and contextually adapt to an individual’s environmental and cognitive state in real time. Additionally a use case within the medical domain is presented, and future research is discussed.

We present the design of a self-contained head-mounted surgical navigation system, which consists of an optical tracking system and an optical see-through head-mounted display (HMD). While the current prototype is bulky, we envision a more compact solution via the eventual integration of the tracking camera(s) into the HMD goggles. Rather than attempting to accurately overlay preoperative models onto the field of view, we adopted a simpler approach of displaying a small "picture-in-picture" virtual view in the HMD. We believe this approach will provide suitable assistance for some image-guided procedures, such as tumor resection, while improving the ergonomics by reducing the need for the surgeon to look away from the patient to view an external monitor. We report the results of initial experiments performed with this system, while preparing for a more clinically realistic study.

Precise tumor identification is crucial in image-guided neurosurgical procedures. With existing navigation systems, the surgeon must turn away from the patient to view the imaging data on a separate monitor. In this study, an innovative system is introduced that illustrates the tumor boundaries precisely augmented on the spot where the tumor is located with regard to the patient. Additionally, it allows the surgeon to track the distal end of the tools contextually, where direct visualization is not possible. In this approach, the tracking system is compact and worn by the surgeon, eliminating the need for additional devices that are bulky and typically limited by line of sight constraints.

An electrophotographic printer includes a photosensitive member, a charger for accumulating electrostatic charges on the surface of the photosensitive member, an optical writing member for forming an electrostatic latent image on the surface of the photosensitive member, a developing device for making the latent image on the photosensitive member visible by means of a developer, a transfer device for transferring the visible image to a recorded material, a charge remover for removing any charge remaining on the photosensitive member, and a cleaner for removing any developer remaining on the photosensitive member. A developer moving passage is defined between the developing device and the cleaner for moving the developer from the cleaner to the developing device. The charger and the optical writing member are disposed between the developing device and the cleaner. A fixing device for fixing a visible image which has been transferred from the photosensitive member to the recording material is disposed in the vicinity of an upper wall of a housing, and an opening is provided in the upper wall of the housing for discharging the recording material passed through the fixing device onto the upper wall.

This workshop will expose participants to how autonomous Virtual Instructors (VI) can be delivered through wearable augmented reality (AR) to provide a personalized and just-in-time instructional intervention during psychomotor learning and play. Distributing VIs for teaching or improving psychomotor skills through wearable AR, provides individual learners with a continually available personal tutor while, at the same time, keeping their hands free to practice a range of skills. These psychomotor skills may range from children learning basic electronics through robot assembly to learning the proper steps as a master plumber by following a VIs instructions. This workshop will address various pedagogical rules that a VI must follow in order to deliver the best instruction and how the multi-modal instructional intervention of a VI enabled wearable AR system can improve task learning and proficiency during learning and play.

One of the biggest challenges faced by most computer science educators is assessing whether a student comprehends programming and robotic design concepts. In this paper, we introduce the benefits of exploring new technologies for learning in the form of LEGO robotics and obtaining problem solving skills. Students use the LEGO (Mindstorms for Schools) Team Challenge kit #9790 in conjunction with a programming environment called ROBOLAB. Finally, we propose a Virtual Instructor as a mixed reality based instructional system that addresses these learning challenges and reduces the learning curve for robotics as well as enhances robotic instruction.

Virtual instructor enabled mobile augmented reality systems (MARS) have the potential to provide continuous and autonomous instruction to human learners anytime, anyplace, and at any-pace. MARS based learning provides the advantage of a natural human-computer interface, flexible mobility, and context-aware instruction allowing learners to interact with their natural environment with augmented perceptual cues. These perceptual cues combining multi-modal animation, graphics, text, video, and voice along with empirical pedagogical techniques can elegantly orchestrate a mobile instructional tool that facilitates life-long learning. The challenge, however, is building a mobile instructional tool with capabilities for adapting to various learning environments ranging from traditional schools and outdoor learning to the workplace while also considering the cultural, geographical, and other contexts about the learner. This paper discusses a novel system/software architecture, CAARS, for developing context-aware mobile augmented reality instructional systems

Mobile Augmented Reality Systems (MARS) e-learning has the potential to provide continuous, context-based, and autonomous instruction to human learners anytime, anyplace, and at any-pace. MARS e-learning enables mobility for the learner and hands free human computer interactivity. Advances to MARS based learning provides the advantage of a natural human-computer interface, flexible mobility, and context-aware instruction allowing learners to develop psychomotor skills while interacting with their natural environment with augmented perceptual cues. These perceptual cues combining multi-modal animation, graphics, text, video, and voice along with empirical instructional techniques can elegantly orchestrate a mobile instructional tool. The challenge, however, is building a MARS e-learning tool with capabilities for adapting to various learning environments while also considering the cultural, geographical, and other contexts about the learner. This paper discusses a novel system/software architecture, CAARS, for developing context-aware mobile augmented reality instructional systems.

Problem Solving Using LEGOS is a 3-credit hour course offered to any student at Pace University of Pleasantville, New York. The class uses a projectbased learning environment, which consists of four design projects. This paper discusses the pedagogical advantages of teaching robotics in a mixed-reality environment with a virtual instructor as opposed to teaching robotics in a traditional laboratory setting.

Scaffolding is a well-established instructional approach that facilitates learning by incrementally removing training aids as the learner progresses. By combining multiple training aids (i.e. multimodal interfaces), a trainer, either human or virtual, must make real-time decisions about which aids to remove throughout the training scenario. A significant problem occurs in implementing scaffolding techniques since the speed and choice of removing training aids must be strongly correlated to the individual traits of a specific trainee. We detail an agent-based infrastructure that supports the customization of scaffolding routines per individual user. We describe the integration of this agent-based approach into a simulated augmented reality (AR) environment

Problem solving using LEGOS is a 3-credit hour course offered to any student at Pace University of Pleasantville, New York. The class uses a project-based learning environment, which consists of four design projects. This paper discusses the pedagogical advantages of teaching robotics in a mixed-reality environment with a virtual instructor as opposed to teaching robotics in a traditional laboratory setting. The implication of this learning intervention is the personalized and immersive learning experience that has the potential for strengthening problem solving skills using science, technology, engineering, and math (STEM) for life-long STEM proficiency

Pedagogical Embodied Conversational Agents (PECA) that autonomously behave in mixed reality environments, respond to muti-modal input across computer networks, interact with human learners using context aware intelligence, and apply proven pedagogical techniques during instruction have the potential to improve and accelerate human learning performance anytime, anywhere, and at any-pace. This paper discusses the PECA Product Line Architecture (PPLA) model for building interactive pedagogical agent systems and discusses a prototype system from the architecture.

Building virtual reality instructors as 3D-animated characters that behave autonomously in networked virtual environments, responds to multi-modal input across computer networks, interact with human learners using context-aware intelligence, and apply proven pedagogical techniques during instruction exemplify the potential to provide human learning anytime, anywhere, at any pace. However, building a virtual reality instructor poses significant challenges for researchers due to interdisciplinary expertise required in areas such as cognitive science, sociology, computational humanities, artificial intelligence, 3D computer graphics, linguistics, and more.