AbstractThe Internet of Production (IoP) promises to be the answer to major challenges facing the Industrial Internet of Things (IIoT) and Industry 4.0. The lack of inter-company communication channels and standards, the need for heightened safety in Human Robot Collaboration (HRC) scenarios, and the opacity of data-driven decision support systems are only a few of the challenges we tackle in this chapter. We outline the communication and data exchange within the World Wide Lab (WWL) and autonomous agents that query the WWL which is built on the Digital Shadows (DS). We categorize our approaches into machine level, process level, and overarching principles. This chapter surveys the interdisciplinary work done in each category, presents different applications of the different approaches, and offers actionable items and guidelines for future work.The machine level handles the robots and machines used for production and their interactions with the human workers. It covers low-level robot control and optimization through gray-box models, task-specific motion planning, and optimization through reinforcement learning. In this level, we also examine quality assurance through nonintrusive real-time quality monitoring, defect recognition, and quality prediction. Work on this level also handles confidence, verification, and validation of re-configurable processes and reactive, modular, transparent process models. The process level handles the product life cycle, interoperability, and analysis and optimization of production processes, which is overall attained by analyzing process data and event logs to detect and eliminate bottlenecks and learn new process models. Moreover, this level presents a communication channel between human workers and processes by extracting and formalizing human knowledge into ontology and providing a decision support by reasoning over this information. Overarching principles present a toolbox of omnipresent approaches for data collection, analysis, augmentation, and management, as well as the visualization and explanation of black-box models.

Teaching in optical systems design is usually performed on an optical bench. While experimentation plays an important role in education, experiments involving expensive or dangerous components are usually limited to short, heavily supervised sessions. Computer simulations, on the other hand, offer high accessibility, but suffer from reduced realism and tangibility when presented on a 2D screen. For this reason, we present the virtual optical bench, an application that lets users explore spherical lens layouts in virtual reality (VR). We implemented a numerically accurate simulation of optical systems using Nvidia OptiX, as well as a prototypical VR application, which we then evaluated in an expert review with 6 optics experts. Based on their feedback, we re-implemented our VR application in Unreal Engine 4. The re-implementation has since been actively used for teaching optical layouts, where we performed a qualitative evaluation with 18 students. We show that our virtual optical bench achieves good usability and is perceived to enhance the understanding of course contents.

In the machining of production parts, the quality of the surface is subject to a variety of influences. Tool wear, thermal effects and vibrations can affect the surface quality of the machined object drastically. Some of those effects on the surface, like marks from vibrations, can be seen by machining experts directly on the object without measurement hardware or metrology systems. Those pieces of information held by the experts are valuable to the surface finishing staff, the metrology staff and data scientists, who could use them to improve both the product and the process. However, to enable this exchange of information, the relevant data needs to be indicated in a spatially resolved and machine-readable format. Additionally, the experts need to be able to mark the areas for all kinds of different requirements in a fast and suitable manner without possibly damaging the part itself. Our approach for this task is using a touch-based Augmented Reality system that allows the users to mark these areas of interest on the production part digitally without touching the production part itself. Therefore, we propose two different marking techniques - Shift&Freeze Marking and Relative Marking - based on literature from pointing tasks. We performed user trials to evaluate the usability, task load and overall performance of those techniques showing a general suitability for touch-based AR marking of relevant regions on productions parts.

Background Although nearly one-third of the world’s disease burden requires surgical care, only a small proportion of digital health applications are directly used in the surgical field. In the coming decades, the application of augmented reality (AR) with a new generation of optical-see-through head-mounted displays (OST-HMDs) like the HoloLens (Microsoft Corp) has the potential to bring digital health into the surgical field. However, for the application to be performed on a living person, proof of performance must first be provided due to regulatory requirements. In this regard, cadaver studies could provide initial evidence. Objective The goal of the research was to develop an open-source system for AR-based surgery on human cadavers using freely available technologies. Methods We tested our system using an easy-to-understand scenario in which fractured zygomatic arches of the face had to be repositioned with visual and auditory feedback to the investigators using a HoloLens. Results were verified with postoperative imaging and assessed in a blinded fashion by 2 investigators. The developed system and scenario were qualitatively evaluated by consensus interview and individual questionnaires. Results The development and implementation of our system was feasible and could be realized in the course of a cadaver study. The AR system was found helpful by the investigators for spatial perception in addition to the combination of visual as well as auditory feedback. The surgical end point could be determined metrically as well as by assessment. Conclusions The development and application of an AR-based surgical system using freely available technologies to perform OST-HMD–guided surgical procedures in cadavers is feasible. Cadaver studies are suitable for OST-HMD–guided interventions to measure a surgical end point and provide an initial data foundation for future clinical trials. The availability of free systems for researchers could be helpful for a possible translation process from digital health to AR-based surgery using OST-HMDs in the operating theater via cadaver studies.

The design of optical lens assemblies is a difficult process that re-quires lots of expertise. The teaching of this process today is done on physical optical benches, which are often too expensive for students to purchase. One way of circumventing these costs is to use software to simulate the optical bench. This work presents a virtual optical bench, which leverages realtime ray tracing in combination with VR rendering to create a teaching tool which creates a repeatable, non-hazardous and feature-rich learning environment. The resulting application was evaluated in an expert review with 6 optical engineers.

The black box problem of artificial neural networks (ANNs) is still a very relevant issue. When communicating basic concepts of ANNs, they are often depicted as node-link diagrams. Despite this being a straight forward way to visualize them, it is rarely used outside an educational context. However, we hypothesize that large-scale node-link diagrams of full ANNs could be useful even to machine learning experts. Hence, we present a visualization tool that depicts convolutional ANNs as node-link diagrams using immersive virtual reality. We applied our tool to a use-case in the field of machine learning research and adapted it to the specific challenges. Finally, we performed an expert review to evaluate the usefulness of our visualization. We found that our node-link visualization of ANNs was perceived as helpful in this professional context.

Applications of Virtual Reality (VR) have been repeatedly explored with the goal to improve the data analysis process of users from different application domains, such as architecture and simulation sciences. Unfortunately, making VR available in professional application scenarios or even using it on a regular basis has proven to be challenging. We argue that everyday usage environments, such as office spaces, have introduced constraints that critically affect the design of interaction concepts since well-established techniques might be difficult to use. In our opinion, it is crucial to understand the impact of usage scenarios on interaction design, to successfully develop VR applications for everyday use. To substantiate our claim, we define three distinct usage scenarios in this work that primarily differ in the amount of mobility they allow for. We outline each scenario's inherent constraints but also point out opportunities that may be used to design novel, well-suited interaction techniques for different everyday usage environments. In addition, we link each scenario to a concrete application example to clarify its relevance and show how it affects interaction design.

In this work we describe the scenario of fully-immersive desktop VR, which serves the overall goal to seamlessly integrate with existing workflows and workplaces of data analysts and researchers, such that they can benefit from the gain in productivity when immersed in their data-spaces. Furthermore, we provide a literature review showing the status quo of techniques and methods available for realizing this scenario under the raised restrictions. Finally, we propose a concept of an analysis framework and the decisions made and the decisions still to be taken, to outline how the described scenario and the collected methods are feasible in a real use case.

This paper introduces the RWTH-PHOENIX-Weather 2014, a video-based, large vocabulary, German sign language corpus which has been extended over the last two years, tripling the size of the original corpus. The corpus contains weather forecasts simultaneously interpreted into sign language which were recorded from German public TV and manually annotated using glosses on the sentence level and semi-automatically transcribed spoken German extracted from the videos using the open-source speech recognition system RASR. Spatial annotations of the signers’ hands as well as shape and orientation annotations of the dominant hand have been added for more than 40k respectively 10k video frames creating one of the largest corpora allowing for quantitative evaluation of object tracking algorithms. Further, over 2k signs have been annotated using the SignWriting annotation system, focusing on the shape, orientation, movement as well as spatial contacts of both hands. Finally, extended recognition and translation setups are defined, and baseline results are presented.