Analysis of human motion data can reveal valuable insights about the utilization of space and interaction of humans with their environment. To support this, we present AvatAR, an immersive analysis environment for the in-situ visualization of human motion data, that combines 3D trajectories with virtual avatars showing people's detailed movement and posture. Additionally, we describe how visualizations can be embedded directly into the environment, showing what a person looked at or what surfaces they touched, and how the avatar's body parts can be used to access and manipulate those visualizations. AvatAR combines an AR HMD with a tablet to provide both mid-air and touch interaction for system control, as well as an additional overview device to help users navigate the environment. We implemented a prototype and present several scenarios to show that AvatAR can enhance the analysis of human motion data by making data not only explorable, but experienceable.

In this work we propose the combination of large interactive displays with personal head-mounted Augmented Reality (AR) for information visualization to facilitate data exploration and analysis. Even though large displays provide more display space, they are challenging with regard to perception, effective multi-user support, and managing data density and complexity. To address these issues and illustrate our proposed setup, we contribute an extensive design space comprising first, the spatial alignment of display, visualizations, and objects in AR space. Next, we discuss which parts of a visualization can be augmented. Finally, we analyze how AR can be used to display personal views in order to show additional information and to minimize the mutual disturbance of data analysts. Based on this conceptual foundation, we present a number of exemplary techniques for extending visualizations with AR and discuss their relation to our design space. We further describe how these techniques address typical visualization problems that we have identified during our literature research. To examine our concepts, we introduce a generic AR visualization framework as well as a prototype implementing several example techniques. In order to demonstrate their potential, we further present a use case walkthrough in which we analyze a movie data set. From these experiences, we conclude that the contributed techniques can be useful in exploring and understanding multivariate data. We are convinced that the extension of large displays with AR for information visualization has a great potential for data analysis and sense-making.

Transparency Setting: When exploring the AR model, the transparency (adjustable) of the model in front of and behind the cutting plane differs. Freezing then Annotating: The current cross-section can be frozen on the tablet, and users can move for a comfortable posture and then further work on it. Combine Augmented Reality (AR) with a spatially tracked tablet To demonstrate our concepts, we realized a proof-of-concept prototype consisting of a HoloLens 2, a Surface Pro 6 tablet and a OptiTrack motion tracking system.

We present a concept and early prototype for exploring volumetric medical data, e.g., from MRI or CT scans, in head-mounted Augmented Reality (AR) with a spatially tracked tablet. Our goal is to address the lack of immersion and intuitive input of conventional systems by providing spatial navigation to extract arbitrary slices from volumetric data directly in three-dimensional space. A 3D model of the medical data is displayed in the real environment, fixed to a particular location, using AR. The tablet is spatially moved through this virtual 3D model and shows the resulting slices as 2D images. We present several techniques that facilitate this overall concept, e.g., to place and explore the model, as well as to capture, annotate, and compare slices of the data. Furthermore, we implemented a proof-of-concept prototype that demonstrates the feasibility of our concepts. With our work we want to improve the current way of working with volumetric data slices in the medical domain and beyond.

We present Augmented Displays, a new class of display systems combining high-resolution interactive surfaces with head-coupled Augmented Reality. This extends the screen estate beyond the display and enables placing AR content directly at the display's borders or within the real environment. Furthermore, it enables people to interact with AR objects with natural pen and touch input in high precision on the surface. This combination allows a variety of interesting applications. To illustrate them, we present two use cases: An immersive 3D modeling tool and an architectural design tool. Our goal is to demonstrate the potential of Augmented Displays as a foundation for future work in the design space of this exciting new class of systems.

The recent trend of emerging high-quality Augmented Reality (AR) glasses offered the possibility for visually exciting application scenarios. However, the interaction with these devices is often challenging since current input methods most of the time lack haptic feedback and are limited in their user interface controls. With this work, we introduce , a combination of a belt-worn interaction device, utilizing a retractable cord, and a set of interaction techniques to enhance AR input capabilities with physical controls and spatial constraints. Building on our previous research, we created a fully-functional prototype to investigate how body-worn string devices can be used to support generic AR tasks. We contribute a radial widget menu for system control as well as transformation techniques for 3D object manipulation. To validate our interaction concepts for system control, we implemented a mid-air gesture interface as a baseline and evaluated our prototype in two formative user studies. Our results show that our approach provides flexibility regarding possible interaction mappings and was preferred for manipulation tasks compared to mid-air gesture input.

We present DesignAR, an augmented design workstation for creating 3D models. Our approach seamlessly integrates an interactive surface displaying 2D views with head-mounted, stereoscopic Augmented Reality (AR). This creates a combined output space that expands the screen estate and enables placing 3D objects beyond display borders. For the effective combination of 2D and 3D views, we define different levels of proximity and alignment. Regarding input, multi-touch and pen mitigate issues of precision and ergonomics commonly found in mid-air VR/AR interaction. For creating and refining 3D models, we propose a set of pen and touch techniques with immediate AR feedback, including sketching of rotational solids or tracing physical objects on the surface. To further support a designer's modeling process, we additionally propose orthographic model views and UI offloading in AR as well as freely placeable model instances with real-world reference. Based on our DesignAR prototype, we report on challenges and insights regarding this novel type of display augmentation. The combination of high-resolution, high-precision interactive surfaces with carefully aligned AR views opens up exciting possibilities for future work and design environments, a vision we call Augmented Displays.

Distributed systems are very complex and in case of errors hard to debug. The high number of messages with non deterministic delivery timings, as well as message losses, data corruption and node crashes cannot be efficiently analyzed with traditional GUI tools. We propose to use immersive technologies in a multi-display environment to tackle these shortcomings. Our DebugAR approach shows a representation of the current systems state, message provenance, and the lifetime of participating nodes and offers layouting techniques. By providing a screen that shows a traditional text-log, we bridge the gap to conventional tools. Additionally, we propose an interactive 3D visualization of the message flow, combining an interactive tabletop with augmented reality using a head-mounted display. We are confident that our proposed solution can not only be used to analyze distributed system, but also for other time-dependent networks.

Three-dimensional visualizations employing traditional input and output technologies have well-known limitations. Immersive technologies, natural interaction techniques, and recent developments in data physicalization may help to overcome these issues. In this context, we are specifically interested in the usage of spatial interaction with mobile devices for improved 3D visualizations. To contribute to a better understanding of this interaction style, we implemented example visualizations on a spatially-tracked tablet and investigated their usage and potential. In this paper, we report on a qualitative study comparing spatial interaction with inplace 3D visualizations to classic touch interaction regarding typical visualization tasks: navigation of unknown datasets, comparison of individual data objects, and the understanding and memorization of structures in the data. We identify several distinct usage patterns and derive recommendations for using spatial interaction in 3D data visualization.

3D data visualizations, while offering a lot of potential, have also well-known issues regarding occlusion and readability. Immersive technologies might help overcoming these issues by addressing the perceptional problems and increasing the tangibility of the data. In this work, we explore the potential of spatial interaction with mobile devices. Building on the related work and our own experiences, we report on visualizations that are fixed in space or fixed on the device, as well as combining them with head-coupled perspective. A number of prototypes we developed, helped us to gain practical insights in the possibilities and limitations of these techniques.

Interactive lenses have proven to be useful for many visualization applications where exploratory analysis is a primary task. Up to now, interaction with lenses is mostly limited to single-user, single-function lenses operated by mouse, keyboard and traditional parameter menus. To overcome these limitations, we propose MultiLens, touch-enabled magic lenses for fluently manipulating functions, parameters, and combinations of lenses on interactive surfaces. We contribute a novel multi-touch menu technique for magic lenses using a widget-based approach with a drag-snap slider for relative parameter adjustment. We also propose a continuous gesture set for rapidly changing lenses and their primary parameters in one seamless phrase. In addition, by supporting the combination of various lens functions, we create a generic multi-purpose lens tool. We illustrate our approach by investigating and implementing the concepts for the field of graph exploration. The prototype was evaluated in a user study with 22 participants comparing it to traditional parameter menus operated with both mouse and touch.

Research on interactive wall displays has thus far focused mostly on professional use. However, as large displays with support for touch and other input modalities become more common, it becomes reasonable to assume use in more casual settings as well. We present Miners, one of the first collaborative games for a touch-sensitive display wall, and investigate multimodal, multi-user interaction in this context. In this fast-paced game, four players cooperate to rescue workers trapped in an underground cave, with each player being able to influence the game world in a different way. In an exploratory study using Miners, we found that players enjoyed the game and showed very high engagement. On the other hand, awareness suffered: Players often missed events in other areas. In addition, we found limited awareness of other players' actions and social cues. We report on these results in detail and discuss implications for touch-based wall interaction in general as well as in other application contexts.

By now, mobile 3D interaction is often limited to simple multi-touch input on standard devices and less expressive or hard to use. We present the concept of mobile dual-display devices that can be folded for the exploration of 3D content. We examine different display modes and introduce new presentation and 3D interaction techniques that make use of the special form factor and the added input modality of folding two displays. In particular, we also consider the advantages of our proposed device for head-coupled perspective rendering -- virtually extending the view and providing independent perspectives for two users.

We present YouTouch!, a system that tracks users in front of an interactive display wall and associates touches with users. With their large size, display walls are inherently suitable for multi-user interaction. However, current touch recognition technology does not distinguish between users, making it hard to provide personalized user interfaces or access to private data. In our system we place a commodity RGB + depth camera in front of the wall, allowing us to track users and correlate them with touch events. While the camera's driver is able to track people, it loses the user's ID whenever she is occluded or leaves the scene. In these cases, we re-identify the person by means of a descriptor comprised of color histograms of body parts and skeleton-based biometric measurements. Additional processing reliably handles short-term occlusion as well as assignment of touches to occluded users. YouTouch! requires no user instrumentation nor custom hardware, and there is no registration nor learning phase. Our system was thoroughly tested with data sets comprising 81 people, demonstrating its ability to re-identify users and correlate them to touches even under adverse conditions.

Interactive lenses have proven to be useful for many visualization applications where exploratory analysis is a primary task. Up to now, interaction with lenses is mostly limited to single-user, single-function lenses operated by mouse, keyboard and traditional parameter menus. To overcome these limitations, we propose MultiLens, touch-enabled magic lenses for fluently manipulating functions, parameters, and combinations of lenses on interactive surfaces. We contribute a novel multi-touch menu technique for magic lenses using a widget-based approach with a drag-snap slider for relative parameter adjustment. We also propose a continuous gesture set for rapidly changing lenses and their primary parameters in one seamless phrase. In addition, by supporting the combination of various lens functions, we create a generic multi-purpose lens tool. We illustrate our approach by investigating and implementing the concepts for the field of graph exploration. The prototype was evaluated in a user study with 22 participants comparing it to traditional parameter menus operated with both mouse and touch.

Magic lenses are popular tools to provide locally altered views of visual data. In this paper, we introduce the concept of BodyLenses, special kinds of magic lenses for wall displays that are mainly controlled by body interactions. After motivating the rationale for body-centric lenses, we present a comprehensive design space of BodyLenses, where we analyse fundamental aspects such as appearance, function, interaction and use in multi-user contexts. Within that space, we investigated and implemented a number of design alternatives and propose solutions for lens positioning, dynamic shape modification, distance-based parameter mappings and the use of BodyLenses as portable tool belts. We demonstrate the practicality of our novel concepts with four realised application scenarios. With this work, we hope to lay the foundation for future research and systems based on body-driven lenses.

As Smart Manufacturing, Industrial Internet, Industrie 4.0, and Cyber-Physical Production System (CPPS) are becoming reality, the way process and manufacturing plants are operated has to change. Additionally, these developments - constituting the pervasive digitalisation of industry - have a profound effect on the way human workers and machines interact. This paper contrasts the current state of the art in plant control with a vision for the future. The scenario is illustrated by a realistic problem-solving process in a chemical plant. It describes an integrated industrial information and interaction space that leverages emerging technologies to enable plant operators to remain in control of future flexible modularised process plants. Our approach shows the advantages of an integrated information space which feeds interaction and collaboration using Virtual Reality (VR), novel display technologies and mobile devices.

We introduce touch-enabled magic lenses that can be manipulated and parametrized through fluent interactions. Interaction with lenses for information visualization and data exploration has mostly been limited to single-user, single-function lenses. In this work, we present our prototype on lenses where lens function, parameters and combination of functions can be manipulated using fluent touch interaction. To achieve this, our tool consists of a widget-based approach for novice users as well as a set of continuous gestures for expert users. Additionally, we support the combination of lenses and thereby create a multi-purpose lens tool.