In mixed reality (MR) telepresence applications, the differences between participants' physical environments can interfere with effective collaboration. For asymmetric tasks, users might need to access different resources (information, objects, tools) distributed throughout their room. Existing intersection methods do not support such interactions, because a large portion of the telepresence participants' rooms become inaccessible, along with the relevant task resources. We propose MRUnion, a Mixed Reality Telepresence pipeline for asymmetric task-aware 3D mutual scene generation. The key concept of our approach is to enable a user in an asymmetric telecollaboration scenario to access the entire room, while still being able to communicate with remote users in a shared space. For this purpose, we introduce a novel mutual room layout called Union. We evaluated 882 space combinations quantitatively involving two, three, and four combined remote spaces and compared it to a conventional Intersect room layout. The results show that our method outperforms existing intersection methods and enables a significant increase in space and accessibility to resources within the shared space. In an exploratory user study (N=24), we investigated the applicability of the synthetic mutual scene in both MR and VR setups, where users collaborated on an asymmetric remote assembly task. The study results showed that our method achieved comparable results to the intersect method but requires further investigation in terms of social presence, safety and support of collaboration. From this study, we derived design implications for synthetic mutual spaces.

The vision of a ‘metaverse’ may soon bring a ubiquitous(ly) Augmented Reality (UAR) delivering context-aware, geo-located, and continuous blends of real and virtual elements into reach. This paper draws on speculative design to explore, question, and problematize consequences of AR becoming pervasive. Elaborating on Desjardin et al.’s bespoke booklets, we co-speculate together with 12 globally dispersed participants. Each participant received a custom-made design workbook containing pictures of their immediate surroundings, which they elaborated on in situated brainstorming activities. We present an integration of their speculative ideas and lived experiences in 3 overarching themes from which 7 ‘dark’ scenarios caused by UAR were formed. The Scenarios are indicative of deceptive design patterns that can (and likely will be) devised to misuse UAR, and anti-patterns that could cause unintended consequences. These contributions enable the timely discussion of potential antidotes and to which extent they can mitigate imminent harms of UAR.

Having a computer do the work for you has become more and more common over time. But in the entertainment area, where a human is a creator, we want to avoid having too much influence on technology. On the other hand, inspiration is still important; we developed a virtual conductor that can generate an emotionally associated interpretation of known music work. This was done by surveying a set number of people to determine, which emotions were associated with a specific interpretation and instruments. As a result of machine learning this conductor was then able to achieve his goal. Unlike earlier studies of virtual conductors, which would replace the role of a human conductor, this new one is supposed to be an assisting tool for conductors. As a result, starting on a new interpretation will be easier because it streamlines research time and provides a technical perspective that can inspire new ideas. By using this technology as a supplement to human creativity, we can create richer, more nuanced interpretations of musical works.

Detecting interpersonal synchrony in the wild through ubiquitous wearable sensing invites promising new social insights as well as the possibility of new interactions between humans-humans and humans-agents. We present the Offset-Adjusted SImilarity Score (OASIS), a real-time method of detecting similarity which we show working on visual detection of Duchenne smile between a pair of users. We conduct a user study survey (N = 27) to measure a user-based interoperability score on smile similarity and compare the user score with OASIS as well as the rolling window Pearson correlation and the Dynamic Time Warping (DTW) method. Ultimately, our results indicate that our algorithm has intrinsic qualities comparable to the user score and measures well to the statistical correlation methods. It takes the temporal offset between the input signals into account with the added benefit of being an algorithm which can be adapted to run in real-time will less computational intensity than traditional time series correlation methods.

We present our interdisciplinary research on gamification as a communication tool in urban planning. Approaching the topic from a user-centered perspective, we combine knowledge from the domains of architecture and human-computer interaction and define domain-specific requirements and areas of interest. To substantiate our findings, we present a prototypical case study of a public participation application, developed in consultation with the urban planning project driver and evaluated on-location with members of the public. Our results highlight the opportunities in pursuing a new sub-field of gamification within urban planning and include concepts of game interfaces and spaces in this research. In our case study, we present a methodological approach to assessing gamification in urban planning participation and provide findings on how gamification changes the users’ relationship to participation as well as evidence that the “avatar” game element can affect the motivational affordances meaning of task, perceived competence, and social relatedness.

With many contemporary video conferencing platforms available, there is still a need for platforms that afford a researcher workflow to conduct controlled online experiments. We have developed an open source experimental video conferencing platform that enables researchers to design and conduct remote experiments. Our platform provides a high level of control over the user interface and video streams, which is essential for studying the differences between remote and in-person social interactions. We give an overview of our platform’s usage and architecture and conduct a take-home study (N=9) to evaluate how accessible our system is to potential new contributors. We also follow up with an initial evaluation of technical performance bottlenecks for when our experimental platform is deployed, and show that the computational resources increases per each video stream as well as the type of filters applied to each participant. We end with a short discussion on next steps and the experimental hub’s potential to be extended as a sandbox for testing browser based augmented reality (WebAR) filters to be adopted in interdisciplinary experimental procedures.

Two-factor authentication (2FA) is a recommended or imposed authentication mechanism for valuable online assets. However, 2FA mechanisms usually exhibit user experience issues that create user friction and even lead to poor acceptance, hampering the wider spread of 2FA. In this article, we investigate user perceptions of 2FA through in-depth interviews with 42 participants, revealing key requirements that are not well met today despite recently emerged 2FA solutions. First, we investigate past experiences with authentication mechanisms emphasizing problems and aspects that hamper good user experience. Second, we investigate the different authentication factors more closely. Our results reveal particularly interesting preferences regarding the authentication factor “ownership” in terms of properties, physical realizations, and interaction. These findings suggest a path toward 2FA mechanisms with considerably better user experience, promising to improve the acceptance and hence, the proliferation of 2FA for the benefit of security in the digital world.

Augmented Reality (AR) is increasingly considered for use in real industrial applications [8]. In our industrial research lab at Siemens Technology, we are continuously discussing suitable AR scenarios in business sectors involving power plants, wind turbine plants, and oil and gas factories. We have developed a company-internal AR system architecture, Hololayer, which provides AR facilities in a reusable manner to development engineers such that they can build their own AR applications for their specialized use cases. The integration of AR technology into industrial processes encounters many complex issues: we need to adhere to many safety, security and quality guarantees while also adapting quickly to the rapidly changing state of the art of AR devices (HMDs, tablets). To increase flexibility, it might be good to integrate Hololayer with one of the open frameworks that have recently been proposed [11], [29], [38]. Yet, care must be taken when converting an already existing, large company-owned framework like Hololayer to such platforms. Some of the proposed standardized APIs and communication protocols may be difficult to abide by, without requiring significant rewriting efforts or even violating company-internal regulations. In this paper, we report on our efforts to combine Hololayer with one such platform, Ubi-Interact [38]. This is a collaboration between Siemens technology and the FAR group at TU Munich. After exemplary descriptions of typical application scenarios, we present the underlying principles of Hololayer to support this range of applications. We then describe rudimentary concepts of Ubi-Interact, followed by an elaboration on the efforts to combine both systems for a selected application scenario and a discussion of the results achieved thus far.

Consequences that deter adoption, such as asymmetrical encounters between wearers and bystanders, need to be explored in order to make Ubiquitous Augmented Reality (UAR) acceptable. In our work we outline how social perception is based on a Head Mounted Displays (HMD) capability, appearance, and the role of the wearer. We fixed the device capability to zooming in AR and explored the privacy implications in 12 interviews through a prototype with the mocked ability to ”super humanly” zoom in on targets. Next, we used the resulting themes to survey 100 participants to deeper explore augmented zoom while we permutate on the device appearance housed in three form-factors: contact lenses, glasses, and helmet and role of wearer based on level of involvement in an abstracted scenario transpiring in a public space. Our results showed that explicit visibility of an AR system provides social translucence as it is rated least likely to cause misuse but also perceived as least likely to have an advantage.

Augmented Reality (AR) is being adopted in industry for communication and in society as a whole. While development continues, challenges still remain. Calibration tasks especially remain a hassle for developers and end users alike. They are time-consuming, follow strict rule-sets and require great care to ensure a usable end result. In this report, the extended version of Langbein et al. [16], we propose a solution to ease calibration tasks on users and keep them motivated by adding game concepts to the procedure. We show that, in a gamified application, participants can be incited to perform longer procedures with up to four times the amount of measurements taken without a necessary decrease in the resulting calibration quality.

Retention is an important success factor for mobile social games. However, in game design, very little literature exists on the impact of game elements and their persuasive effects on user retention. We scraped ten high grossing mobile games and play tested them over the course of two weeks to gather persuasive game mechanics. After categorization and filtering of the persuasive mechanics, we simultaneously relate them to base game mechanics and a corresponding psychological theory from behavioral economics and psychology. Our results help increase the accessibility of using persuasive mechanics for future game developers.

Embedding conductive material into 3D printed objects enables non-interactive objects to become tangible without the need to attach additional components. We present a novel use for such touch-sensitive objects in an augmented reality (AR) setting and explore the use of gestures for enabling different types of interaction with digital and physical content. In our demonstration, the setting is an urban planning scenario. The multi-material 3D printed buildings consist of thin layers of white plastic filament and a conductive wireframe to enable touch gestures. Attendees can either interact with the physical model or with the mobile AR interface for selecting, adding or deleting buildings.

Physical models are a key component in the architectural process and play an important role in understanding material and space relationships. We present Tangible Urban Models, an approach for leveraging the use of conductive material for 3D printed architectural prototypes. This enables non-interactive objects, such as buildings, to become tangible without the need to attach additional components. We combine this capability with an augmented reality (AR) app and explore the use of gestures for interacting with digital and physical content. The multi-material 3D printed buildings consist of thin layers of white plastic filament and a conductive wireframe to enable touch gestures. In this way, we enable a two-way interaction either with the physical model or with the mobile AR interface.

In this paper, we propose a solution to ease registrational tasks on users and keep them motivated during long lasting procedures using gamification elements. Augmented Reality (AR) technology appears in many different hardware setups ranging from mobile applications to full scale room tracking constructions. Many of which contain non-rigid sensors that require re-registrational maintenance to preserve satisfactory tracking capabilities. While important for a functioning setup, these tasks can become tiring over time leading to less care spent on a qualitative registration. We report on our preliminary study results, showing that a gami-fied registration routine incites participants to perform these tasks for a longer amount of time with up to four times as many measurements taken as the non-gamified version.

Can natural interaction requirements be fulfilled while still harnessing the "supernatural" fantasy of Virtual Reality (VR)? In this work we used off the shelf Electromyogram (EMG) sensors as an input device which can afford natural gestures to preform the "supernatural" task of growing your arm in VR. We recorded 18 participants preforming a simple retrieval task in two phases; an initial and a learning phase where the stretch arm was disabled and enabled respectively. The results show that the gestures used in the initial phase are different than the main gestures used to retrieve an object in our system and that the times taken to complete the learning phase are highly variable across participants.

Brain Computer Interfaces (BCIs) are a promising interaction method that currently face many problems before they can be used in everyday applications, such as games. One of the main problems is that calibration required for each new session using the BCI can be as long as 30 min. In this work in progress we hold 6 workshops with 2-4 participants per workshop, 18 participants total, to brainstorm the wide range of BCI applications and how to gamify the calibration process for each application. We propose an overview of the workshop results and an analysis and discussion around criteria for calibration games based on a BCI application.

Academia and industry engage in major efforts to develop technologies for augmenting human senses and activities. Many of these technologies, such as augmented reality (AR) and virtual reality (VR) head mounted displays (HMD), haptic augmentation systems, and exoskeletons can be applied in numerous usage contexts and scenarios. We argue that these technologies may strongly affect the perceptions of and interactions with other people in the social contexts where they are used. The altered interactions may lead to rejection of the augmentations. In this position paper we present a set of potential usage scenarios and an initial model of acceptance of augmentation technologies by users and other people involved in the social context.

Heat maps are a commonplace tool to communicate geolocated information. There are a lot of features in a heat map's visualization that are able to be perceived differently. Parameters such as color and threshold can influence how the data is understood. In the specific context of visualizing restaurant density, we conducted a mixed methods study to measure how these two parameters can affect a user's perception of data quality. We present preliminary results of interviews, and an online survey.

Brain Computer Interfaces (BCIs) are a promising interaction method that currently face many problems before they can be used in everyday applications, such as games. One of the main problems is that calibration required for each new session using the BCI can be as long as 30 min. In this work in progress we hold 6 workshops with 2-4 participants per workshop, 18 participants total, to brainstorm the wide range of BCI applications and how to gamify the calibration process for each application. We propose an overview of the workshop results and an analysis and discussion around criteria for calibration games based on a BCI application.