In a rapidly evolving digital landscape autonomous tools and robots are becoming commonplace. Recognizing the significance of this development, this paper explores the integration of Large Language Models (LLMs) like Generative pre-trained transformer (GPT) into human-robot teaming environments to facilitate variable autonomy through the means of verbal human-robot communication. In this paper, we introduce a novel simulation framework for such a GPT-powered multi-robot testbed environment, based on a Unity Virtual Reality (VR) setting. This system allows users to interact with simulated robot agents through natural language, each powered by individual GPT cores. By means of OpenAI’s function calling, we bridge the gap between unstructured natural language input and structured robot actions. A user study with 12 participants explores the effectiveness of GPT-4 and, more importantly, user strategies when being given the opportunity to converse in natural language within a simulated multi-robot environment. Our findings suggest that users may have preconceived expectations on how to converse with robots and seldom try to explore the actual language and cognitive capabilities of their simulated robot collaborators. Still, those users who did explore were able to benefit from a much more natural flow of communication and human-like back-and-forth. We provide a set of lessons learned for future research and technical implementations of similar systems.

Mind-wandering, i.e., letting the mind drift away from the task at hand, is mostly seen as a state of mind to avoid, as it may negatively impact the current task. However, evidence in cognitive science shows that mind-wandering can also positively affect creativity and problem-solving. Still, there is a lack of technological solutions to facilitate and utilize mind-wandering in such a specific way. In this short paper, we present MuseFlow, a video game designed to facilitate mind-wandering deliberately. Our study shows that MuseFlow induces mind-wandering significantly more often compared to a demanding game condition while maintaining the players’ motivation to play and succeed in the game.

Assistive technologies such as human-robot collaboration, have the potential to ease the life of people with physical mobility impairments in social and economic activities. Currently, this group of people has lower rates of economic participation, due to the lack of adequate environments adapted to their capabilities. We take a closer look at the needs and preferences of people with physical mobility impairments in a human-robot cooperative environment at the workplace. Specifically, we aim to design how to control a robotic arm in manufacturing tasks for people with physical mobility impairments. We present a case study of a sheltered-workshop as a prototype for an institution that employs people with disabilities in manufacturing jobs. Here, we collected data of potential end-users with physical mobility impairments, social workers, and supervisors using a participatory design technique (Future-Workshop). These stakeholders were divided into two groups, primary (end-users) and secondary users (social workers, supervisors), which were run across two separate sessions. The gathered information was analyzed using thematic analysis to reveal underlying themes across stakeholders. We identified concepts that highlight underlying concerns related to the robot fitting in the social and organizational structure, human-robot synergy, and human-robot problem management. In this paper, we present our findings and discuss the implications of each theme when shaping an inclusive human-robot cooperative workstation for people with physical mobility impairments.

"Mensch ARgere Dich Nicht" is a Mixed Reality (MR) game based on a popular German board game "Mensch Ärgere Dich Nicht", from the early 1900s, similar to Pachisi. Developed to be a test bed for investigating mixed interactions, the application offers real and virtual variations of all core game elements (board, dice, pieces) for a fully modifiable experience. Using the Microsoft HoloLens, players can interact with the virtual game elements and receive additional information about the current game state. The game also recognizes interactions with real game elements and translates them into the virtual world.

The concept of Mixed Reality has existed in research for decades but has experienced rapid growth in recent years, mainly due to technological advances and peripherals such as the Microsoft HoloLens reaching the market. Despite this, certain design aspects of Mixed Reality experiences, such as the different nuances of real and virtual elements, remain largely unexplored. This paper presents an explorative study with 15 participants which aims to investigate and gain a better understanding of the different qualities of real and virtual objects. To that end, we developed a Mixed Reality board game that offered different combinations of real and virtual game components, such as the board, the pieces and the dice. Our analysis shows that the participants generally preferred the completely virtual variant but appreciated different qualities of real and virtual elements. The results also indicate that virtual interaction elements work better on a real background than vice versa. However, this conflicts with some participants' preference of using physical pieces for the haptic experience, creating a design trade-off. This study represents a first step in exploring how the experience changes when swapping elements of differing realities for one another and identifying these trade-offs.

The concept of Mixed Reality has existed in research for decades but has experienced rapid growth in recent years, mainly due to technological advances and peripherals such as the Microsoft HoloLens reaching the market. Despite this, certain design aspects of Mixed Reality experiences, such as the different nuances of real and virtual elements, remain largely unexplored. This paper presents an explorative study with 15 participants which aims to investigate and gain a better understanding of the different qualities of real and virtual objects. To that end, we developed a Mixed Reality board game that offered different combinations of real and virtual game components, such as the board, the pieces and the dice. Our analysis shows that the participants generally preferred the completely virtual variant but appreciated different qualities of real and virtual elements. The results also indicate that virtual interaction elements work better on a real background than vice versa. However, this conflicts with some participants' preference of using physical pieces for the haptic experience, creating a design trade-off. This study represents a first step in exploring how the experience changes when swapping elements of differing realities for one another and identifying these trade-offs.