This paper introduces a novel approach to tackle the challenges of preserving and transferring tacit knowledge–deep, experience-based insights that are hard to articulate but vital for decision-making, innovation, and problem-solving. Traditional methods rely heavily on human facilitators, which, while effective, are resource-intensive and lack scalability. A promising alternative is the use of Socially Interactive Agents (SIAs) as AI-driven knowledge transfer facilitators. These agents interact autonomously and socially intelligently with users through multimodal behaviors (verbal, paraverbal, nonverbal), simulating expert roles in various organizational contexts. SIAs engage employees in empathic, natural-language dialogues, helping them externalize insights that might otherwise remain unspoken. Their success hinges on building trust, as employees are often hesitant to share tacit knowledge without assurance of confidentiality and appreciation. Key technologies include Large Language Models (LLMs) for generating context-relevant dialogue, Retrieval-Augmented Generation (RAG) to integrate organizational knowledge, and Chain-of-Thought (CoT) prompting to guide structured reflection. These enable SIAs to actively elicit knowledge, uncover implicit assumptions, and connect insights to broader organizational contexts. Potential applications span onboarding, where SIAs support personalized guidance and introductions, and knowledge retention, where they conduct structured interviews with retiring experts to capture heuristics behind decisions. Success depends on addressing ethical and operational challenges such as data privacy, algorithmic bias, and resistance to AI. Transparency, robust validation, and a culture of trust are essential to mitigate these risks.

Oliver Gustke, Stefan Schaffer, Aaron Ruß, CHIM-Chatbot in the Museum: Exploring and Explaining Museum Objects with Speech- Based AI CHIM-Chatbot in the Museum was a research project during which we developed a chatbot prototype that is able to provide answers to users' questions about museum objects. CHIM was developed by Linon Medien KG and the Deutsches Forschungs- zentrum für Künstliche Intelligenz (DFKI). The interactive conversation system was implemented as an Android- based demonstration app and tested at the Städel Mu- seum, Frankfurt am Main, in the spring of 2022. The results showed that upcoming chatbot- based systems might motivate and encourage museum visitors and contri- bute to participation.

In this paper, we summarize the initial results of a field test with the ChiM (Chatbot in the Museum) system, a conversational user interface for the museum. The system contains a Natural Language Understanding (NLU) component that translates the user input into intentions and produces a multimodal (mainly spoken and textual) output. Museum visitors can use the system to freely ask questions about the exhibits in the exhibition. We conducted a field test with 140 participants in the Städel Museum, Frankfurt, and recorded over 4600 interactions between the participants and the system. After the test, participants gave their perceived feedback on the user experience (UX) and completed a custom system-specific questionnaire. We exploratively analyzed the feedback. The results show an overall medium UX for the system. We assume that the NLU component must be improved. Participants who rarely or never use audio guides rate the pragmatic quality (PQ) of the system significantly better than people who often or always use audio guides. People who rated the speech quality of the system as good also rated the attractiveness of the system significantly better than people who rated the speech quality as bad. In our future work, we will deepen the UX analysis and further put focus on recorded interaction data.

Social group cycling shows a positive impact on facilitating urban cycling as a sustainable means of mobility while increasing cycling safety in urban areas [1]. We present a new urban mobility concept, Connected Swarm Cycling, that creates a group of people cycling together for a while in a common direction or destination. We assume that the concept of Swarm Cycling can significantly change the mobility behaviour of citizens and will be a building block of green mobility for sustainable cities in the future. Utilizing an OSRM 1 1 http://project-osrm.org/ routing service with support of trip intersection computing, the system inducts the cyclists into a cycling swarm. The swarms are formed automatically via peer-to-peer connection when cyclists come in proximity, and the information of the swarm and individual cyclist will be synchronized within the swarm via a Nearby Mesh Network. Supporting the implicit interaction within or between swarms, smart wearables are utilized to realize use cases like swarm member identification or signalling in case of merging or splitting of swarms. In this paper, we also present a technical description of our system, including the protocol and network model to support the coordination and synchronization within the swarms.

In this paper, we describe our work within the research project “CHIM - Chatbot in the Museum”. CHIM is an AI-based chatbot prototype that enables conversational interaction using text and speech input: visitors can ask questions about certain artworks and receive answers in multimodal formats (text, audio, image, video). The application will be tested in the Städel Museum, Frankfurt/Main, Germany. To develop a proper Natural Language Understanding module, we adapted an existing categorization approach, gathered visitor questions, and structured them into twelve distinct content types. The preliminary results suggest that our approach to subdivide the previously overloaded content type meaning into further categories was successful, leading to a more balanced distribution of the data. We further describe the Natural Language Processing mechanisms employed here; these follow a multi-tiered approach using techniques like Rasa, BERT, and cosine-similarity to generate answers with different degrees of effort. Future steps are the implementation of dialog management, the refinement of the NLP strategies by integrating additional answers for selected exhibits, and the implementation of the final layout and interaction design. We are planning to test and evaluate the CHIM prototype on site in the Städel Museum in late 2021.

This work is focusing on the automatic generation of subtitles using different tools that can be categorized as Forced Aligners (FAs) or Automatic Speech Recognizers (ASRs). A comparison of the performance of FA and ASR for the task of generating same-language subtitles was conducted. The prime motivation was a previous task, which was the extraction of sentence-utterances in different audio files using word-timestamps. Three different tools were used for this work: aeneas [1] which is an FA, Cerence [2], which is an ASR and Sonix [3], which is also an ASR. We conducted a technical evaluation and a subjective evaluation based on a case study. In this study people were presented with different stimuli, each stimulus using generated subtitles based on the time-information given by the different tools mentioned above. The resulting data of a case study confirmed a rise in performance of Cerence compared to aeneas.

. Most question answering tasks are oriented towards open domain factoid questions. In comparison, much less work has studied both factoid and open ended questions in closed domains. We have chosen a current state-of-art BERT model for our question answering exper-iment, and investigate the effectiveness of the BERT model for both factoid and open-ended questions in the museum domain, in a realistic setting. We conducted a web based experiment where we collected 285 questions relating to museum pictures. We manually determined the answers from the description texts of the pictures and classified them into answerable/un-answerable and factoid/open-ended. We passed the questions through a BERT model and evaluated their performance with our created dataset. Matching our expectations, BERT performed better for factoid questions, while it was only able to answer 36% of the open-ended questions. Further analysis showed that questions that can be answered from a single sentence or two are easier for the BERT model. We have also found that the individual picture and description text have some implications for the performance of the BERT model. Finally, we pro-pose how to overcome the current limitations of out of the box question answering solutions in realistic settings and point out important factors for designing the context for getting a better question answering model using BERT.

In this paper, we introduce RARSS, a reporting assistant for railway security staff. RARSS is a demonstration application with a multi-modal interface based on the Mobile Multimodal Interaction and Rendering (MMIR) framework. The system should support the security staff at railway premises (stations, trains, etc.) in Germany and inform about security relevant information about the travel of football fans or a group of people on their way to a major event. In the application we leverage multi keyword spotting (KWS) for detecting of the actual context and a grammar with specific voice commands to improve the semantic interpretation. The results of friendly user testing showed that the multimodal conversational interface was positively rated according the simplicity and the efficiency to make security reports by the security staff.

In this paper we highlight several difficulties and insights that we discovered during our practical work in the area of conversational user interfaces. We review four fields in which we conducted research and industries projects. We dig into issues like developing for various domains, disclose insights into "fun bot" development, and derive ideas for design guidelines for developing for domains with legal restrictions and for efficient interaction.

Easy2Go [1] is a seamless multimodal mobility service through a pay-as-you-go solution supported by EIT Digital. In this paper, we present Easy2Go’s intelligent trip recommendation system, an optimised commuter data-based routing service that can be used to learn the actual usage of the traffic system and to automatically create trip recommendations including or avoiding information about frequently used routes. Utilizing transport mode detection technology, we are able to collect commuter route data, from which our system leverages crowd knowledge by machine learning. Our system combines machine learning with a classical routing service to provide an improved trip recommendation. The underlying functional concepts of the overall system is presented as well as details about the used machine learning algorithm.

In this paper we try to provoke by teasing the question "if conversational user interfaces should be multimodal?". Of course they should! In decades of research in multimodal HCI excellent arguments can be found. We substantiate our perspective with an example showing how conversational interaction becomes more robust and efficient through the use of multimodality.

Human agents in technical customer support provide users with instructional answers to solve a task that would otherwise require a lot of time, money, energy, physical costs. Developing a dialogue system in this domain is challenging due to the broad variety of user questions. Moreover, user questions are noisy (for example, spelling mistakes), redundant and have various natural language expressions. In this work, we introduce a conversational system, MOLI (the name of our dialogue system), to solve customer questions by providing instructional answers from a knowledge base. Our approach combines models for question type and intent category classification with slot filling and a back-end knowledge base for filtering and ranking answers, and uses a dialog framework to actively query the user for missing information. For answer-ranking we find that sequential matching networks and neural multi-perspective sentence similarity networks clearly outperform baseline models, achieving a 43% error reduction. The end-to-end P@1(Precision at top 1) of MOLI was 0.69 and the customers’ satisfaction was 0.73.

Research Interests The main interest of my research is the development of model-based methods for simulation and automated usability prediction of multimodal interfaces. In particular I want to investigate how modality choice of users can be predicted and simulated by a computational model that estimates the quality of multimodal interfaces. Therefore I am also interested in exploring rules and cognitive processes that impact users’ modality selection in multimodal humancomputer interaction.

In this paper we examine influencing factors and changes of perceived longitudinal user experience. An implemented mobility app for Berlin was tested in a three month field trial. The app offers unique features for bicyclists and integrates a gamification concept. The study consisted of three game phases. Each phase was completed by a demographic and a user experience questionnaire. Possible differences resulting from differing gamifi-cation incentives between acquired participants and voluntary participants could not be found. Further, general changes in subjective long-term user experience could not be detected. The results reveal several significant effects of age, gender, and spe-cific user groups on pragmatic quality, attractiveness, and hedonic qualities.

In this paper, we review three experiments with a mobile application that integrates graphical input with a touch-screen and a speech interface and develop a model for input modality choice in multimodal interaction. The model aims to enable simulation of multimodal human-computer interaction for automatic usability evaluation. The experimental results indicate that modality efficiency and input performance are important moderators of modality choice. Accordingly, we establish a utility-driven model that provides probability estimations of modality usage, based on the parameters of modality efficiency and input performance. Four variants of the model that differ in training data are fitted by means of Sequential Least Squares Programming. The analysis reveals a considerable fit regarding averaged modality usage. When applied to individual modality usage profiles, the accuracy decreases significantly. In an application example it is shown how the modality choice mechanism can be deployed for simulating interaction in the field of automatic usability evaluation. Results and possible limitations are discussed. Decreased input performance of a specific modality, decreases the usage of this modality.Decreased efficiency of the GUI, increases VUI usage.A model predicting modality choice is proposed.The model can be deployed for automatic usability evaluation.System designers are provided with working knowledge about expected modality usage.

An increasing amount of available data sources and intermodal travelling pose a challenge for the design of mobility apps. This paper examines the concept of an interface for intermodal personalized travel assistance and routing. The analysis of two central use cases intermodal routing and personalization revealed pitfalls and gaps for the design of mobility apps.

This paper describes an experimental study involving a smartphone-based application using speech and touch screen as input modalities. Following our previous work on the effects of modality efficiency and input performance of voice user interfaces (VUI) and graphical user interfaces (GUI) on modality usage and perceived mental effort, we particularly investigate the so far not considered influence of touch screen errors. Participants had to perform several tasks with the mobile device in four error conditions of the two input modalities: (1) no GUI errors & no VUI errors (2) GUI errors & no VUI errors (3) no GUI errors & VUI errors (4) GUI errors & VUI errors. The results show significantly increasing speech usage in the presence of GUI errors. Regarding mental effort and perceived pragmatic quality, no differences emerge as long as one modality functions properly. However, if errors affect both, VUI and GUI, mental effort increases significantly and ratings on perceived pragmatic quality decrease.