Affective Interactions with AI Controlled Conversational Agents in Virtual Reality

The exchange of affective information lies at the core of social interactions and allows inference of the intentions and states of an interactive partner. In experimental settings, this exchange is often limited to nonverbal expressions, resulting in a reduced form of complex social behavior. The implementation of embodied conversational agents in Virtual Reality promises highly naturalistic and interactive verbal communication with AI-controlled agents. However, the degree to which affective information can be generated in such paradigms remains unclear. The goal of the present study is to evaluate a behavioral paradigm in which an embodied conversational agent provides affective information in a conversation and to test whether target emotions can be induced and maintained in real-time interactions. Data from 48 human-AI interactions demonstrate that AI-controlled agents were able to generate specific affective speech content for different conversational contexts (happy, angry, and sad events). Analysis within conversations shows that context-specific emotions were most frequent at the beginning of a conversation and then decreased with turns. Overall, the results suggest that AI-controlled embodied conversational agents are a promising way to include affective information in the simulation of naturalistic conversations.