Affect-adaptive systems (AAS) monitor the emotional user state, analyze it concerning the current context and apply appropriate adaptation strategies when necessary to support the user. In safety-critical work environments, human-machine systems aim at maintaining high performance and avoid failure. The detection of critical emotional user states requiring adaptation should take individual differences in the emotion-performance relationship into account. Hence, we suggested the Affective Response Categories (ARCs), a classification concept considering these differences. In the present contribution, we investigated if the proposed ARCs are part of a stable trait or states changing over time. Physiological measures, such as pupil diameter and facial expression; as well as performance data from two air space surveillance laboratory experiments differing in context and sample origin (N1 = 50; N2 = 17) were analyzed. Results of two one-sided tests (TOSTs) and intraclass correlation coefficients (ICC) based on data from both experiments provided evidence for stability of the categories over time, indicating that a continuous user state diagnosis based on a single classification of a user's ARC is possible. With these results, the present contribution provides the foundation for developing a diagnostic engine of users' emotional state.

Affective Tutoring Systems (ATS) detect and mitigate critical emotional learner states with the aim of providing individualized support. In tutoring systems for safety-critical work environments, students are trained to achieve and maintain high performance, therefore an ATS should be capable of identifying critical emotional states hindering performance. Interindividual differences in the emotion-performance-relationship can be considered by using the ARC categorization system. The present contribution aims at developing a questionnaire-based method of classifying new learners to the categories. To that end, we investigated differences in personality traits between the different categories. In an airspace surveillance task, we measured performance, emotional valence, emotional arousal, and personality traits in N = 50 subjects. Results showed that a positive valence-performance-relationship, compared to a negative valence-performance-relationship, is associated with higher Neuroticism, lower Conscientiousness, and lower Openness to experience. There were no significant differences in the traits Agreeableness and Extraversion. Based on these results, a future ATS for safety-critical work environments could classify new learners in the ARCs using self-report data and thus dispense with physiological sensors. Thereby, user state diagnosis and evaluation for high performance is possible, setting the ground for an ATS adapting to critical emotional learner states.

Affect-adaptive systems detect the emotional user state, assess it against the current situation, and adjust interaction accordingly. Tools for real-time emotional state detection, like the Emotient FACET engine (Littlewort et al., 2011), are based on the analysis of facial expressions. When developing affect-adaptive systems, output from the diagnostic engine must be mapped onto theoretical models of emotion. The Circumplex Model of Affect (Russell, 1980) describes emotion on two dimensions: valence and arousal. However, FACET offers three classifiers for valence: positive, neutral, and negative valence. The present study aimed at developing an algorithm that converts these into a unified valence scale. We used FACET to analyze valence-labeled images from the AffectNet database. In a multiple regression analysis, FACET classifier values predicted database valence and explained 38% of the variance. By inserting classifier values into the regression equation, a unified valence scale can be calculated that matches dimensional models of emotion. This research forms the groundwork for adaptation of the emotional user state based on the FACET engine. A future affect-adaptive system can now use the FACET engine to detect the emotional user state on a unified valence dimension, which allows for distinct classification and interpretation of emotions.

Affect-adaptive tutoring systems detect the current emotional state of the learner and are capable of adequately responding by adapting the learning experience. Adaptations could be employed to manipulate the emotional state in a direction favorable to the learning process; for example, contextual help can be offered to mitigate frustration, or lesson plans can be accelerated to avoid boredom. Safety-critical situations, in which wrong decisions and behaviors can have fatal consequences, may particularly benefit from affect-adaptive tutoring systems, because accounting for affecting responses during training may help develop coping strategies and improve resilience. Effective adaptation, however, can only be accomplished when knowing which emotions benefit high learning performance in such systems. The results of preliminary studies indicate interindividual differences in the relationship between emotion and performance that require consideration by an affect-adaptive system. To that end, this article introduces the concept of Affective Response Categories (ARCs) that can be used to categorize learners based on their emotion-performance relationship. In an experimental study, N = 50 subjects (33% female, 19–57 years, M = 32.75, SD = 9.8) performed a simulated airspace surveillance task. Emotional valence was detected using facial expression analysis, and pupil diameters were used to indicate emotional arousal. A cluster analysis was performed to group subjects into ARCs based on their individual correlations of valence and performance as well as arousal and performance. Three different clusters were identified, one of which showed no correlations between emotion and performance. The performance of subjects in the other two clusters benefitted from negative arousal and differed only in the valence-performance correlation, which was positive or negative. Based on the identified clusters, the initial ARC model was revised. We then discuss the resulting model, outline future research, and derive implications for the larger context of the field of adaptive tutoring systems. Furthermore, potential benefits of the proposed concept are discussed and ethical issues are identified and addressed.