Emotion regulation (ER) is essential to mental well-being but often difficult to access, especially in high-intensity moments or for individuals with clinical vulnerabilities. While existing technology-based ER tools offer value, they typically rely on self-reflection (e.g., emotion tracking, journaling) or co-regulation through verbal modalities (reminders, text-based conversational tools), which may not be accessible or effective when most needed. The biological role of the touch modality makes it an intriguing alternate pathway, but empirical evidence is limited and under-theorized. Building on our prior theoretical framework describing how a comforting haptic co-regulating adjunct (CHORA) can support ER, we developed a zoomorphic robot CHORA with looped biomimetic breathing and heartbeat behaviors. We evaluated its effects in a mixed-methods in-lab study (N=30), providing physiological, self-report, custom questionnaire, and retrospective interview data. Our findings demonstrate the regulatory effects of haptically experienced animacy, corroborate prior work, and validate CHORA's theoretically grounded potential to facilitate four ER strategies.

Emotion regulation (ER) is essential to mental well-being but often difficult to access, especially in high-intensity moments or for individuals with clinical vulnerabilities. While existing technology-based ER tools offer value, they typically rely on self-reflection (e.g., emotion tracking, journaling) or co-regulation through verbal modalities (reminders, text-based conversational tools), which may not be accessible or effective when most needed. The biological role of the touch modality makes it an intriguing alternate pathway, but empirical evidence is limited and under-theorized. Building on our prior theoretical framework describing how a comforting haptic co-regulating adjunct (CHORA) can support ER, we developed a zoomorphic robot CHORA with looped biomimetic breathing and heartbeat behaviors. We evaluated its effects in a mixed-methods in-lab study (N=30), providing physiological, self-report, custom questionnaire, and retrospective interview data. Our findings demonstrate the regulatory effects of haptically experienced animacy, corroborate prior work, and validate CHORA's theoretically grounded potential to facilitate four ER strategies.

Abstract: We propose that social robots able to interact in a “bidirectionally physical” manner, by continuously modulating their own movement and impedance in a direct coupling with the user's rather than simply triggering pre-set physical responses, have high potential for social expressiveness and durable engagement. This study explores how the dynamics of robot bunting, a feline behavior involving playful, affective force exchanges, can influence user perception of robot Personality Attributes ( e.g., extroversion and openness) and Interaction Qualities ( e.g., intentionality, liveliness, aliveness). With a minimalist one-degree-of-freedom (1-DF) haptic display, we created a control architecture to portray an abstract zoomorphic creature and conducted a user study (N=14) to evaluate the effects of Robot Role ( Leader or Follower ) and three Physical Dynamics variables (rendered stiffness, motion frequency and time delay). Participants stroked, scratched and petted the robot under each robot condition and reported on their social perceptions of it. Varying Robot Role significantly impacted attributes in both groups, while Physical Dynamics influence was concentrated on Interaction Qualities. Physical Dynamics factor variables behaved differently in each Role; e.g., when the robot was a Leader , high stiffness enhanced perceptions of intentionality and high motion frequency conveyed liveliness, while as a Follower , increased stiffness and delay raised user impressions of aliveness. This research demonstrates how physical qualities can shape human social perceptions of even a simple, abstract robot and that their subtle adjustment can create a wide range of social manifestations, with implications for generating sustained human-robot engagement.

Naturalistic human touch expression can be an emotionally potent modality, and promising as an informative but unobtrusive input into machine-learning models of affect. However, application-ready emotion-aware technologies must be trained on labeled samples of authentic (felt) emotion of a significant range of intensity and valence. These may come at significant, even traumatic, personal cost for any modality. We examined (a) performance of the novel touch modality, and (b) how a care-centered protocol might manage personal burden. Participants (N=5; 3 team members), shared autobiographical stories that elicited powerful emotional dynamics, in 1-3 sessions each (total 10). During storytelling, incidental touch was captured on a pillow-mounted custom flexible 10 & times; 10-taxel pressure sensor, alongside physiological signals; then labeled with multiple passes of rich, multimodal self-reports. Protocol, study and analysis design prioritized reflexivity and participant experience. Accuracy: Participant-specific, touch-only models predicted emotion direction (trajectory slope) with 65.2 +/- 16.3% accuracy (2s windows; chance 25%, physiology-only models 64.1 +/- 16.9%), confirming the value of this unobtrusive channel. Personal cost: Qualitative analysis contributed an extensive picture of the emotional toll of generating such data, but also some benefits. We offer recommendations for sustainable ethical sourcing of affective data which balance personalization, performance, therapeutic insight and participant care.

Many people use physical objects to self-calm and regulate difficult emotions. Studies evaluating the effectiveness of Touchable Comfort Objects (TCOs) imbued with haptic expressivity, affect awareness, and adaptive support have been primarily lab-based, so we know little about underlying user needs and real-world practices. We asked 132 participants about their favorite TCO: how they use it, where it helps (or fails), and what they would value or avoid in a technology-enhanced TCO. Our survey analysis, based on a proposed data-validated dimensional space, reveals that people choose TCOs as much for emotional significance as for physical soothing, engage with them multiple times daily, and about half desire deeper support for emotion awareness, social-emotional agency, and communication facilitation. We identified four profiles (Physical Soothers, Frodo Fidgeters, Sentimental Functionalists, Nostalgic Comfort-Seekers) that capture diverse patterns. We further contribute a dataset with an interactive visualization, and describe TCO design opportunities to support researchers and designers.

Computational emotion recognition relies on observable expressions. However, negative situations can evoke regulation mechanisms that obscure and mask emotional experiences, often by smiling. As smiles are typically associated with positive emotions, this mismatch of emotional experience and expression may lead to misinterpretations by most current algorithmic affective computing approaches. To improve computational modeling of real-life experiences and expressions in negative social situations, we explore connections between smile appearance and function, incorporating participants' rich personal self-reports into ground truth labels for their expressions. We present an empirically grounded smile corpus of 199 smiles that is based on a) recordings of N = 30 participants in negative social situations that are analyzed regarding smile morphology and b) a category system of smile functions based on participants‘ self-reports. In a computational model, we used cleaned corpus data of 183 unique smile instances to classify five smile function categories based on observable nonverbal signals, with results benchmarked at above chance. Applying a theory- and data-driven approach, our analyses confirm a complex relationship between internal smile functions and observable signals. Finally, we discuss smile functions in negative social situations, including ‘despising’, ‘provoking’, and 'kiss my ass'-smiles.

Humans physically express emotion by modulating parameters that register on mammalian skin mechanoreceptors, but are unavailable in current touch-sensing technology. Greater sensory richness combined with data on affect-expression composition is a prerequisite to estimating affect from touch, with applications including physical human-robot interaction. To examine shear alongside more easily captured normal stresses, we tailored recent capacitive technology to attain performance suitable for affective touch, creating a flexible, reconfigurable and soft 36-taxel array that detects multitouch normal and 2-dimensional shear at ranges of 1.5kPa-43kPa and ± 0.3-3.8kPa respectively, wirelessly at 43Hz (1548 taxels/s). In a deep-learning classification of 9 gestures (N=16), inclusion of shear data improved accuracy to 88%, compared to 80% with normal stress data alone, confirming shear stress’s expressive centrality. Using this rich data, we analyse the interplay of sensed-touch features, gesture attributes and individual differences, propose affective-touch sensing requirements, and share technical considerations for performance and practicality.

A generally adaptive way to regulate emotions involves using reappraisal to change the motivational meaning of a distressing situation. Learning and using this strategy can be challenging, especially in intense situations and for vulnerable individuals. Technologies intended to facilitate learning and using reappraisal have mostly relied on verbal communication. Here, we consider the potential for complementing existing approaches with haptic technologies, on the premise that crafted touch interaction can increase intervention accessibility and adaptiveness, both during intense situations and over longer time scales. We discuss the psychological and physiological pathways through which a haptic intervention could make reappraisal easier to learn and use; then propose requirements for CHORA (comforting haptic co-regulating adjunct) technology and a research approach to its validation.

Humans physically express emotion by modulating parameters that register on mammalian skin mechanoreceptors, but are unavailable in current touch-sensing technology. Greater sensory richness combined with data on affect-expression composition is a prerequisite to estimating affect from touch, with applications including physical human-robot interaction. To examine shear alongside more easily captured normal stresses, we tailored recent capacitive technology to attain performance suitable for affective touch, creating a flexible, reconfigurable and soft 36-taxel array that detects multitouch normal and 2-dimensional shear at ranges of 1.5kPa-43kPa and +/- 0.3-3.8kPa respectively, wirelessly at 43Hz (1548 taxels/s). In a deep-learning classification of 9 gestures (N=16), inclusion of shear data improved accuracy to 88%, compared to 80% with normal stress data alone, confirming shear stress's expressive centrality. Using this rich data, we analyse the interplay of sensed-touch features, gesture attributes and individual differences, propose affective-touch sensing requirements, and share technical considerations for performance and practicality.

A teenager's experience of chronic pain reverberates through multiple interacting aspects of their lives. To self-manage their symptoms, they need to understand how factors such as their sleep, social interactions, emotions and pain intersect; supporting this capability must underlie an effective personalized healthcare solution. While adult use of personal informatics for self-management of various health factors has been studied, solutions intended for adults are rarely workable for teens, who face this complex and confusing situation with unique perspectives, skills and contexts. In this design study, we explore a means of facilitating self-reflection by youth living with chronic pain, through visualization of their personal health data. In collaboration with pediatric chronic pain clinicians and a health-tech industry partner, we designed and deployed MyWeekInSight, a visualization-based self-reflection tool for youth with chronic pain. We discuss our staged design approach with this intersectionally vulnerable population, in which we balanced reliance on proxy users and data with feedback from youth viewing their own data. We report on extensive formative and in-situ evaluation, including a three-week clinical deployment, and present a framework of challenges and barriers faced in clinical deployment with mitigations that can aid fellow researchers. Our reflections on the design process yield principles, surprises, and open questions.

In-body lived emotional experiences can be complex, with time-varying and dissonant emotions evolving simultaneously; devices responding in real-time to estimate personal human emotion should evolve accordingly. Models assuming generalized emotions exist as discrete states fail to operationalize valuable information inherent in the dynamic and individualistic nature of human emotions. Our multi-resolution emotion self-reporting procedure allows the construction of emotion labels along the Stressed-Relaxed scale, differentiating not only what the emotions are, but how they are transitioning - e.g., "hopeful but getting stressed" vs. "hopeful and starting to relax". We trained participant-dependent hierarchical models of contextualized individual experience to compare emotion classification by modality (brain activity and keypress force from a physical keyboard), then benchmarked classification performance at F1-scores = [0.44, 0.82] (chance F1=0.22, σ = 0.01) and examined high-performing features. Notably, when classifying emotion evolution in the context of an experience that realistically varies in stress, pressure-based features from keypress force proved to be the more informative modality, and more convenient when considering intrusiveness and ease of collection and processing. Finally, we present our FEEL (Force, EEG and Emotion-Labelled) dataset, a collection of brain activity and keypress force data, labelled with self-reported emotion collected during tense videogame play (N = 16) and open-sourced for community exploration.

The increased prevalence of online collaborative work, through necessity or preference, is accompanied by measurable drops in satisfaction, creativity and energy, often termed "zoom fatigue." As loss of physical co-presence and associated nonverbal communication are identified as contributors, we introduce Synchrobots – robots designed to channel human biophysiology for group connectedness. We propose an Interactivity demo wherein two participants perform an online problem-solving task while wearing physiological sensors and holding a Synchrobot as it physically renders a translation of their partner's heartrate. The setup involves two stations, each with a laptop running Zoom, a set of wearable sensors recording heart rate, respiratory rate, and electrodermal activity, and a Synchrobot. After the problem-solving task, we will invite participants to reflect on how connected they felt with each other as well as their satisfaction with the collaboration quality. Participants may consent to release this data for later inclusion as part of a study.

Touch is valued for supporting emotional bonds. How can people access its warmth and nuance remotely, when tech-mediated proxies are so different from direct touch? We assessed the viability of haptic animations as affect-embedded tactile messages, highlighting findings which demonstrate how crucial relationship and shared history is in influencing these expressions in design and interpretation. To investigate haptic messaging, we first identified a set of 10 common emotion-imbued scenarios by surveying 201 people in distance relationships. Then, using a novel prototype of a wearable spatial vibrotactile display, 10 intimate dyads designed 167 haptic encodings matching the provided scenarios plus 17 user-defined "wildcards". A week later, 21 individuals interpreted sentiment from encodings designed by themselves, a partner or a stranger. We examined design strategies, engagement, and compared human versus machine interpretation accuracy. A striking finding was participants' facile use of shared context when it was available, building on "inside stories" to communicate subtle meanings with high effectiveness despite the unfamiliar medium, and doing so with evident fun. We analyze recognition accuracy and share insights on what it might take to make interpersonal haptic messaging work.

The field of human–drone interaction (HDI) has investigated an increasing number of applications for social drones, all while focusing on the drone’s inherent ability to fly, thus overpassing interaction opportunities, such as a drone in its perched (i.e., non-flying) state. A drone cannot constantly fly and a need for more realistic HDI is needed, therefore, in this exploratory work, we have decoupled a social drone’s flying state from its perched state and investigated user interpretations of its physical rendering. To do so, we designed and developed BiRDe: a Bodily expressIons and Respiration Drone conveying Emotions. BiRDe was designed to render a range of emotional states by modulating its respiratory rate (RR) and changing its body posture using reconfigurable wings and head positions. Following its design, a validation study was conducted. In a laboratory study, participants ( N=30 ) observed and labeled twelve of BiRDe’s emotional behaviors using Valence and Arousal based emotional states. We identified consistent patterns in how BiRDe’s RR, wings, and head had influenced perception in terms of valence, arousal, and willingness to interact. Furthermore, participants interpreted 11 out of the 12 behaviors in line with our initial design intentions. This work demonstrates a drone’s ability to communicate emotions even while perched and offers design implications and future applications.

EDITORIAL article Front. Comput. Sci., 27 July 2023Sec. Human-Media Interaction Volume 5 - 2023 | https://doi.org/10.3389/fcomp.2023.1255784

Practical affect recognition needs to be efficient and unobtrusive in interactive contexts. One approach to a robust realtime system is to sense and automatically integrate multiple nonverbal sources. We investigated how users' touch, and secondarily gaze, perform as affect-encoding modalities during physical interaction with a robot pet, in comparison to more-studied biometric channels. To elicit authentically experienced emotions, participants recounted two intense memories of opposing polarity in Stressed-Relaxed or Depressed-Excited conditions. We collected data (N=30) from a touch sensor embedded under robot fur (force magnitude and location), a robot-adjacent gaze tracker (location), and biometric sensors (skin conductance, blood volume pulse, respiration rate). Cross-validation of Random Forest classifiers achieved best-case accuracy for combined touch-with-gaze approaching that of biometric results: where training and test sets include adjacent temporal windows, subject-dependent prediction was 94% accurate. In contrast, subject-independent Leave-One-participant-Out predictions resulted in 30% accuracy (chance 25%). Performance was best where participant information was available in both training and test sets. Addressing computational robustness for dynamic, adaptive realtime interactions, we analyzed subsets of our multimodal feature set, varying sample rates and window sizes. We summarize design directions based on these parameters for this touch-based, affective, and hard, realtime robot interaction application.

Data tracking is a common feature of pain e-health applications, however, viewing visualizations of this data has not been investigated for its potential as an intervention itself. We conducted a pilot feasibility parallel randomized cross-over trial, 1:1 allocation ratio. Participants were youth age 12–18 years recruited from a tertiary-level pediatric chronic pain clinic in Western Canada. Participants completed two weeks of Ecological Momentary Assessment (EMA) data collection, one of which also included access to a data visualization platform to view their results. Order of weeks was randomized, participants were not masked to group assignment. Objectives were to establish feasibility related to recruitment, retention, and participant experience. Of 146 youth approached, 48 were eligible and consented to participation, two actively withdrew prior to the EMA. Most participants reported satisfaction with the process and provided feedback on additional variables of interest. Technical issues with the data collection platform impacted participant experience and data analysis, and only 48% viewed the visualizations. Four youth reported adverse events not related to visualizations. Data visualization offers a promising clinical tool, and patient experience feedback is critical to modifying the platform and addressing technical issues to prepare for deployment in a larger trial.

An emerging view in cognitive neuroscience holds that the extraction of emotional relevance from sensory experience extends beyond the centralized appraisal of sensation in associative brain regions, including frontal and medial‐temporal cortices. This view holds that sensory information can be emotionally valenced from the point of contact with the world. This view is supported by recent research characterizing the human affiliative touch system, which carries signals of soft, stroking touch to the central nervous system and is mediated by dedicated C‐tactile afferent receptors. This basic scientific research on the human affiliative touch system is informed by, and informs, technology design for communicating and regulating emotion through touch. Here, we review recent research on the basic biology and cognitive neuroscience of affiliative touch, its regulatory effects across the lifespan, and the factors that modulate it. We further review recent work on the design of haptic technologies, devices that stimulate the affiliative touch system, such as wearable technologies that apply the sensation of soft stroking or other skin‐to‐skin contact, to promote physiological regulation. We then point to future directions in interdisciplinary research aimed at both furthering scientific understanding and application of haptic technology for health and wellbeing.

Sharing feelings is essential to empathic communication. This demo allows participants to send and receive emotional signals using a pair of small robots. The robots’ gestures take advantage of the mobility and embodied qualities of robots, and explore a new modality of social signaling that could enhance what is possible through purely digital interfaces. This prototype is part of a larger inquiry into the expressive capabilities of distributed bot systems for empathic communication and connection. Participants will be able to engage in a robot-based emotional exchange using a mobile-device-based interface that offers 6 pre-set robot gestures.