While the number of studies about online content warning labels are on the rise, the graphical design of these labels has been understudied. To begin filling this gap, we perform a 13-arm between-subjects experiment (N = 1,288) across two graphic elements of warning labels: intrusiveness and color. We find that highly intrusive labels that require click-through to view content are significantly less effective at decreasing trust in false information than lesser intrusive designs—conditional on those consumers clicking through the highly intrusive label to actively engage with the content. However, highly intrusive warning labels are still better than having no warning label. We find no evidence that color impacts the recall or effectiveness of content warning labels. These results should be taken in concert with the limitations of the study, which are discussed in depth in the article.
New augmented reality (AR) techniques enable increasingly advanced modifications of body representation. This representation and its control are crucial from a social perspective, with many studies focusing on body representation in medical contexts. In this article, we propose a device that allows, in AR, the modification of the user’s appearance by controlling the width of their body. We then conduct a user study to investigate the impact of this modification on body perception, specifically its perceived affordance. Additionally, we record the participant’s internal perception (Sense of Embodiment, body-shape perception) to study their interaction with body perception. Our results show that this system can effectively modify the perceived affordances. They also highlight the strong variability of body perception and its interaction with several factors such as Sense of Embodiment and initial body perception. This work contributes to expanding knowledge of body perception within the specific context of AR and provides significant perspectives for enriching the understanding and treatment of individuals suffering from body image misperception.
Tactile properties of fabrics convey vital information and influence customer decision and satisfaction. The understanding of the tactile properties is not comprehensive due to the multiple tactile properties and the various ways of assessing them. Here, we designed a series of psychophysical experiments to evaluate the multiple tactile properties, including flexible-stiff, smooth-rough, soft-firm, spongy-crisp, warm-cool, using images of flat fabric, draped fabric, rotating fabric, and real fabrics where only touch was allowed, only vision was allowed, and both touch and vision were allowed. Our results show that it is necessary to study the separate tactile properties rather than treat them as a whole. The tactile perception remained consistent yet slightly different among images, videos, and real fabrics. However, for warm-cool, highly consistent perception can be obtained regardless of the condition. Additionally, the tactile properties are not completely independent to each other. Positive and negative correlations among the different tactile perceptions were also observed in our study. In addition, the differences between visual representations of fabrics and real fabrics were also observed for the perception of flexible-stiff, smooth-rough, soft-firm, and warm-cool.
This paper presents an innovative pipeline to eye-tracking with computational efficiency, utilizing a fully event-based algorithm. Traditional eye-tracking methods often rely on grayscale or infrared imaging, which can be computationally intensive and raise privacy concerns. Our research addresses these issues by developing an algorithm that exclusively uses data from event-based sensors, optimizing for settings with limited computational power. The system relies on simple geometrical operations, enabling efficient real-time processing. Experimental results demonstrate the feasibility of this approach, offering a promising solution for eye-tracking in compact, computationally constrained devices.
Driver eye movements are commonly used as a predictor of visual attention and to estimate situational awareness of a road scene. However, these gaze patterns will change in automated vehicles, where drivers can engage in distracting non-driving related tasks (NDRTs). There is some suggestion that using augmented reality (AR) to present NDRTs at eye-level with the road might be beneficial. This paper evaluates gaze behaviour and situational awareness of experienced drivers (N = 22) while performing an NDRT presented in AR. Eye-tracking data were collected during an experiment where participants were simultaneously performing a Hazard Prediction task and an AR keypad dialling task, presented either as a Heads-Up Display (HUD) or Heads-Down Display (HDD). The results suggest that eye movements were significantly affected when engaged with an NDRT, resulting in an impaired ability to predict hazards. This was also evident even when attention was explicitly cued towards the area of the road where a hazardous event was occurring. The results from this study indicate that driver fixations alone do not suggest they have situational awareness, in particular when interacting with a distracting NDRT. From this, we provide considerations for designing AR interfaces to be displayed to drivers of conditionally automated vehicles.
Through digital augmentation it is already possible to smooth one's skin, don virtual bunny ears, or sparkle. With future augmented reality glasses we will be able to see and wear such augmentations while out and about. Yet, the effects of these digital changes to our appearance are presently poorly understood. We investigate how different kinds of facial and bodily augmentations influence how we are perceived by others. For this purpose we recruited four speakers to enact eight different scenarios, creating a dataset of 32 short videos. We apply a selection of beautification and scenario-specific augmented reality filters to these clips to create a set of 160 videos. We then evaluate how the speakers in these videos are perceived through a crowdsourced study with 165 participants. Our analysis shows that filters indeed have an effect on how speakers are perceived, modulating effects from other sources, such as who is speaking. The direction and magnitude of this effect is filter-dependent, pointing to the potential of designing them for specific social situations.
Real-world augmented reality (AR) systems are increasingly network connected in order to succeed in operational spaces dependent on cutting edge sensor and computing technologies, all connected together via the network. This dependency inevitably introduces new sources of error to the AR system through instability in network performance. These errors create familiar problems, such as registration errors, on scales unfamiliar to modern AR systems. The effects of these problems on user performance are difficult to investigate as user performance involves many interrelated factors, some of which are difficult to measure. In this work, we control for user behavior to reduce the complexity of investigating the effects of real-world scale latency and signal dropouts on users’ objective task performance and subjective assessments of the AR system. Behavior was controlled by instructing participants to adopt specific behaviors based on different levels of trust and reliance on AR tags and environmental visual cues. We found significant negative effects of both network error sources on subjective assessments of both the AR system and task, as well as objective task completion time and errors committed. Participants’ subjective assessments and objective task also varied unexpectedly with increasing network error levels, suggesting that human perception of the effects of network errors are not well calibrated to the actual effects on performance. We also found significant interaction between behavior and network errors on task performance, drawing implications not just for AR system design, but also for AR system operation guidelines.
Virtual reality (VR) technology offers highly controlled immersive experiences that can be used to conduct spatial cognition research. However, there are VR-specific perceptual biases that reduce its ability to generalize all findings to real-world scenarios. One specific bias causes observers to perceive objects as closer than intended, a phenomenon called distance compression. Although prior research has focused primarily on visual and technological solutions to distance compression, using other senses as a potential remedy (e.g., hearing) for the bias has not been explored. Here, participants completed both a blind-walking task for distance estimation and a cross-modal perceptual-matching task to evaluate audiovisual spatial displacement. The results showed that real-world sounds spatially displaced both in front of and behind a corresponding visual target increased the accuracy of blind walking relative to vision-only cues, albeit with a small effect. Individual differences measures, including performance on the perceptual-matching task and the relative variability of auditory and visual stimuli, did not account for the individual differences in the effectiveness of spatially displaced sounds. Overall, this study demonstrates the potential for adding multisensory information (specifically the addition of non-virtual sounds) to improve the accuracy of distance perception in VR.
Studies in visual cognition highlight the importance of visual, spatial, and auditory cues in influencing human attention. Such cues often tend to be indicative of actions or events, thereby serving as predictive indicators in both passive observation as well as in interactive engagement. Our research focuses on visual attention in passive observation, particularly examining the manner in which visual, spatial, and auditory cues—henceforth visuo-auditory (shorthand) cues—influence attention on everyday multimodal interaction. We systematically develop a visuo-auditory event model for investigating visual attention in naturalistic embodied settings. Rooted in this event model, we explore the influence of five select visuo-auditory cues—namely, speaking, gaze, relative motion, hand action, and visibility—on visual attention. Our analysis utilizes eye-tracking data from 90 participants observing 27 carefully designed naturalistic event scenarios and correlating their attentional metrics with the select visuo-auditory cues in the backdrop of the developed event model. Findings reveal strong associations between attention and both intra-modal (irrespective of other cues) and cross-modal (combined with other cues) cueing effects, thereby highlighting the nuanced interplay amongst the cues influencing attentional patterns. We develop a systematic and generalized method for analyzing interactions and behavioral parameters, thereby characterizing the impact of visuo-auditory cues on attentional dynamics. Our methodology, combined with the obtained insights into the attentional cueing effects, provides an analytical framework explicating the manner in which everyday (interactive) events directly drive attention under naturalistic conditions. This facilitates not only the precise modeling of behavior and attention allocation but also offers a high-level “experimental lens” for examining interactions in relation to behavioral parameters. Taken together, our methodological and behavioral findings are well-positioned to benefit multiple fields, particularly by advancing human-centered design across diverse application domains. Lastly, towards promoting open-science and for wider dissemination, the complete experimental basis of this research—e.g., event-scenarios, high-quality annotated data, dataset supplementary—has been documented together with instructions on how to use and access experimental data ( https://codesign-lab.org/cognitive-vision/multimodal-cues ).
The interaction between stroke patients and rehabilitation training robots is difficult because the motion intention cannot be executed by limbs due to their weak motor ability. Although brain-computer interaction (BCI) is beneficial for perceiving the motion of patients, it cannot solve the problem independently without receiving enough data about the effectiveness of the robotic assistance. This study proposes a BCI method that integrates rehabilitation training and training effect evaluation in a mixed reality (MR) environment. Three Electroencephalogram (EEG) experimental paradigms were designed, using motor imagery to convert the four-classification problem of motor execution into three binary classification problems. An EEG classification model of a multi-scale convolution residual network based on a multi-head attention mechanism was built, consisting of a multi-head attention mechanism layer, four convolutional layers, and a pooling layer. The highest accuracy and best Kappa coefficient of 12 participants on the testing dataset were recorded. The average classification accuracy of MI, ME1, and ME2 achieves 80.35%, 87.52%, and 86.51%, respectively, proving the proposed classification method has sufficient decoding accuracy. Comparing the hemodynamic response curves of the participants' upper limbs before and after robotic assistance during rehabilitation training, hemodynamic response curves were analyzed quantitatively. With the paired-sample t-test to compare the hemodynamics of a single participant, the significance of the overall mean difference was calculated to evaluate the influence of robotic assistance on enhancing local tissue blood oxygen, improving tissue function, and accelerating local tissue repair. The study verifies the effectiveness of robotic rehabilitation training and helps develop new types of BCI with better coordination between motion intention perception and motor ability.
The present longitudinal quasi-experimental study examined the extent to which music education is related to the development of attentional control. Control of visual attention was examined with the use of an antisaccade task in an eye tracking study. Fifty primary school children (6-7 years old), 25 from music school matched on fluid intelligence with their peers from non-music primary school, performed the antisaccade task three times, at the beginning of school education, after 12 and 24 months. Their eye movements were recorded each time. Over time, attentional control increased in both groups. Music school children performed significantly better than general school children in antisaccadic trials. In line with the prediction, all students' correct responses in the antisaccade task were faster over the time of education, supporting growth in their ability of attentional control. Yet this growth was significantly greater in music school children. Only music school children significantly decrease the latency of saccades toward the target in the correct antisaccade trials. No such trend was significant for children without music education. Finally, fluid intelligence increased over time in both groups. The present study demonstrated a relationship between systematic music education for the development of children's attentional control.
Visualization is essential to studying and communicating data-intensive information. Small variations in a graph can lead to significantly different perceptions, much of which relies on preattentive attributes. Hundreds of millions of people use right-to-left (RTL) scripts, such as Arabic and Hebrew, and these users are known to perceive graphical information differently than left-to-right (LTR) readers. This study addresses the challenge of designing data visualizations for RTL audiences, whose reading habits can lead to misinterpretation of visual elements like timelines and directional data. Through experiments with direction-sensitive and direction-free stimuli, we examined differences in visual interpretation between RTL and LTR readers. The results highlight a significant effect of reading direction on perception, underscoring the need for adaptive visualization techniques that support diverse reading directions.
This research explores the impact of video quality on viewer experience (VX) in the digital age. Videos are ubiquitous in our lives, yet our understanding of how quality variations affect satisfaction and engagement remains limited. By introducing a one-to-many relationship between Quality of Service (QoS) and Quality of Experience (QoE), the study aims to provide practical and deeper insights for content creators and streaming platforms that contemporary subjective metrics cannot provide. It introduces the concept of VX, a novel extension of the QoE, to better capture the complexities of human response to multimedia content. The research combines qualitative and quantitative methods, utilizing established quality assessment frameworks like SSIMplus. Through a combination of statistical and thematic explorations, we provide the basis of a novel framework that has real-world implications for enhancing user satisfaction and the overall quality of video-based content in an increasingly digital world.
Measurement of the cognitive load in mobile situations using wearable devices plays important roles in smart human-computer interactions, physical health monitoring, mental health monitoring, and so forth in daily life. Among wearable devices, wristbands are similar to traditional watches and suitable for most scenarios, such as classrooms and offices; thus, they are widely accepted and becoming increasingly popular. However, most wristbands are only used for step counting and heart rate monitoring. In this article, we propose a cognitive load monitoring method based on wristband heart rate sensors. The sensors used to detect heart rate are generally photoplethysmogram sensors, but the current major manufacturers extract only the heart rate parameter of the photoplethysmogram. We make full use of the waveform of the wristband photoplethysmogram signal to measure real-time cognitive load and establish a real-time cognitive load measurement system, which expands the application scope of the wristband photoplethysmogram sensor and shows great potential for measuring cognitive load in daily life. In experimental validation using an n-back task, our waveform-based CNN model achieved approximately 20% higher accuracy in cognitive load classification versus traditional feature-based methods (e.g., LR, SVM, and GNB), reaching an average of 80.11%. The system successfully classifies the cognitive load and demonstrates the practical viability of the proposed approach.
A perceptual deadzone of a reference force, quantified in terms of the just noticeable difference (JND), is a region where any change in the force with respect to the reference force is not perceived. This perceptual deadzone is employed for efficient encoding of the haptic data through an adaptive sampling mechanism. However, for multi-dimensional force stimuli, the effect of the magnitude and direction of the stimuli on the JND has not been investigated yet, and thus, the structure of the deadzone is yet to be determined. This article aims to investigate the perceptual deadzone for 2D haptic force. To achieve this, we designed a psychophysical experiment in which a subject experiences a 2D kinesthetic force stimulus via a force-feedback device and is asked to respond to the change. We employ a machine learning classification-based approach to analyze the 2D perceptual deadzone on collected haptic responses (perceived and non-perceived) of 12 subjects. Our hypothesis is that a classifier optimized on the collected responses, which yields the least prediction error, will define the structure of the deadzone. The characteristics of the optimal deadzone structure have been analyzed in terms of shape, effect of force direction, and symmetry. The results demonstrate that the deadzone is direction variant and radially asymmetric. Thus, both the force magnitude and direction affect force perception. These findings are significant for the fundamental understanding of multi-dimensional force perception by humans, which can be crucial for enhancing haptic force-feedback control systems in virtual reality or robotic controls.
Virtual health assistants, digital characters designed to guide patients through healthcare interventions, offer scalable solutions, but their effectiveness may depend on user perceptions of realism. This study investigated how the visual fidelity and voice modality of virtual health assistants influence perceived realism, perceived voice naturalness, and intentions to screen for colorectal cancer among Black American adults aged 45–75. In a 2 × 4 factorial between-participants experiment (N = 266), participants were randomized to virtual health assistants varying in visual fidelity and voice modality. Structural equation modeling revealed that voice naturalness had a strong positive effect on perceived realism, which in turn significantly predicted screening intentions. Visual fidelity also contributed directly to perceived realism, although to a lesser extent. These results suggest that enhancing voice naturalness in virtual health assistants may be a critical design priority for improving engagement and promoting preventive health behaviors in digital health interventions.
Negative psychological effects from losing in online competitive games, such as frustration and reduced self-evaluation, can lead to player disengagement. Previous research using puzzle tasks has shown that providing losers with positive feedback on their performance after the task can enhance their intrinsic motivation. However, such findings were demonstrated in analog settings where feedback was delivered verbally by a human experimenter, and it remains unclear whether similar effects hold in digital environments such as online competitive games. This study investigates a feedback intervention using a virtual agent on a computer, aimed at mitigating such negative impacts and promoting sustained intrinsic motivation. In a controlled experiment with 45 participants, we compared three types of feedback: No Feedback (NF), Feedback for the Participant (FP), which included praise for the player's good moves, and Feedback for Both (FB), which also praised the opponent and emphasized social connection. The results indicated that providing feedback, whether it was directed at the player alone or at both the player and the opponent, significantly increased perceived competence, autonomy, intrinsic motivation, and the intention to play again compared to not providing any feedback at all. We discuss the potential of personalized, praise-based feedback in sustaining healthy motivation during competitive gameplay.
In this study, we examined the impact of agent familiarity and knowledgeability on several variables spanning agent perceptions (i.e., perceived knowledge, familiarity, trust, anthropomorphism, uncanny valley effect, and likability), social and emotional experiences (i.e., co-presence, rapport, cognitive process expectations, and willingness for future interaction), and conversation dynamics (i.e., conversation transcript, participants' response word count, and response time). We created two virtual agents for the study: a digital replica of a professor from our department (i.e., familiar agent) and an agent with similar demographic variables (i.e., age, gender, and ethnicity) but with a fabricated appearance and voice (i.e., unfamiliar agent). We implemented both agents to exhibit two levels of knowledgeability (i.e., low and high) in the domain of game development and course-specific information. We used large language models (LLMs) to provide the agents with persona information and domain knowledge through prompt engineering. For our user study, we followed a 2 (familiarity: unfamiliar vs. familiar agent) x 2 (knowledgeability: low vs. high knowledgeability) within-group study design and recruited 32 participants who engaged in a 5-minute, conversation-based virtual reality (VR) interaction with all four experimental conditions: unfamiliar agent with low knowledgeability (ULK), unfamiliar agent with high knowledgeability (UHK), familiar agent with low knowledgeability (FLK), and familiar agent with high knowledgeability (FHK). The findings demonstrated a significant main effect of agent familiarity on perceived knowledge, suggesting that familiarity plays a crucial role in shaping users' perception of the agent's knowledgeability level. Besides perceived knowledge, familiarity also affected all other variables, apart from co-presence. Conversely, agent knowledgeability affected perceived familiarity, trust, anthropomorphism, cognitive process expectations, willingness for future interaction, conversation content, and participants' response word count. Finally, we found an interaction effect between agent familiarity and perceived knowledge, indicating that familiarity has a significant influence on users' perceptions of the agent's knowledgeability. This study contributes to the field of conversational human-agent interaction in VR by providing empirical evidence on how adapting both familiarity and knowledgeability of virtual agents can significantly enhance user experience, offering valuable insights into designing more engaging, trustworthy, and effective embodied conversational agents.
Several studies have examined the potential of immersive technologies, such as Virtual Reality (VR), to enhance the effectiveness of memorization. However, existing research has not specifically focused on individuals with Attention Deficit Hyperactivity Disorder (ADHD), and only a limited number of studies have examined the effectiveness of Memory Palace (MP) as a memorization aid for this population, who often experience working memory impairments. To address this gap, we recruited participants with an official diagnosis of ADHD and conducted an experiment in which we investigated the impact of both the Traditional MP technique and a VR version to assess their impact on a memorization task. Our findings indicate that the effectiveness of a VR-based MP might be influenced by prior experience with VR systems, the level of familiarity with the virtual environment, and the MP technique. Nevertheless, our results demonstrate that the MP technique has the potential to enhance recall performance in individuals diagnosed with ADHD.
