Locomotion is a key factor in virtual reality (VR) navigation but remains underexplored in asymmetric setups combining VR and mixed reality (MR). Our study investigates whether visualized teleportation can serve as a middle ground, balancing the social connection afforded by real walking with the efficiency of teleportation within asymmetric VR-MR setups. We conducted a mixed-design study (N = 24, 12 pairs) to examine how locomotion methods (teleport, real walking, visualized teleport), platform (VR vs. MR), and task type (social vs. navigation) affect user experience and performance. Results show that locomotion technique significantly influences both user experience and navigation effectiveness. Our findings highlight visualized teleport as a promising compromise, offering improved presence and user experience without imposing physical demand, providing practical guidance for designing locomotion in asymmetric mixed reality environments.
Interaction in Augmented Reality primarily relies on raycast pointing and mid-air touch. An alternative consists of using the non-dominant hand as a touch-sensitive surface, enabling more comfortable, less fatiguing input. AR UI design guidelines have so far discouraged this alternative because of poor hand tracking performance when the hands overlap, favoring touchpads in the air near the hand, rather than on the hand. But significant improvements to the hand tracking capabilities of recent commodity headsets suggest that on-hand pads may now be feasible. We develop an on-hand touchpad prototype and conduct two studies that involve both discrete input and continuous control tasks. The first study compares such on-hand pads to baseline in-air and on-object pads, showing comparable performance despite some limitations in tracking accuracy. The second study quantifies the advantage of on-hand and in-air pads over on-object pads during transitions between touchpad input and other physical hand activities.
Smartphones allow users to control the sharing of their data with apps according to their privacy preferences. Yet, users struggle to enact their privacy preferences via the available permission settings. To understand whether these difficulties result from inaccurate understanding and/or suboptimal interface design of the permissions manager, we designed and administered an online questionnaire to smartphone users from the United States (n = 151). We asked the participants to rate and rank the importance of the permissions commonly available on smartphones and to describe their understanding of what each setting controls. We found that a majority of users deem some permissions as important or unimportant, with the importance of other permissions varying across users based on use and privacy concerns. Our findings indicate that users misunderstand several permissions and express unfamiliarity with how some of them operate. We apply the insight from our study to derive suggestions to enhance smartphone permission managers by promoting personalized and efficient user interaction and more accurate user comprehension of functional operation.
Interaction mining is a popular technique for capturing design and interaction data while a mobile app is being used. Over the years, researchers have leveraged interaction mining systems to build large repositories of interaction data, enabling novel, ML-based tools for UX practitioners, designers, and programmers. Existing interaction mining systems range from simple screen recorders — which are easy to use but capture sparse, unstructured data — to complex installations requiring dedicated hardware and custom OS forks — which yield rich, high-fidelity traces but are difficult to deploy outside of a lab environment. This paper presents ODIM, an on-device framework for mobile interaction mining that produces detailed trace metadata. The framework is reified in an Android implementation based on a simple APK that users can install on their personal devices. The paper outlines ODIM’s design principles, describes its implementation, evaluates the system on traces collected from 100 popular apps on the Google Play Store, and discusses future avenues for scaling the utility and impact of interaction mining systems. The ODIM software, source code, and online trace repository are all freely available at interactionmining.org, for anyone to use and contribute to.
Experiential accounts of personal informatics are important as they inform us about users’ lived experiences with tracking technologies. However, these accounts describe experience at a meta-level and are not specifically related to how personal informatics is often composed of a collection of interconnected artefacts—what we call an ecology. To design personal informatics artefacts, we need a thorough understanding of how they are experienced as ecologies in practice. To this end, we interviewed 12 users of these ecologies using an interpretative phenomenological analysis approach. Our results show that users experience these ecologies through four aspects: effort minimising, sustainable tracking, performance explainability, and disconnection. We conceptualised these aspects as an automation experience—where technologies work collectively to minimise user effort while maximising insight. This conceptualisation provides a novel lens for analysis that can inform the design of more integrated and user-centered personal informatics ecologies.
Advancements in virtual reality (VR) and portable head-mounted displays are transforming how surgical training is delivered, particularly in minimally invasive procedures like endoscopic neurosurgery. Current VR-based surgical training systems often rely on expensive proprietary hardware and lack features such as patient specificity, adaptive difficulty, or standardized evaluation metrics. This work addresses these challenges by utilizing commercially available VR headsets and open-source components to build a unified platform that integrates anatomy visualization, procedural simulation, performance evaluation, and collaborative learning. The platform includes features such as AI-based patient-specific modeling, gamified progression, "ghost surgery" instructional modules, and a privacy-preserving training environment. The research contributes new methods for performance benchmarking in VR surgery, cost-effective system design, and pedagogically informed simulation design. The project is validated through user studies and technical evaluations in collaboration with neurosurgical experts.
Isolated, Confined, and Extreme (ICE) environments, such as encountered in space exploration missions, pose unique physical and psychological challenges that influence user interactions with computer systems, yet remain considerably less documented compared to conventional settings. To investigate the impact of such environments on mobile interaction, we conducted an experiment involving a crew of analog astronauts operating a drone via a handheld controller in both a conventional Earth-based setting and an ICE environment represented by the extreme landscape of the Mars Desert Research Station. Our findings reveal how the user experience of mobile interaction evolves over multiple evaluation sessions conducted over a two-week period in the ICE environment, for which we analyze both pragmatic and hedonic dimensions, such as perceived efficiency, adaptability, novelty, usefulness, and trust. Based on our findings, we outline a set of implications for the design of mobile interaction intersecting space research through the distinctive lens of astronaut-drone interaction.
The automobile is a complex mobile multimedia system with an operating system that is mostly hidden and closed. For research, this creates two issues: collecting vehicle data for AI is arduous and rapidly building dashboard prototypes requires specialized expertise. We introduce CANViz to overcome these barriers and allow for data capture, AI integration, and rapid visualizations. CANViz logs real time CANBUS/OBD-II vehicle data combined with a GPS logger to collect real world position data. It additionally uses computer vision (Berkeley Deep Drive) to identify road signs, vehicles, and pedestrians. Beyond logging, CANViz pipes these signals into Node-RED where an end user programmer can combine the inputs into a desired web socket that outputs to a web page instrument. The system is built on ROS and Raspberry Pi OS and operates on a local network of low cost CPUs, enabling simplified data collection and rapid prototyping research for the automotive vehicles.
Technology increasingly shapes our social interactions, both online and in person. Strong social connections and face-to-face interactions are vital for wellbeing, especially with close relationships. In this context, technology can play an ambivalent role: whereas it often has a negative impact on the quality of these interactions, it carries potential to enrich conversations and improve social interactions if used in a meaningful way. We design a prototype that implements subtle intervention strategies to foster meaningful technology use, specifically aimed at enhancing close relationships during in-person interactions. We evaluate the prototype within an exploratory, two-week in-the-wild user study with 6 tandems (N=12). Our findings suggest that the strategy of "us-reflection" – a social approach to reflection – contributes to mutual awareness of participants’ shared time. Our prototype encouraged more meaningful interactions by proposing conversation topics or suggesting activities, ultimately strengthening close relationships and fostering more intentional, engaging, and rewarding social experiences.
Research on understanding and supporting the experiences of people with noise sensitivity (PWNS) and their challenges is limited within HCI. Therefore, we build on prior work to understand the challenges they consider and what technological solutions they create to support them. Through eight participatory design workshops involving PWNS and their carers, we considered their needs and challenges and how technology can be designed to support their well-being. Results indicate that wearable and mobile technology can facilitate awareness of sensory triggers and impacts on their well-being. Further, enabling both self and collaborative regulation is also necessary, especially as end users seek independence or interdependence with those around them to manage their experiences. We identified three tensions for designing technology to support PWNS and their sensory experiences.
With the widespread use of online conferencing tools like Zoom on both PCs and mobile devices, there is a growing risk of unintentional exposure of private information through screen sharing. To address this issue, automated systems have been developed to detect and obscure private information on screens. However, the effectiveness of visual obfuscation methods—such as masking and blurring—has not been thoroughly evaluated. This study aims to identify an effective obfuscation method that achieves a balance between privacy protection and viewing experience for on-screen information. We conducted an online experiment with 55 participants, in which six types of filters were evaluated, including variations in blurring, pixelation, and masking strength and color. The results indicate that users preferred filters with high privacy protection, particularly those that appeared more natural. Among the tested methods, background-color masking received the highest ratings for both privacy protection and viewing experience, suggesting that it is a well-balanced obfuscation method.
To enable automated testing of gesture-based applications, synthetic touch gestures must closely mimic real touch gestures. Although various generation methods exist, comparing them remains difficult due to the lack of standardized evaluation procedures. This work builds upon an existing initial evaluation framework that organizes relevant metrics into categories based on a taxonomy. We specify open aspects of the framework to support its practical and consistent application. We also propose a hierarchical structure for weighting and comparing metrics across categories. Finally, we outline how the framework can be extended to evaluate multi-stroke and multi-touch gestures.
Current smart home technologies rely on touchscreens and voice assistants for interaction. These interfaces lack tactile engagement and fail to support users’ daily routines and preferences, leading to poor user experiences (UX). Designing tangible user interfaces (TUIs) that align with user preferences can improve the status quo. This paper explores the potential of TUIs using everyday objects for smart home control. Four prototypes — a vase, pillow, coaster, and flower — were evaluated for UX and metaphor alignment through a within-subjects study with 25 participants. Using meCUE questionnaires and semi-structured interviews, we examined how physical and contextual attributes influence UX. Our findings indicate that everyday objects are effective TUI and produce positive UX, provided careful consideration is given to their physical and contextual attributes. This research expands our understanding of TUIs’ role in bridging the digital-physical divide and offers practical guidelines for embedding intuitive smart home controls into everyday objects.
With the rapidly growing number of articles published in the field of Human-Computer Interaction (HCI), it has become increasingly difficult to keep track of a given research area. Consequently, secondary research methods such as evidence synthesis are of growing importance. In this work, we transfer an evidence synthesis method from epidemiology to the field of HCI, develop a novel visualization technique specifically aimed at evidence synthesis, and make this technique available to the HCI community via the open source web application LayEv. To demonstrate how our approach can reveal new relational and causal paths between variables, we apply it to a set of publications identified in a prior literature review and visualize the results with LayEv. Future researchers can leverage our method to obtain comprehensive overviews of research areas, identify trends, and visualize existing evidence synthesis results.
Today's context-aware mobile phones allow developers to build intelligent and adaptive applications. The data demand induced by context awareness leads to decreased trust and increased privacy concerns. However, users' deeper reasons and real-world fears that underlie these concerns are not fully understood. We conducted an online survey (N=100) and semi-structured interviews (N=20) to understand users' concerns about smartphone data privacy. We investigated three key areas: general user understanding and misconceptions, specific indepth concerns, and mitigation strategies. We found that effective transparency and control are the central themes across all areas. Users are concerned about privacy issues negatively impacting their lives, especially through financial loss, physical harm, or manipulation. We show that privacy measures should be implemented with a stronger focus on the user by keeping the user in the loop through transparency and control.
We present Surrogate Avatar, an adaptive telepresence method that enhances user mobility and situated co-presence in symmetric avatar-mediated communication. The system enables a remote user’s avatar to autonomously position itself in socially and environmentally appropriate locations within the local user’s space—based on spatial affordances, interactional norms, and environmental constraints—supporting fluid interaction without requiring a shared environmental context. Through a formative study, we derived key adaptation objectives and implemented them using a distributed optimization framework based on the AUIT system. The framework distributes adaptation tasks across server and client to balance responsiveness and computational efficiency. A user study involving both stationary and nomadic scenarios demonstrated consistently high usability and presence, with some limitations observed under walking conditions. An additional exploratory field study in a semi-structured public setting demonstrated the system’s viability beyond controlled lab conditions. These findings motivate future designs of mobile telepresence systems that dynamically adapt to spatial and conversational context while mitigating misunderstandings that can arise from asymmetric environmental awareness and supporting privacy-sensitive interaction.
A current trend in mobile user interface design is to provide alternative color modes (ACMs), such as light mode, dark mode, and high contrast mode, to improve people's interaction experiences according to their vision access needs and/or the environment. For example, high contrast mode can improve UI visibility for people with low vision and for people using their smartphones in bright sunlight. However, little is known about the experiences of people using ACMs. To address this, we interviewed 29 people with and without vision impairments to discuss the benefits and challenges of ACMs. We found that while ACMs are beneficial, the current implementation results in accessibility and usability issues, particularly for people with vision impairments (e.g., negative health consequences and affected work performance). Using our findings, we outline steps the HCI community should consider to address current limitations and improve future ACMs.
In mobile, AI-enhanced work environments, many professionals find themselves both empowered and overwhelmed. While automation promises to save time, conversations with designers, analysts, and managers reveal a different reality: expectations accelerate, interruptions multiply, and the space for focused, meaningful work continues to shrink. This article draws on real-world experiences and workplace reflections to explore the paradox of digital productivity. We examine how fragmented attention, notification overload, and performance pressure are reshaping not only workflows but also workers’ sense of clarity and fulfillment. Through three illustrative examples from industry, we underscore the need to redesign work with intention. This perspective encourages readers to reconsider how presence, rhythm, and small rituals can help restore attention and satisfaction in an age of constant connectivity.
Connecting personal devices to the in-vehicle infotainment system has become mainstream in modern vehicles, contouring a distinctive context of use characterized by the user interface being distributed across multiple interactive systems, including those in the vehicle and the users’ personal digital devices, likely involving different input and output modalities. However, distributed user interfaces (DUIs), despite extensively studied in other application domains, have not been addressed to the same extent for in-vehicle interactions. In this context, we examine in-vehicle DUIs and report the results of an exploratory study conducted with twenty-four drivers, who shared their preferences regarding digital device use inside the vehicle. To complement our findings, we also present a demonstrative application featuring a user interface with interaction modalities distributed across the in-vehicle infotainment system and the driver’s smartwatch.
Empirical findings in gesture-based interaction often stem from highly controlled experimental settings, which raises concerns about their generalizability. To explore how variations in such settings influence discoveries on user-defined gestures, we selected an end-user elicitation study involving smart rings that had been replicated at least once. By reusing the same stimuli, equipment, and data collection method, we conducted four new replications of the original study, involving a total of 120 participants across four different research teams. Our results show that smart ring gestures elicited in these replications overlap only partially, with differences in agreement rate, thinking time, and goodness of fit with corresponding system functions. We argue that systematic replication of gesture elicitation studies is essential for generalizable gesture sets.
