Due to the increasing availability of Augmented Reality (AR) devices, it is now possible to 3D sketch over physical objects. However, users still encounter challenges that affect their stroke accuracy and drawing experience when sketching in AR. This paper presents the design and implementation of the Surface Extension Guides for Augmented Reality (SEGAR) system, a pen-based user interface that allows users to extend the physical surfaces to mid-air while sketching. SEGAR provides users with visual and haptic feedback to help them seamlessly transition between sketching on physical surfaces and mid-air. We ran a pilot study, where four participants sketched a cube in mid-air using SEGAR. Our results show that participants found the system useful.
Extended-reality (XR) training with generative AI could give athletes instant coaching, but current systems may produce feedback that diverges from expert practice. This work aims to bridge this in sabre fencing by giving an AI agent a set of tools for analyzing athletes' performances. We designed these tools based on a formative study involving specialized literature and interviews with three elite coaches. We evaluated the impact of pose analysis tools in AI-generated feedback through an expert evaluation with a two-alternative forced-choice task, comparing a tool-augmented vision-language model with a baseline without tools. Surprisingly, the tool-augmented agent was preferred in 40% of cases (p = 0.15), indicating no significant advantage at this sample size. Coaches valued its richer kinematic insights but noted limitations in capturing intentional tactical deviations. Our study contributes to design considerations that can inform the next generation of AI fencing coaches in XR that are aligned with human coaching practices.
Agility ladder training is commonly used in fitness and therapy to improve coordination in lower-limb exercises by training precise foot placement. Previous research adapted this training approach to immersive environments and emphasized both the potential and the need for enhanced visual feedback in virtual reality (VR) to improve training outcomes. However, effective visual guidance for such tasks in VR remains underexplored, as it is unclear how spatial user interfaces (UIs) should be designed to support accurate foot placement. To address this gap, we conducted a within-subject study with 40 participants, investigating the effects of two visualization techniques: a Foot-Aligned UI (exocentric, attached to the feet) and a Head-Aligned UI (egocentric, floating in view); combined with color-coded performance feedback. Foot positioning accuracy, rotational control, success rate, and perceived workload were measured during VR-based agility ladder tasks. Results show that the Foot-Aligned UI significantly improved foot placement success rates without increasing cognitive load, compared to the Head-Aligned UI and No UI conditions, which was supported by qualitative feedback. In contrast, color-coded step feedback was perceived as helpful but showed no measurable performance benefit. Based on these findings, we derive design recommendations for spatial UIs to support lower-limb motor training in VR.
We present Flick-in, a Japanese text entry method for indirect touch on a smartwatch. Indirect touch is performed without looking at the input surface, which makes it difficult to touch down accurately at the correct location. This difficulty limits the usability of conventional Japanese text entry methods, which require visual confirmation of the input surface. In contrast, few Japanese text entry methods have been proposed specifically for indirect touch. In Flick-in, users first select a vowel using a bezel-initiated swipe, followed by selecting a consonant using a touch-up gesture while observing the output surface. This makes Japanese text entry with indirect touch feasible. We conducted two studies to evaluate the performance of Flick-in using an external display and a mixed reality (MR) environment as output surfaces. The results showed a text entry speed of 29.5 CPM with a total error rate (TER) of 10.0% on the external display and 36.5 CPM with a TER of 6.14% in the MR environment.
Despite its centrality to the overall text entry experience, the task of editing text in virtual reality has received limited attention in the literature. In this paper, we focus on the task of editing text in virtual reality and explore the opportunities afforded by Large Language Models (LLMs) in supporting this task. We specifically investigate how an LLM can be employed to mediate free-form speech-based text editing commands given by users. This flexible approach is inspired by the concept of shared control and allows users to potentially focus less on the specific edit operations required, and more on the desired outcome. However, such flexibility in the user commands can introduce potential ambiguity regarding the desired target of the edit. We therefore study three different interaction conditions representing promising alternative methods for expressing the intended location of an edit: (i) directly via voice commands; (ii) by highlighting words in the text field with a touchbased interaction; and (iii) implicitly via gaze fixations. We find that participants develop effective strategies for collaborating with the LLM-based agent to make edits but also appreciate the more explicit interaction based on touch for expressing edit context.
