AI capabilities for document reader software are usually presented in separate chat interfaces. We explore integrating AI into document comments, a concept we formalize as AI margin notes. Three design parameters characterize this approach: margin notes are integrated with the text while chat interfaces are not; selecting text for a margin note can be automated through AI or manual; and the generation of a margin note can involve AI to various degrees. Two experiments investigate integration and selection automation, with results showing participants prefer integrated AI margin notes and manual selection. A third experiment explores human and AI involvement through six alternative techniques. Techniques with less AI involvement resulted in more psychological ownership, but faster and less effortful designs were generally preferred. Surprisingly, the degree of AI involvement had no measurable effect on reading comprehension. Our work shows that AI margin notes are desirable and contributes implications for their design.

We propose a technical method and system to enable an unmodified FDM 3D printer to fasten fabric in a manner analogous to traditional sewing. Rows of printed “needles” pierce layers of fabric with “thread” printed to fasten them together. This recreates traditional kinds of stitches while also enabling new ones. A technical evaluation shows the strength is as good or better than traditionally sewn seams. Using this sewing inspired joinery method, a software system translates traditional sewing patterns into a multi-step workflow. Steps are ordered and grouped to balance automation with traditional sequencing for human fabric handling. Intermittent requests by the system require the user to place fabric pieces aided by printed registration marks. A study suggests people with little experience can use the system and complete necessary fabric handling steps. Finally, enhanced types of seams are presented, such as decorative stitches, rivets, fringes, and print-in-place fasteners like buttons and zippers.

People often use handwritten notes and sketches to externalize ideas for ideation. To integrate large language models (LLMs) into this practice, we propose Thinkink. Prompts can be handwritten text or drawn sketches with LLM-generated responses visualized as ink-like text and sketches spatially integrated into a shared canvas. A semantic tree streamlines ink interpretation, and a lightweight UI provides explicit control using a state machine. The tool was designed using a three-stage process. A formative study (N=12) examined current practices with conventional and digital inking methods. The results informed a technical probe for a diagnostic study (N=6) identifying usability and human-LLM interaction challenges. This motivated the design of Thinkink, with a final study (N=10) examining how people incorporate it into their ideation practices. We contribute design implications and a tool for ink-native LLM interaction where the user and LLM write and draw in a shared 2D canvas.

Word suggestion is a common feature of typing interfaces, but previous studies have found unclear or negative impacts. We report on three studies controlling for word suggestion accuracy and typing efficiency. Our accuracy factor uses a new methodology based on common word suggestion metrics. Typing efficiency is controlled by device type in the first study, and by artificial impairments in the following two. Results show that suggestions are used less as typing efficiency increases, and only improve speed when highly accurate, even with low typing efficiency. Inline suggestions save about 4% more keystrokes and increase typing speed by 2 words per minute compared to a bar suggestions, though they are more distracting. Based on our findings, we propose a model linking suggestion usage to accuracy and typing speed, and discuss implications for designing automation features in typing systems.

We investigate sketch-like pen input as an alternative way to support execution control in interactive debugging. In our interface, programmers draw lightweight marks to set breakpoints, use symbolic strokes to control execution, and extend strokes into spirals to repeat traversal actions. The prototype combines gesture recognition with Python execution tracing in a conventional editor interface. In a controlled study with 24 programmers, we compared the sketch interface with conventional mouse-and-keyboard input on debugging tasks that required breakpoint placement, step-wise execution, and runtime state inspection. The results show that sketch-like input can support these execution-control tasks, while also introducing challenges in precision, recognition, and gesture recall. Our findings suggest that pen input is most promising where debugger interactions benefit from spatial grounding or continuous movement, rather than as a wholesale replacement for conventional debugging controls.

The photo-taking-impairment effect is when memory is hindered by the act of taking photos. Henkel established this effect for memory recognition only, and subsequent work primarily uses simulated photography tasks with stimuli from museums. To gain more clarity on effects of photography on memory recall, we conducted an experiment to reproduce Henkel’s findings. Participants observed objects individually arranged around a large room, photographed some of them, and after a break listed the objects they remembered. When they took photos, they could recall 14% fewer objects than just observing it, confirming the existence of a photo-taking-impairment effect for recall.

Intermittent Interaction is a turn-taking approach used to interact with fabrication devices to do something that otherwise would be impractical or impossible for the machine. We investigate how people perceive intermittent interactions in a controlled study. A LEGO assembly task with timed lock boxes simulates human involvement with a semi-automated machine process, similar to a 3D printer. This is used in an in situ study with 12 participants over 4-hour sessions with experimental controls for number of interactions and step complexity. Results suggest complex interactions during assembly can amplify the perceived value of the assembled object and increase enjoyment. Participants used either a clustered or evenly distributed strategy to schedule interactions, which can be modelled with simple heuristics. We contribute evidence that intermittent interaction is generally acceptable for creation tasks and practical guidelines for integrating intermittent interactions into semi-automated fabrication systems.

Movement-based spatial interaction in VR can present significant challenges for people with limited mobility, particularly due to the mismatch between the upper body motion a VR app requires and the user’s capabilities. We describe MotionBlocks, an approach which enables 3D spatial input with smaller motions or simpler input devices using modular geometric motion remapping. A formative study identifies common accessibility issues within VR motion design, and informs a design language of VR motions that fall within simple geometric primitives. These 3D primitives enable collapsing spatial or non-spatial input into a normalized input vector, which is then expanded into a second 3D primitive representing larger, more complex 3D motions. An evaluation with people with mobility limitations found that using geometric primitives for highly customized upper body input remapping reduced physical workload, temporal workload, and perceived effort.

We introduce the concept of code shaping, an interaction paradigm for editing code using free-form sketch annotations directly on top of the code and console output. To evaluate this concept, we conducted a three-stage design study with 18 different programmers to investigate how sketches can communicate intended code edits to an AI model for interpretation and execution. The results show how different sketches are used, the strategies programmers employ during iterative interactions with AI interpretations, and interaction design principles that support the reconciliation between the code editor and sketches. Finally, we demonstrate the practical application of the code shaping concept with two use case scenarios, illustrating design implications from the study.

Marine science researchers are heavy users of software tools and systems such as statistics packages, visualization tools, and online data catalogues. Following a constructivist grounded theory approach, we conduct a semi-structured interview study of 23 marine science researchers and research supports within a North American university, to understand their perceptions of and approaches towards using both graphical and code-based software tools and systems. We propose the concept of fragmentation to represent how various factors lead to isolated pockets of views and practices concerning software tool use during the research process. These factors include informal learning of tools, preferences towards doing things from scratch, and a push towards more code-based tools. Based on our findings, we suggest design priorities for user interfaces that could more effectively help support marine scientists make and use software tools and systems.

We conduct the first ever two-session controlled lab study (n = 36) where end-users are prompted to install real benign and malicious software on a standard Windows laptop. The first session observes typical decision making strategies when participants assess software for potential threats without any instructions. The second session repeats the experiment after introducing an "enhanced task manager" application with information like CPU usage, files accessed, and network destination country to examine if decision making strategies change with more system-level information. The time, confidence, and accuracy to classify software as benign or malicious is recorded, along with participant comments using a "think-aloud" protocol. These comments form a dataset of 2,651 excerpts that are coded into four top-level categories of "indicators" with 25 sub-categories. These indicators provide a perspective into how end-users examine and analyze software in-situ. Overall, end-users are surprisingly accurate at classifying malware and become even better when provided with more process-level statistics. Our analysis uncovers common misconceptions, shows reliance on indicators that bad actors could circumvent, and provides actionable insights for software and operating system providers to improve user interfaces and notifications.

We contribute an analysis of the prevalence and relative performance of archetypal VR menu techniques. An initial survey of 108 menu interfaces in 84 popular commercial VR applications establishes common design characteristics. These characteristics motivate the design of raycast, direct, and marking menu archetypes, and a two-experiment comparison of their relative performance with one and two levels of hierarchy using 8 or 24 items. With a single-level menu, direct input is the fastest interaction technique in general, and is unaffected by number of items. With a two-level hierarchical menu, marking is fastest regardless of item number. Menus using raycasting, the most common menu interaction technique, were among the slowest of the tested menus but were rated most consistently usable. Using the combined results, we provide design and implementation recommendations with applications to general VR menu design.

The feeling of something belonging to someone is called "psychological ownership." A common assumption is that writing with generative AI lowers psychological ownership, but the extent to which this occurs and the role of prompt length are unclear. We report on two experiments to examine the relationship between psychological ownership and prompt length. Participants wrote short stories either completely by themselves or wrote prompts of varying lengths. Results show that when participants wrote longer prompts, they had higher levels of psychological ownership. Their comments suggest they thought more about their prompts, often adding more details about the plot. However, benefits plateaued when prompt length was 75-100% of the target story length. To encourage users to write longer prompts, we propose augmenting the prompt submission button so it must be held down a long time if the prompt is short. Results show that this technique is effective at increasing prompt length.

Scrolling and paginating can both be used to read documents on smartphones. Prior work mainly suggests that pagination leads to higher reading comprehension, but these studies have either focused on desktop environments, are over 10 years old, or lack ecological validity. Therefore, we replicate these experiments to better understand the differences between scrolling and pagination. Through a large-scale, between-subjects online study, participants read a short story using either pagination or scrolling, and answered multiple-choice questions. Our results found no significant differences between these two techniques for reading comprehension, duration, and task workload, which differs from findings presented in prior work.

Writing longer prompts for an AI assistant to generate a short story increases psychological ownership, a user's feeling that the writing belongs to them. To encourage users to write longer prompts, we evaluated two interaction techniques that modify the prompt entry interface of chat-based generative AI assistants: pressing and holding the prompt submission button, and continuously moving a slider up and down when submitting a short prompt. A within-subjects experiment investigated the effects of such techniques on prompt length and psychological ownership, and results showed that these techniques increased prompt length and led to higher psychological ownership than baseline techniques. A second experiment further augmented these techniques by showing AI-generated suggestions for how the prompts could be expanded. This further increased prompt length, but did not lead to improvements in psychological ownership. Our results show that simple interface modifications like these can elicit more writing from users and improve psychological ownership.

We conducted two studies: one on a horizontal surface and another across horizontal, front and side surfaces, to understand how coupled and decoupled methods affect target acquisition in dynamic peephole. While results show similar accuracy between the methods, the coupled method shows a faster acquisition and shorter total time in a cross-surface space. Using the same technique to search for the targets, participants in the coupled condition moved the peephole faster but found targets later, while in the decoupled condition, they moved the peephole slower but found targets earlier. Overall, participants preferred the coupled condition for ease of use and reduced physical demand, and the decoupled condition for its accuracy. These findings suggest that coupled and decoupled methods offer advantages for specific scenarios on surfaces, providing insights for future design on dynamic peephole interfaces.

We characterize and demonstrate how the principles of direct manipulation can improve interaction with large language models. This includes: continuous representation of generated objects of interest; reuse of prompt syntax in a toolbar of commands; manipulable outputs to compose or control the effect of prompts; and undo mechanisms. This idea is exemplified in DirectGPT, a user interface layer on top of ChatGPT that works by transforming direct manipulation actions to engineered prompts. A study shows participants were 50% faster and relied on 50% fewer and 72% shorter prompts to edit text, code, and vector images compared to baseline ChatGPT. Our work contributes a validated approach to integrate LLMs into traditional software using direct manipulation. Data, code, and demo available at https://osf.io/3wt6s.

Highlighting text in a document is a common active reading strategy to remember information from documents. Learning theory suggests that for highlights to be effective, readers must be selective with what they choose to highlight. We investigate if an imposed user interface constraint limiting the number of highlighted words in a document reader can improve reading comprehension. A large-scale between-subjects experiment shows that constraining the number of words that can be highlighted leads to higher reading comprehension scores than highlighting nothing or highlighting an unlimited number of words. Our work empirically validates theories in psychology, which in turn enables several new research directions within HCI.