Touch is an inherent part of human social interactions and the diversity of its functions has been highlighted in numerous works. Given the varied roles of touch, with technology-mediated communication being a big part of our everyday lives, research has been interested in enabling and enhancing distant social interactions with mediated touch over networks. Due to the complexity of the sense of touch and technological limitations, multimodal devices have been developed and investigated. In this article, we explore the use of mediated visual touch in distant social interaction. Adopting an interactionist and collaborative approach to human communication, we focus on the communicative functions of distant touch behaviours which interactants co-elaborate throughout their mediated interactions. For this purpose, we conducted an exploratory study placing five romantically involved couples in interaction, where each discussed shared biographical events via a video call, using mediated touch devices (producing vibration and coloured lights). Their interactions were recorded, and excerpts were presented to participants in interviews using a collective confrontation technique (participants are confronted with a recording of their activity and encouraged to comment on it). This technique allows a better understanding of the participants’ points of view on their use of the touch devices in context. Through analysis of the interviews, our results highlight: (1) a variety of visual-touch functions with a redistribution of functions mostly supported by other modalities of communication in face-to-face interactions, such as illustrating aspects of the ongoing conversation; (2) the visual-touch characteristics as well as the verbal, paraverbal and non-verbal indicators of the interactional context considered by the participants to make sense of the stimuli and; (3) the multifactorial and dynamic aspects of the co-elaboration process of the visual-touch functions, reaffirming the role of interactional context, combined with cultural and biographical knowledge, in the meaning making.

This paper explores the potential of pneumatic haptic interfaces in enhancing human-computer interaction. We present two projects: one augmenting movie experiences with emotion-synchronized haptic feedback, and another integrating pneumatic interfaces into steering wheels for improved driver takeover in autonomous vehicles. The movie experience project demonstrated enhanced emotional engagement, while the automotive application showed improved safety and user trust. These applications highlight the versatility of pneumatic haptic technology across entertainment and safety contexts. We discuss the advantages of the Baromorph technique and outline future research directions, including comparative studies with vibrotactile feedback and machine learning approaches. This work contributes to the development of more insightful and emotionally intelligent interactive systems.

Human memory has notable limitations (e.g., forgetting) which have necessitated a variety of memory aids (e.g., calendars). As we grow closer to mass adoption of everyday Extended Reality (XR), which is frequently leveraging perceptual limitations (e.g., redirected walking), it becomes pertinent to consider how XR could leverage memory limitations (forgetting, distorting, persistence) to induce memory manipulations. As memories highly impact our self-perception, social interactions, and behaviors, there is a pressing need to understand XR Memory Manipulations (XRMMs). We ran three speculative design workshops (n=12), with XR and memory researchers creating 48 XRMM scenarios. Through thematic analysis, we define XRMMs, present a framework of their core components and reveal three classes (at encoding, pre-retrieval, at retrieval). Each class differs in terms of technology (AR, VR) and impact on memory (influencing quality of memories, inducing forgetting, distorting memories). We raise ethical concerns and discuss opportunities of perceptual and memory manipulations in XR.

As Virtual Reality (VR) headsets become mobile, people can interact in public places with applications often requiring large arm movements. However, using these open gestures is often uncomfortable and sometimes impossible in confined and public spaces (e.g., commuting in a vehicle). We introduce the concept of finger mapping, re-associating small-scale finger motions onto virtual arms in a larger VR space. Finger mapping supports various interactions (e.g., arms swinging movement, selection, manipulation, and locomotion) when the environment is constrained and does not allow large gestures. Finally, we discuss the opportunities and challenges of using finger mapping for VR interactions.

New immersive devices (e.g., virtual or augmented reality) enable displaying large amounts of data in space to better support data analysis. Manipulating this data efficiently is crucial, but challenging because the user must be able to activate various commands or adjust various values while remaining free to move. Using the whole body offers several valuable advantages: 1) The body provides a physical support as an interactive surface, which improves accuracy and makes it less tiring to interact; 2) Using the body does not impair mobility and avoids handling devices; 3) Proprioception makes it possible to interact eyes-free, including for choosing values in a range; 4) By leveraging spatial memory, the body helps memorizing commands, thus interacting in expert mode (i.e., perform quick actions without visual feedback). In this position paper, we analyze various ways of interacting with the body and discuss their advantages and challenges for immersive analytics.

As one of the most important non-verbal communication channel, touch plays an essential role in interpersonal affective communication. Although some researchers have started exploring the possibility of using wearable devices for conveying emotional information, most of the existing devices still lack the capability to support affective and dynamic touch in interaction. In this paper, we explore the effect of dynamic visual cues on the emotional perception of vibrotactile signals. For this purpose, we developed VisualTouch, a haptic sleeve consisting of a haptic layer and a visual layer. We hypothesized that visual cues would enhance the interpretation of tactile cues when both types of cues are congruent. We first carried out an experiment and selected 4 stimuli producing substantially different responses. Based on that, a second experiment was conducted with 12 participants rating the valence and arousal of 36 stimuli using SAM scales.

We propose a paradigm called Skin-On interfaces, in which interactive devices have their own (artificial) skin, thus enabling new forms of input gestures for end-users (e.g. twist, scratch). Our work explores the design space of Skin-On interfaces by following a bio-driven approach: (1) From a sensory point of view, we study how to reproduce the look and feel of the human skin through three user studies; (2) From a gestural point of view, we explore how gestures naturally performed on skin can be transposed to Skin-On interfaces; (3) From a technical point of view, we explore and discuss different ways of fabricating interfaces that mimic human skin sensitivity and can recognize the gestures observed in the previous study; (4) We assemble the insights of our three exploratory facets to implement a series of Skin-On interfaces and we also contribute by providing a toolkit that enables easy reproduction and fabrication.

In this paper, we explore the interaction space of MobiLimb, a small 5-DOF serial robotic manipulator attached to a mobile device. It (1) overcomes some limitations of mobile devices (static, passive, motionless); (2) preserves their form factor and I/O capabilities; (3) can be easily attached to or removed from the device; (4) offers additional I/O capabilities such as physical deformation and (5) can support various modular elements such as sensors, lights or shells. We illustrate its potential through three classes of applications: As a tool, MobiLimb offers tangible affordances and an expressive controller that can be manipulated to control virtual and physical objects. As a partner, it reacts expressively to users' actions to foster curiosity and engagement or assist users. As a medium, it provides rich haptic feedback such as strokes, pat and other tactile stimuli on the hand or the wrist to convey emotions during mediated multimodal communications.

Menus are used for exploring and selecting commands in interactive applications. They are widespread in current systems and used by a large variety of users. As a consequence, they have motivated many studies in Human-Computer Interaction (HCI). Facing the large variety of menus, it is difficult to have a clear understanding of the design possibilities and to ascertain their similarities and differences. In this article, we address a main challenge of menu design: the need to characterize the design space of menus. To do this, we propose a taxonomy of menu properties that structures existing work on visual menus. As properties have an impact on the performance of the menu, we start by refining performance through a list of quality criteria and by reviewing existing analytical and empirical methods for quality evaluation. This taxonomy of menu properties is a step toward the elaboration of advanced predictive models of menu performance and the optimization of menus. A key point of this work is to focus both on menus and on the properties of menus, and then enable a fine-grained analysis in terms of performance.

System schematics, such as those used for electrical or hydraulic systems, can be large and complex. Fisheye techniques can help navigate such large documents by maintaining the context around a focus region, but the distortion introduced by traditional fisheye techniques can impair the readability of the diagram. We present SchemeLens, a vector-based, topology-aware fisheye technique which aims to maintain the readability of the diagram. Vector-based scaling reduces distortion to components, but distorts layout. We present several strategies to reduce this distortion by using the structure of the topology, including orthogonality and alignment, and a model of user intention to foster smooth and predictable navigation. We evaluate this approach through two user studies: Results show that (1) SchemeLens is 16-27% faster than both round and rectangular flat-top fisheye lenses at finding and identifying a target along one or several paths in a network diagram; (2) augmenting SchemeLens with a model of user intentions aids in learning the network topology.

In this article we explore the design space of gesture shortcuts on touchpad with four novel interaction techniques: SpotPad and LociPad rely on one-finger static gestures, but differ in their graphical representation (Grid vs. loci). ChordPad relies on two-finger static gestures with a hierarchical representation. Finally, InOutPad relies on dynamic gestures crossing the edges of the touchpad. We present and compare the properties of these techniques of interaction and describe how they can be deployed on Mac OS X.

Locating an object in an unfamiliar and dense physical environment, such as a control room, supermarket, or warehouse, can be challenging. In this paper, we present the Projection-Augmented Arm (PAA), a motorized robotic arm augmented with a pico-projector to help users to localize targets in such environments. The arm moves and displays a projected spotlight on the target. We present the results of a study that shows that the PAA helps users to more quickly locate target objects in a dense environment. We further study the influence of the visibility of the projected spotlight while moving versus that of the physical movement of the projection arm on user performance and search strategy, finding that (1) information about the orientation of the arm has a stronger impact on performance than moving spotlight projected on the search space; (2) the orientation of the arm is useful (24 % improvement) and especially when the target is behind the user (26 % improvement); and (3) users’ strategies relied mainly on the arm when it is visible.

The usability of small devices such as smartphones or interactive watches is often hampered by the limited size of command vocabularies. This paper is an attempt at better understanding how finger identification may help users invoke commands on touch screens, even without recourse to multi-touch input. We describe how finger identification can increase the size of input vocabularies under the constraint of limited real estate, and we discuss some visual cues to communicate this novel modality to novice users. We report a controlled experiment that evaluated, over a large range of input-vocabulary sizes, the efficiency of single-touch command selections with vs. without finger identification. We analyzed the data not only in terms of traditional time and error metrics, but also in terms of a throughput measure based on Shannon's theory, which we show offers a synthetic and parsimonious account of users' performance. The results show that the larger the input vocabulary needed by the designer, the more promising the identification of individual fingers.

We present Bezel-Tap Gestures, a novel family of interaction techniques for immediate interaction on handheld tablets regardless of whether the device is alive or in sleep mode. The technique rests on the close succession of two input events: first a bezel tap, whose detection by accelerometers will awake an idle tablet almost instantly, then a screen contact. Field studies confirmed that the probability of this input sequence occurring by chance is very low, excluding the accidental activation concern. One experiment examined the optimal size of the vocabulary of commands for all four regions of the bezel (top, bottom, left, right). Another experiment evaluated two variants of the technique which both allow two-level selection in a hierarchy of commands, the initial bezel tap being followed by either two screen taps or a screen slide. The data suggests that Bezel-Tap Gestures may serve to design large vocabularies of micro-interactions with a sleeping tablet.

Interaction with TV sets, set-top boxes or media centers strongly differs from interaction with personal computers: not only does a typical remote control suffer strong form factor limitations but the user may well be slouching in a sofa. In the face of more and more data, features, and services made available on interactive televisions, we propose to exploit the new capabilities provided by gesture-aware remote controls. We report the data of three user studies that suggest some guidelines for the design of a gestural vocabulary and we propose five novel interaction techniques. Study 1 reports that users spontaneously perform pitch and yaw gestures as the first modality when interacting with a remote control. Study 2 indicates that users can accurately select up to 5 items with eyes-free roll gestures. Capitalizing on our findings, we designed five interaction techniques that use either device motion, or button-based interaction, or both. They all favor the transition from novice to expert usage for selecting favorites. Study 3 experimentally compares these techniques. It reveals that motion of the device in 3D space, associated with finger presses at the surface of the device, is achievable, fast and accurate. Finally, we discuss the integration of these techniques into a coherent multimedia menu system.