Fleets of drones are actively being explored for search and rescue (SAR) missions to accelerate the exploration of large areas and support the delivery of supplies. However, the dynamic and unpredictable nature of these operations makes the automated planning and control difficult. We conducted interviews with four SAR professionals to identify critical scenarios and interaction needs at different levels of control. We then designed a virtual reality prototype allowing the planning, supervision, and manual takeover of a drone fleet during search missions. An exploratory study with five drone pilots evaluated the proposed interactions and helped identify design opportunities for such systems. Our results highlight the importance of enabling smooth manual takeover and the challenges of monitoring and controlling the fleet simultaneously. We discuss implications for the design of future SAR multi-drone systems.
In situated visualization, digital data is linked to physical referents in the user's environment. However, when the environment contains multiple physical referents (e.g., buildings in an urban scale model), some may be occluded by others, complicating the user's ability to point and select them. This issue is well-known in VR, and the literature offers various techniques of occlusion reduction to support pointing at the initially occluded virtual objects in VR. These techniques can be organized into three main categories: moving the objects to reveal them; turning the occluder transparent to reveal the objects behind it; using an additional view to reveal the occluded objects. However, it is uncertain whether these techniques are well-suited to reveal occluded physical referents in AR environments. In this paper, we propose occlusion reduction techniques dedicated to AR, i.e., used for revealing occluded physical referents in AR and allowing their selection. To this end, we apply and evaluate the three types of VR techniques in an AR context. The results suggest the need for improvements to better fit the physical context. We therefore propose three future works based on the results of the study.
Users use only a limited subset of software commands. This article introduces a a novel approach to discovering commands in graphical user interfaces. Unlike previous approaches to recommend a command, which relies on command prediction - determining what the user wants to do in the future - our approach relies on command postdiction - explaining how a task in the past could have been performed more efficiently. Given the current state of the interface and previous states, our approach infers the command that would have been most appropriate and suggests it to the users. We discuss three case studies: shape, image and text editors. For each of them, we study the feasibility of the approach through proofs of concept. Finally, we discuss the challenges posed by deplying post-diction in existing software.
Constraint satisfaction problems (CSPs), whether they involve scheduling, deliveries routing or allocating resources, are ubiquitous in our daily lives. These problems often involve a large number of variables and constraints, making them complex to perceive. At present, they are mainly designed and manipulated via computer code, which adds to this difficulty of appropriation. We believe that a tangible interface (TUI) could facilitate their design and/or manipulation. To explore this hypothesis, we carried out an initial systematic literature review focusing on tangible representations of generic CSPs. However, this revealed the absence of such representations for problems that are not specific to a particular application. As a result, the challenge of realizing a tangible interface representing a generic CSP is unexplored. Consequently, we conducted a second systematic literature review, focusing on tangible interfaces for specialized CSP applications (scheduling, routing, etc.). We extracted from this review forteen articles, which we listed, described, analyzed and compared.
Mixed Reality (MR) is increasingly being studied for synchronous remote collaboration in various fields, from industrial maintenance to education. It allows to combine virtual reality, augmented reality, and 2D interfaces to visualize and share real and virtual content. The notion of coupling of collaborative activities, central to studies of Computer-Supported Cooperative Work (CSCW) applications, takes various forms with MR due to its 3D and immersive characteristics, as well as the heterogeneity of interactive technologies. To cope with this diversity, we propose a graphical notation, PV*, that precisely describes the coupling relationships between multiple viewpoints on physical or virtual views, as well as their control by collaborators. We study the descriptive power of PV* by describing several collaborative situations from the literature and discuss its generative power by exploring several design possibilities described with PV*.
