Given a polygonal workspace W, a depth sensor placed at point p=(x,y) inside W and oriented in direction θ measures the distance d=h(x,y,θ) between p and the closest point on the boundary of W along a ray emanating from p in direction θ. We study the following problem: For a polygon W with n vertices, possibly with holes, preprocess it such that given a query real value d> 0, one can efficiently compute the preimage h^-1(d) ⊂ W×𝕊^1, namely determine all the possible poses (positions and orientations) of a depth sensor placed in W that would yield the reading d, in an output-sensitive fashion. We describe such an output-sensitive data structure, which answers queries in O(k log n) time, where k is the number of vertices and maximal arcs of low degree algebraic curves constituting the answer. We also obtain analogous results for the more useful case (narrowing down the set of possible poses), where the sensor performs two antipodal depth measurements from the same point in W. We then describe simpler data structures for the same two problems, where we employ a decomposition of W×𝕊^1, and where the query time is output-sensitive relative to this decomposition. Our software implementation for these latter structures is open source and publicly available. Although robot localization is often carried out by exploring the full visibility polygon of a sensor placed at a point of the environment, the approach that we propose here opens the door to sufficing with only few depth measurements, which is advantageous as it allows for usage of inexpensive sensors and could also lead to savings in storage and communication costs.

Whisker-like touch sensors offer unique advantages for short-range perception in environments where visual and long-range sensing are unreliable, such as confined, cluttered, or low-visibility settings. This paper presents a framework for estimating contact points and robot localization in a known planar environment using a single whisker sensor. We develop a family of virtual sensor models. Each model maps robot configurations to sensor observations and enables structured reasoning through the concept of preimages - the set of robot states consistent with a given observation. The notion of virtual sensor models serves as an abstraction to reason about state uncertainty without dependence on physical implementation. By combining sensor observations with a motion model, we estimate the contact point. Iterative estimation then enables reconstruction of obstacle boundaries. Furthermore, intersecting states inferred from current observations with forward-projected states from previous steps allow accurate robot localization without relying on vision or external systems. The framework supports both deterministic and possibilistic formulations and is validated through simulation and physical experiments using a low-cost, 3D printed, Hall-effect-based whisker sensor. Results demonstrate accurate contact estimation and localization with errors under 7 mm, demonstrating the potential of whisker-based sensing as a lightweight, adaptable complement to vision-based navigation.

This paper bridges some of the gap between optimal planning and reinforcement learning (RL), both of which share roots in dynamic programming applied to sequential decision making or optimal control. Whereas planning typically favors deterministic models, goal termination, and cost minimization, RL tends to favor stochastic models, infinite-horizon discounting, and reward maximization in addition to learning-related parameters such as the learning rate and greediness factor. A derandomized version of RL is developed, analyzed, and implemented to yield performance comparisons with value iteration and Dijkstra's algorithm using simple planning models. Next, mathematical analysis shows: 1) conditions under which cost minimization and reward maximization are equivalent, 2) conditions for equivalence of single-shot goal termination and infinite-horizon episodic learning, and 3) conditions under which discounting causes goal achievement to fail. The paper then advocates for defining and optimizing truecost, rather than inserting arbitrary parameters to guide operations. Performance studies are then extended to the stochastic case, using planning-oriented criteria and comparing value iteration to RL with learning rates and greediness factors.

Feedback motion planning over cell decompositions provides a robust method for generating collision-free robot motion with formal guarantees. However, existing algorithms often produce paths with unnecessary bending, leading to slower motion and higher control effort. This paper presents a computationally efficient method to mitigate this issue for a given simplicial decomposition. A heuristic is introduced that systematically aligns and assigns local vector fields to produce more direct trajectories, complemented by a novel geometric algorithm that constructs a maximal star-shaped chain of simplexes around the goal. This creates a large & quot;funnel & quot; in which an optimal, direct-to-goal control law can be safely applied. Simulations demonstrate that our method generates measurably more direct paths, reducing total bending by an average of 91.40% and LQR control effort by an average of 45.47%. Furthermore, comparative analysis against sampling-based and optimization-based planners confirms the time efficacy and robustness of our approach. While the proposed algorithms work over any finite-dimensional simplicial complex embedded in the collision-free subset of the configuration space, the practical application focuses on low-dimensional ($d\leq 3$) configuration spaces, where simplicial decomposition is computationally tractable.

This paper presents a framework for safe navigation of a unicycle point robot to a goal position in an environment populated with obstacles from almost any admissible state, considering input limits. We introduce a novel QP formulation to create a Cinfinity-smooth vector field with reduced total bending and total turning. Then we design an analytic, non-linear feedback controller that inherently satisfies the conditions of Nagumo's theorem, ensuring forward invariance of the safe set without requiring any online optimization. We have demonstrated that our controller, even under hard input limits, safely converges to the goal position. Simulations confirm the effectiveness of the proposed framework, resulting in a twice faster arrival time with over 50% lower angular control effort compared to the baseline.

In this paper, we consider robotic tasks which require a desirable outcome to be achieved in the physical world that the robot is embedded in and interacting with. Accomplishing this objective requires designing a filter that maintains a useful representation of the physical world and a policy over the filter states. A filter is seen as the robot's perspective of the physical world based on limited sensing, memory, and computation and it is represented as a transition system over a space of information states. To this end, the interactions result from the coupling of an internal and an external system, a filter, and the physical world, respectively, through a sensor mapping and an information-feedback policy. Within this setup, we look for sufficient structures, that is, sufficient internal systems and sensors, for accomplishing a given task. We establish necessary and sufficient conditions for these structures to satisfy for information-feedback policies that can be defined over the states of an internal system to exist. We also show that under mild assumptions, minimal internal systems that can represent a particular plan/policy described over the action-observation histories exist and are unique. Finally, the results are applied to determine sufficient structures for distance-optimal navigation in a polygonal environment.

Telepresence robots enable users to interact with remote environments, but efficient and intuitive navigation remains a challenge. In this work, we developed and evaluated a shared control method, in which the robot navigates autonomously while allowing users to affect the path generation to better suit their needs. We compared this with control switching, where users toggle between direct and automated control. We hypothesized that shared control would maintain efficiency comparable to control switching while potentially reducing user workload. The results of two consecutive user studies (each with final sample of n=20) showed that shared control does not degrade navigation efficiency, but did not show a significant reduction in task load compared to control switching. Further research is needed to explore the underlying factors that influence user preference and performance in these control systems.

Immersive telepresence, when a user views the video stream of a 360^∘ camera in a remote environment using a Head Mounted Display (HMD), has great potential to improve the sense of being in a remote environment. In most cases of immersive robotic telepresence, the camera is mounted on a mobile robot which increases the portion of the environment that the remote user can explore. However, robot motions can induce unpleasant symptoms associated with Virtual Reality (VR) sickness, degrading the overall user experience. Previous research has shown that unwinding the rotations of the robot, that is, decoupling the rotations that the camera undergoes due to robot motions from what is seen by the user, can increase user comfort and reduce VR sickness. However, that work considered a virtual environment and a simulated robot. In this work, to test whether the same hypotheses hold when the video stream from a real camera is used, we carried out a user study (n=36) in which the unwinding rotations method was compared against coupled rotations in a task completed through a panoramic camera mounted on a robotic arm. Furthermore, within an inspection task which involved translations and rotations in three dimensions, we tested whether unwinding the robot rotations impacted the performance of users. The results show that the users found the unwinding rotations method to be more comfortable and preferable, and that a reduced level of VR sickness can be achieved without a significant impact on task performance.

Telepresence robots offer the promise of remote presence, but user experience, usability, and performance challenges hinder widespread adoption. This study introduces a novel and low-cost user interface for telepresence robots that integrates insights from virtual reality (VR) and robotics to address these limitations. The novel setup was designed holistically, considering several different factors: an inclined rotating chair for embodied rotation, a joystick for precise translation, dual displays for enhanced spatial awareness, and an immersive setup with controlled lighting and audio. A user study (N = 42) with a simulated robot in a virtual environment compared this novel setup with a standard setup, that mimicked the typical user interface of commercial telepresence robots. Results showed that this novel setup significantly improved the user experience, particularly increasing presence, enjoyment, and engagement. This novel setup also improved task performance over time, reducing obstacle collisions and distance traveled. These findings highlight the potential for combining and incorporating insights from VR and robotics to design more effective and user-friendly interfaces for telepresence robots, paving the way for increased adoption.

Cybersickness poses a serious challenge for users of virtual reality (VR) technology. Consequently, there has been significant effort to track its occurrence during VR use with passive measures like brain activity recorded through electroencephalogram (EEG). To classify cybersickness accurately, including in real time, machine learning algorithms which can extract meaningful signals from the rest of the brain data will be required. However, EEG datasets are typically very small and very high in variability between participants, which makes building effective models extremely challenging. To address these concerns, we first introduce a framework for neural networks which has subject-adaptive training with calibration and interpretation for classification given limited and imbalanced EEG data. Which features the models determine are most useful can be visualized by plotting interpretability maps from integrated gradients and class activation. The framework is demonstrated here with convolutional neural networks and transformer models. Using a set of brain data recorded with EEG while participants viewed a stimulus in VR designed to elicit cybersickness, we show which spatio-temporal EEG features (from electrodes and time steps) were most important for discomfort classification. Across 12 runs of our framework with three different neural networks over multiple random seeds, the models consistently pointed to the same scalp locations as having patterns of brain data that were the most helpful in determining whether or not a sample of EEG data belonged to someone who was experiencing cybersickness. These results help clarify a hidden pattern in other related research and can be used as tagged features for better real-time cybersickness classification with EEG in the future. We provide our code at [anonymized] to enable feature interpretation across different neural network architectures.

Presence in virtual reality (VR), the subjective sense of "being there" in a virtual environment, is notoriously difficult to measure. Electroencephalography (EEG) may offer a promising, unobtrusive means of assessing a user's momentary state of presence. Unlike traditional questionnaires, EEG does not interrupt the experience or rely on users' retrospective self-reports, thereby avoiding interference with the very state it aims to capture. Previous research has attempted to quantify presence in virtual environments using event-related potentials (ERPs). We contend, however, that previous efforts have fallen short of fully realizing this goal, failing to either A) independently manipulate presence, B) validate their measure of presence against traditional techniques, C) adequately separate the constructs of presence and attention, and/or D) implement a realistic and immersive environment and task. We address these shortcomings in a preregistered ERP experiment in which participants play an engaging target shooting game in VR. ERPs are time-locked to the release of a ball from a sling. We induce breaks in presence (BIPs) by freezing the ball's release on a minority of trials. Embodiment is manipulated by allowing manual manipulation of the sling with a realistic avatar in one condition (embodied condition) and passive manipulation with only controllers in another (non-embodied condition). We support our predictions that the N2, the P3b, and the N400, are selectively sensitive towards specific components of these manipulations. The pattern of findings carries significant implications for theories of presence, which have been seldom addressed in previous ERP investigations on this topic.

This paper presents a novel problem of creating and regulating localization and navigation illusions considering two agents: a receiver and a producer. A receiver is moving on a plane localizing itself using the intensity of signals from three known towers observed at its position. Based on this position estimate, it follows a simple policy to reach its goal. The key idea is that a producer alters the signal intensities to alter the position estimate of the receiver while ensuring it reaches a different destination with the belief that it reached its goal. We provide a precise mathematical formulation of this problem and show that it allows standard techniques from control theory to be applied to generate localization and navigation illusions that result in a desired receiver behavior.

Mobile telepresence robots allow users to feel present and explore remote environments using technology. Traditionally, these systems are implemented using a camera onboard a mobile robot that can be controlled. Although high-immersion technologies, such as 360-degree cameras, can increase situational awareness and presence, they also introduce significant challenges. Additional processing and bandwidth requirements often result in latencies of up to seconds. The current delay with a 360-degree camera streaming over the internet makes real-time control of these systems difficult. Working with high-latency systems requires some form of assistance to the users. This study presents a novel way to utilize optical flow to create an illusion of self-motion to the user during the latency period between user sending motion commands to the robot and seeing the actual motion through the 360-camera stream. We find no significant benefit of using the self-motion illusion to performance or accuracy of controlling a telepresence robot with a latency of 500 ms, as measured by the task completion time and collisions into objects. Some evidence is shown that the method might increase virtual reality (VR) sickness, as measured by the simulator sickness questionnaire (SSQ). We conclude that further adjustments are necessary in order to render the method viable.

Rodents and Felidae whiskers are highly sensitive, detecting extrinsic inputs such as airflow or contact and intrinsic inputs such as base vibrations or self-induced motion. Building effective artificial whisker sensors faces a challenge due to the intricate coupling of responses at the whisker base. There is a research gap in understanding whisker sensors' responses to intrinsic and extrinsic inputs. To address this, we propose two methods, using base acceleration as a reference input: 1) employing frequency-domain adaptive filtering (FDAF) and 2) introducing the base vibration response model (BVRM) that mathematically represents the whisker sensor's behavior to base vibrations or self-induced motion. Validation of FDAF and BVRM is conducted through simulation and experimentation. The BVRM excels in both simulation and experiment, demonstrating a signal-to-noise ratio (SNR) of 35.20, slightly outperforming the laboriously tuned partitioned constrained FDAF with an SNR of 34.96, despite FDAFs slower convergence and poorer performance in experiments. In addition, BVRM can be useful in filtering sensor responses for independent use cases, such as terrain identification, flow sensing, and surface profile identification. By separating responses to extrinsic and intrinsic inputs without discarding either, whisker sensors become more versatile and multipurpose.

Immersive mobile robotic telepresence enables humans to feel present in a remote environment. These systems often use 360-degree panoramic cameras to stream video over a network to a head-mounted display (HMD) where the video feed is rendered to the user. This enables the user to freely look around in a remote environment. A drawback of using highly immersive technologies instead of a more traditional computer screen is that users often experience virtual reality (VR) sickness. Therefore, sometimes the users are only able to use these systems for brief durations. Moreover, the increase in bandwidth requirements of panoramic cameras and the time necessary to process the 360-degree panoramic view contributes to an often unacceptable amount of latency between the user’s actions and the observed reaction of the mobile robot, which can be referred to as perception-actuation loop. We present a novel method to mitigate these problems in immersive mobile robotic telepresence systems. We call this method virtual environment as an interface to a physical environment (VEIPE). In VEIPE, a digital twin of the remote environment is used to interface with the telepresence robot in the real remote environment. We present a study comparing teleportation through VEIPE as a locomotion method against a more traditional joystick-based continuous locomotion method for controlling a telepresence robot. Our results indicate that VEIPE induces less VR sickness compared to the joystick condition as measured by the simulator sickness questionnaire (SSQ) and users perform about 31 percent better in a simple navigation task. Furthermore, the users subjectively prefer teleportation through VEIPE over the joystick. We also present exploratory data about cognitive load measured with the NASA task-load-index (NASA-TLX) questionnaire, presence measured with the Slater-Usoh-Steed (SUS) questionnaire, and accumulated yaw in the navigation tasks.

This paper formally defines a robot system, including its sensing and actuation components, as a general, topological dynamical system. The focus is on determining general conditions under which various environments in which the robot can be placed are indistinguishable. A key result is that, under very general conditions, covering maps witness such indistinguishability. This formalizes the intuition behind the well studied loop closure problem in robotics. An important special case is where the sensor mapping reports an invariant of the local topological (metric) structure of an environment because such structure is preserved by (metric) covering maps. Whereas coverings provide a sufficient condition for the equivalence of environments, we also give a necessary condition using bisimulation. The overall framework is applied to unify previously identified phenomena in robotics and related fields, in which moving agents with sensors must make inferences about their environments based on limited data. Many open problems are identified.

According to previous research, humans are exceptionally skilled at distinguishing earth-congruent object trajectories from earth-discrepant ones. Inconsistent scale cues in Virtual Reality (VR), however, are shown to easily confuse VR users’ scale, size, and distance estimations. This paper describes the results of two studies, with 40 participants each, which tested the effects of self-scaling, and size cues from active virtual characters, on perceived physics coherence (i.e., subjective realism). In the first study (Study A), a humanoid robot the size of a doll manipulated household objects to demonstrate their rigid body dynamics. The demonstration sequence was performed twice, once using a correct approximation of physics, and once using an incorrect one in which gravity was simulated similarly to as if the doll-sized robot was human-sized and the surrounding room was enlarged. The participants observed both demonstrations, once while standing at the normal scale and once at a reduced scale, similar to that of the humanoid robot. The second study (Study B) was similar, except that the virtual character demonstrating the physics was a regular-sized cat. Our preregistered hypotheses predicted that participants would consider the correct approximation of physics as better matching their expectations at normal scale, and the incorrect one as better matching their expectations at the reduced scale. However, only the second of these hypotheses was supported. According to our exploratory analyses, the participant’s own scale was a poor predictor of physics preference, and instead there was a significant effect regarding the virtual character’s identity. Participants observing the virtual cat were about eight times more likely to select the realistic physics model compared to those observing the humanoid robot. The results indicate that familiar cues tied to virtual character identity overrode any potential effects related to changes in the participants’ own scale.

Children with attention and executive function disabilities often have a long-lasting need for rehabilitation to support their functional ability. Yet the availability of rehabilitation services is insufficient, regionally unevenly distributed, and unequal in terms of access to rehabilitation. There is a need for easily accessible services. In this paper, we present the VREALFUN project where the major aim is to develop a novel Virtual Reality (VR) rehabilitation method for children with deficits in attention and executive functions. This ongoing Randomized Control Study (RCT) includes two arms, one in children with attention deficit hyperactivity disorder (ADHD) and the other in children with mild to moderate traumatic brain injury (TBI).

This paper makes the case that a powerful new discipline, which we term perception engineering, is steadily emerging. It follows from a progression of ideas that involve creating illusions, from historical paintings and film, to video games and virtual reality in modern times. Rather than creating physical artifacts such as bridges, airplanes, or computers, perception engineers create illusory perceptual experiences. The scope is defined over any agent that interacts with the physical world, including both biological organisms (humans, animals) and engineered systems (robots, autonomous systems). The key idea is that an agent, called a producer, alters the environment with the intent to alter the perceptual experience of another agent, called a receiver. Most importantly, the paper introduces a precise mathematical formulation of this process, based on the von Neumann-Morgenstern notion of information, to help scope and define the discipline. It is then applied to the cases of engineered and biological agents with discussion of its implications on existing fields such as virtual reality, robotics, and even social media. Finally, open challenges and opportunities for involvement are identified.

A user's perceived height can have a significant impact on their experience in an immersive telepresence environment. However, virtually manipulating the user's height (if physical adjustment of the camera is not possible) introduces distortions which may counteract positive effects caused by an adjusted height. In a user study of 68 participants, we implemented a simple method for virtually adjusting a user's height in an immersive telepresence meeting which was prerecorded via a 360 degrees camera to observe the trade-off between the height shift and its ensuing distortions. The shifted-height condition was created via software by changing the position of the virtual camera within the 3-D projection sphere, a simple technique which introduces mild visual distortions. Participants were asked to attend two meetings in immersive telepresence, at normal and increased heights. Our results indicate that while participants were able to detect the visual distortions at an above chance rate, these distortions had little influence on the participants' preferences between conditions, supporting this technique as a viable method of virtually altering a user's height in immersive telepresence environments.