Compliance is essential for dexterous manipulation, yet existing solutions often rely on external tactile or force sensors that are costly, fragile, and difficult to deploy on low-cost robot hands. We propose a proprioception-driven framework that learns contact-aware compliance cues from motor current and joint states. Since motor current is closely related to actuator torque, it provides an intrinsic signal for perceiving contact force, object resistance, and grasp stability without additional sensing hardware. Rather than estimating external wrenches or commanding torque, our method predicts a compliance reference position: an ideal joint-position target for a standard PD controller whose induced position error generates appropriate grasping force. This position-based formulation is compatible with mainstream teleoperation and policy-learning pipelines, while enabling the robot to adapt interaction forces from real-time proprioceptive feedback. Thus, motor current serves not only as a force proxy but also as a learnable proprioceptive contact signal for compliance reference prediction. Experiments on multiple dexterous hands and contact-rich tasks, including fragile object handling, sustained surface contact, thin-object retrieval, and dynamic load adaptation, show stable compliant grasping, safer and more efficient teleoperation, and improved downstream policy learning without external tactile or force sensors.

IntroductionExtended exposure to a microgravity environment has been related to cognitive and neural decrements in astronauts, including changes in brain morphology and connectivity. Future long-duration exploration missions, such as those to Mars, will require the development of new countermeasures to counteract these decrements. Training in virtual reality (VR) has been identified as a promising potential countermeasure. Though there has been extensive research into VR as a tool for neurorehabilitation and microcognitive testing, little is understood about the neural effects of training in VR for operationally relevant tasks.MethodsThis research utilized functional near infrared spectroscopy (fNIRS) and electroencepholography (EEG) to measure neural activation during task completion in a VR environment with spaceflight-relevant tasks as compared to one with microcognitive corollary tasks.ResultsWe find that a complex, operationally relevant VR environment elicits enhanced brain activation compared to the matching corollary tasks designed to target equivalent specific cognitive domains, for both EEG (p < 0.0005) and fNIRS (p < 0.001).DiscussionThese results indicate that brain activation and recruitment is increased when the task has higher ecological validity, providing an objective assessment to inform countermeasure development for spaceflight associated neural decrements.

In this study, we examined whether current 3D reconstruction methods can support the creation of realistic holographic representations of laboratory objects for educational use. In this regard, we compared four approaches: photogrammetry, a neural radiance field (NeRF)-based method, Gaussian splatting, and LiDAR. These methods were used to generate holographic models of common laboratory items and their fidelity was evaluated by graduate students. Participants assessed the models for shape, color, texture, and visual defects using a repeated-measures design. Across objects, the NeRF-based method produced the most consistently high-fidelity representations, particularly for transparent, reflective, or low-texture items that were difficult to capture with other approaches. Shape and color were generally reproduced more successfully than texture, suggesting that some visual properties remain more challenging to represent accurately in educational holograms. Beyond identifying the strengths and limitations of each reconstruction method, the study demonstrates a practical workflow for creating immersive learning objects that may support pre-laboratory preparation, spatial reasoning, and student engagement in AR/MR-based educational environments. These findings offer design-relevant insights for educators and researchers developing immersive digital learning experiences.

Scientists perform diverse manual procedures that are tedious and laborious. Such procedures are considered a bottleneck for modern experimental science, as they consume time and increase burdens in fields including material science and medicine. We employ a user-centered approach to designing a robot-assisted system for dialysis, a common multi-day purification method used in polymer and protein synthesis. Through two usability studies, we obtain participant feedback and revise design requirements to develop the final system that satisfies scientists' needs and has the potential for applications in other experimental workflows. We anticipate that integration of this system into real synthesis procedures in a chemical wet lab will decrease workload on scientists during long experimental procedures and provide an effective approach to designing more systems that have the potential to accelerate scientific discovery and liberate scientists from tedious labor.

Vision-language-action (VLA) models present a promising paradigm by training policies directly on real robot datasets like Open X-Embodiment. However, the high cost of real-world data collection hinders further data scaling, thereby restricting the generalizability of VLAs. In this paper, we introduce ReBot, a novel real-to-sim-to-real approach for scaling real robot datasets and adapting VLA models to target domains, which is the last-mile deployment challenge in robot manipulation. Specifically, ReBot replays real-world robot trajectories in simulation to diversify manipulated objects (real-to-sim), and integrates the simulated movements with inpainted real-world background to synthesize physically realistic and temporally consistent robot videos (sim-to-real). Our approach has several advantages: 1) it enjoys the benefit of real data to minimize the sim-to-real gap; 2) it leverages the scalability of simulation; and 3) it can generalize a pretrained VLA to a target domain with fully automated data pipelines. Extensive experiments in both simulation and real-world environments show that ReBot significantly enhances the performance and robustness of VLAs. For example, in SimplerEnv with the WidowX robot, ReBot improved the in-domain performance of Octo by 7.2% and OpenVLA by 21.8%, and out-of-domain generalization by 19.9% and 9.4%, respectively. For real-world evaluation with a Franka robot, ReBot increased the success rates of Octo by 17% and OpenVLA by 20%. More information can be found at our project page.

Augmented reality (AR) offers promising opportunities to support movement-based activities, such as personal training or physical therapy, with real-time, spatially-situated visual cues. While many approaches leverage AR to guide motion, existing design guidelines focus on simple, upper-body movements within the user's field of view. We lack evidence-based design recommendations for guiding more diverse scenarios involving movements with varying levels of visibility and direction. We conducted an experiment to investigate how different visual encodings and perspectives affect motion guidance performance and usability, using three exercises that varied in visibility and planes of motion. Our findings reveal significant differences in preference and performance across designs. Notably, the best perspective varied depending on motion visibility and showing more information about the overall motion did not necessarily improve motion execution. We provide empirically-grounded guidelines for designing immersive, interactive visualizations for motion guidance to support more effective AR systems.

Robotic systems that can traverse planetary or lunar surfaces to collect environmental data and perform physical manipulation tasks, such as assembling equipment or conducting mining operations, are envisioned to form the backbone of future human activities in space. However, the environmental conditions in which these robots, or "rovers," operate present challenges towards achieving fully autonomous solutions, meaning that rover missions will require some degree of human teleoperation or supervision for the foreseeable future. As a result, human operators require training to successfully direct rovers and avoid costly errors or mission failures, as well as the ability to recover from any issues that arise on-the-fly during mission activities. While analog environments, such as JPL's Mars Yard, can help with such training by simulating surface environments in the real world, access to such resources may be rare and expensive. As an alternative or supplement to such physical analogs, we explore the design and evaluation of a virtual reality digital twin system to train human teleoperation of robotic rovers with mechanical arms for space mission activities. We conducted an experiment with 24 human operators to investigate how our digital twin system can support human teleoperation of rovers in both pre-mission training and in real-time problem solving in a mock lunar mission in which users directed a physical rover in the context of deploying dipole radio antennas. We found that operators who first trained with the digital twin showed a 28% decrease in mission completion time, an 85% decrease in unrecoverable errors, as well as improved mental markers, including decreased cognitive load and increased situation awareness. (c) 2025 COSPAR. Published by Elsevier B.V. All rights are reserved, including those for text and data mining, AI training, and similar technologies.

Calibrating users to the capabilities and limitations of their autonomous robotic partners is crucial for successful human-robot teaming. User inexperience, mission complexities, and degradation of the robot over time can impact user decision-making and lead to ineffective use of the robot. This can spell disaster in high-risk and uncertain environments like space exploration. In this paper, we explore how to mitigate such risks by enabling robots to quantify and communicate estimates of their competencies to human partners. Specifically, we investigated robot competency information in the context of a complex planetary exploration analogue mission in which a human had to balance monitoring an autonomous mobile robot with attending to a concurrent task. Leveraging the Factorized Machine Self-Confidence framework for robot competency self-assessment, we varied the robot’s ability to report competency. We found that human-robot teams whose robot was able to report competency and update that report online demonstrated superior mission performance (30% improvement) compared to baseline methods.

Future concepts for space exploration envision astronauts relying on autonomous robots to accomplish a variety of tasks, including planetary surface exploration, science, and mining operations. An astronaut’s understanding of the robots’ competencies will be crucial to informed, risk-aware decision-making during both mission planning and mission execution. A key challenge is to develop methods to calibrate future astronauts (or more generally users of autonomous robots in safety critical operations) to the capabilities and limitations of these machines. In the spring of 2024, we partnered with the Mars Society’s Mars Desert Research Station (MDRS) Crew #297 to conduct a two-week exploratory field deployment and limited user study evaluating an autonomous ground robot with a competency-awareness capability—autonomy algorithms capable of quantifying and communicating estimates of the robot’s ability to meet mission objectives. During the analog Mars expedition, the crew utilized a robot to explore areas and gather data during Extravehicular Activities. They overcame several technical challenges and were able to safely and efficiently leverage the robot for their mission needs. The competency information reported by the robot served as a valuable component of the crew’s planning and decision-making process, with 50% of the crew indicating that they directly relied on the assessments while planning tasks for the robot. We believe that integrating competency assessment capabilities into robotic autonomy can enable future astronauts to better manage risk and make more informed decisions while tasking and supervising autonomous robots.

Lab assignments, in which students build and program robots to accomplish tasks in various environments, are a central component in many undergraduate robotics classes. Such activities require that students operationalize concepts learned in class. However, physical robots are prone to uncertain real-world behavior, making debugging challenging and causing many students to feel stressed about being graded based on their robot's performance. Therefore, we incorporated retakes into our undergraduate robotics class, allowing students to learn from their mistakes and master class content while improving their robots. Initial results show that students widely embrace retakes, use the opportunity to improve, and feel less stressed about the assignments.

Introduction: Human-robot teams are being called upon to accomplish increasingly complex tasks. During execution, the robot may operate at different levels of autonomy (LOAs), ranging from full robotic autonomy to full human control. For any number of reasons, such as changes in the robot's surroundings due to the complexities of operating in dynamic and uncertain environments, degradation and damage to the robot platform, or changes in tasking, adjusting the LOA during operations may be necessary to achieve desired mission outcomes. Thus, a critical challenge is understanding when and how the autonomy should be adjusted. Methods: We frame this problem with respect to the robot's capabilities and limitations, known as robot competency. With this framing, a robot could be granted a level of autonomy in line with its ability to operate with a high degree of competence. First, we propose a Model Quality Assessment metric, which indicates how (un)expected an autonomous robot's observations are compared to its model predictions. Next, we present an Event-Triggered Generalized Outcome Assessment (ET-GOA) algorithm that uses changes in the Model Quality Assessment above a threshold to selectively execute and report a high-level assessment of the robot's competency. We validated the Model Quality Assessment metric and the ET-GOA algorithm in both simulated and live robot navigation scenarios. Results: Our experiments found that the Model Quality Assessment was able to respond to unexpected observations. Additionally, our validation of the full ET-GOA algorithm explored how the computational cost and accuracy of the algorithm was impacted across several Model Quality triggering thresholds and with differing amounts of state perturbations. Discussion: Our experimental results combined with a human-in-the-loop demonstration show that Event-Triggered Generalized Outcome Assessment algorithm can facilitate informed autonomy-adjustment decisions based on a robot's task competency.

Humans working with autonomous artificially intelligent systems may not be experts in the inner workings of their machine teammates, but need to understand when to employ, trust, and rely on the system. A critical challenge is to develop machine agents with the capacity to understand their own capabilities and limitations, and the ability to communicate this information to human partners. Self-assessment is an emerging field that tackles this challenge through the development of algorithms that enable autonomous agents to understand and communicate their competency. These methods can engender appropriate trust and align human expectations with autonomous assistant abilities. However, current research in self-assessment is dispersed across many fields, including artificial intelligence, robotics, and human factors. This survey connects work from these disparate areas and reviews state-of-the-art methods for algorithmic self-assessments that enable autonomous agents to estimate, understand, and communicate valuable information pertaining to their competency, with focus on methods that can improve interactions within human-machine teams. To better understand the landscape of self-assessment approaches, we present a framework for categorizing work in self-assessment based on underlying algorithm type: test-based , learning-based , or knowledge-based . We synthesize common features across these approaches and discuss relevant future directions for research in this emerging space.

In teleoperation of redundant robotic manipulators, translating an operator’s end effector motion command to joint space can be a tool for maintaining feasible and precise robot motion. Through optimizing redundancy resolution, the control system can ensure the end effector maintains maneuverability by avoiding joint limits and kinematic singularities. In autonomous motion planning, this optimization can be done over an entire trajectory to improve performance over local optimization. However, teleoperation involves a human-in-the-loop who determines the trajectory to be executed through a dynamic sequence of motion commands. We present two systems, PrediKCT and PrediKCS, for utilizing a predictive model of operator commands in order to accomplish this redundancy resolution in a manner that considers future expected motion during teleoperation. Using a probabilistic model of operator commands allows optimization over an expected trajectory of future motion rather than consideration of local motion alone. Evaluation through a user study demonstrates improved control outcomes from this predictive redundancy resolution over minimum joint velocity solutions and inverse kinematics-based motion controllers.

The field of end-user robot programming seeks to develop methods that empower non-expert programmers to task and modify robot operations. In doing so, researchers may enhance robot flexibility and broaden the scope of robot deployments into the real world. We introduce PRogramAR (Programming Robots using Augmented Reality), a novel end-user robot programming system that combines the intuitive visual feedback of augmented reality (AR) with the simplistic and responsive paradigm of trigger-action programming (TAP) to facilitate human-robot collaboration. Through PRogramAR, users are able to rapidly author task rules and desired reactive robot behaviors, while specifying task constraints and observing program feedback contextualized directly in the real world. PRogramAR provides feedback by simulating the robot’s intended behavior and providing instant evaluation of TAP rule executability to help end-users better understand and debug their programs during development. In a system validation, 17 end-users ranging from ages 18 to 83 used PRogramAR to program a robot to assist them in completing three collaborative tasks. Our results demonstrate how merging the benefits of AR and TAP using elements from prior robot programming research into a single novel system can successfully enhance the robot programming process for non-expert users.

In this work, we present the design and evaluation of an immersive Cyber-Physical Control Room interface for remote mobile robots that provides users with both robot-egocentric and robot-exocentric 3D perspectives. We evaluate the Cyber-Physical Control room against a traditional robot interface in a mock disaster response scenario that features a mixed human-robot field team. In our evaluation, we found that the Cyber-Physical Control Room improved robot operator effectiveness by 28% while navigating a complex warehouse environment and performing a visual search. The Cyber-Physical Control Room also enhanced various aspects of human-robot teaming, including social engagement, the ability of a remote robot teleoperator to track their human partner in the field, and opinions of human teammate leadership qualities.

We envision a future in which telepresence is available to users anytime and anywhere, enabled by sensors and displays embedded in accessories worn daily, such as watches, jewelry, belt buckles, shoes, and eyeglasses. We present a collaborative approach to 3D reconstruction that combines a set of IMUs worn by a target person with an external view from another nearby person wearing an AR headset, used for estimating the target person's body pose and reconstructing their appearance, respectively.

Virtual, Augmented, and Mixed Reality for Human-Robot Interaction (VAM-HRI) has been gaining considerable attention in HRI research in recent years. However, the HRI community lacks a set of shared terminology and framework for characterizing aspects of mixed reality interfaces, presenting serious problems for future research. Therefore, it is important to have a common set of terms and concepts that can be used to precisely describe and organize the diverse array of work being done within the field. In this article, we present a novel taxonomic framework for different types of VAM-HRI interfaces, composed of four main categories of virtual design elements (VDEs). We present and justify our taxonomy and explain how its elements have been developed over the past 30 years as well as the current directions VAM-HRI is headed in the coming decade.

We propose a method for detecting the group’s focus of attention: the visual point at which a majority of participants direct their gaze in a conversation. This information enables a robot to infer important conversational cues and adjust its behavior to support more natural conversational interactions. Our approach uses a Hidden Markov Model based on mimicry, where the robot observes the head orientation of participants and infers their gaze direction to identify the group’s focus of attention. We demonstrate our method by replicating the gaze patterns of the group members, showing that the robot can accurately determine the focal point. We evaluated our algorithm using a combination of datasets and real-world scenarios with a Fetch robot, demonstrating an accuracy of 81% compared to a baseline of 54%. Our proposed method has the potential to significantly improve group-oriented human-robot interaction.

BackgroundThe uncertain environments of future space missions means that astronauts will need to acquire new skills rapidly; thus, a non-invasive method to enhance learning of complex tasks is desirable. Stochastic resonance (SR) is a phenomenon where adding noise improves the throughput of a weak signal. SR has been shown to improve perception and cognitive performance in certain individuals. However, the learning of operational tasks and behavioral health effects of repeated noise exposure aimed to elicit SR are unknown. ObjectiveWe evaluated the long-term impacts and acceptability of repeated auditory white noise (AWN) and/or noisy galvanic vestibular stimulation (nGVS) on operational learning and behavioral health. MethodsSubjects (n = 24) participated in a time longitudinal experiment to access learning and behavioral health. Subjects were assigned to one of our four treatments: sham, AWN (55 dB SPL), nGVS (0.5 mA), and their combination to create a multi-modal SR (MMSR) condition. To assess the effects of additive noise on learning, these treatments were administered continuously during a lunar rover simulation in virtual reality. To assess behavioral health, subjects completed daily, subjective questionnaires related to their mood, sleep, stress, and their perceived acceptance of noise stimulation. ResultsWe found that subjects learned the lunar rover task over time, as shown by significantly lower power required for the rover to complete traverses (p < 0.005) and increased object identification accuracy in the environment (p = 0.05), but this was not influenced by additive SR noise (p = 0.58). We found no influence of noise on mood or stress following stimulation (p > 0.09). We found marginally significant longitudinal effects of noise on behavioral health (p = 0.06) as measured by strain and sleep. We found slight differences in stimulation acceptability between treatment groups, and notably nGVS was found to be more distracting than sham (p = 0.006). ConclusionOur results suggest that repeatedly administering sensory noise does not improve long-term operational learning performance or affect behavioral health. We also find that repetitive noise administration is acceptable in this context. While additive noise does not improve performance in this paradigm, if it were used for other contexts, it appears acceptable without negative longitudinal effects.