In this paper, we present a robotic ultrasound acquisition system that integrates stereo vision, touch-based feedback, and expert-informed strategies to perform autonomous and adaptive abdominal scans. The system records freehand motion and force data from expert radiologists, creating a framework to capture transducer motion, applied forces, and anatomical scanning strategies. This expert data is replayed to replicate characteristic scans with the robot, forming a foundation for further autonomous capabilities. Using stereo vision, the system generates three-dimensional topography maps of the patient's abdomen, which are refined through stiffness measurements at key points to delineate the rib cage boundary. These combined techniques enable the robot to execute two distinct scanning paths: an upward-angled sweep beneath the rib cage to visualize structures near the upper abdomen and a perpendicular sweep across soft tissue regions. A compliant, torque-controlled seven degree-of-freedom robotic manipulator is controlled to maintain consistent probe contact through closed-loop force control over the varied anatomical surfaces. Physical experiments demonstrate that the system achieves high-quality imaging comparable to expert scans while dynamically adapting to patient-specific topographies. Furthermore, the robotic system surpasses expert capabilities by enabling three-dimensional volume acquisition, which enhances diagnostic potential and provides volumetric data for advanced analyses. This work highlights the integration of expert knowledge into autonomous robotic systems and underscores the potential of combining perception-based autonomy with physical reasoning for enhanced diagnostic performance.

In this paper, we present a robotic ultrasound system that implements explicit contact force control and force-based full probe orientation optimization to achieve stable, responsive, and high-quality ultrasound imaging. Traditional robotic ultrasound systems often rely on implicit force control methods, such as admittance control, which are limited by their underlying motion-control loops. In contrast, our approach directly regulates contact force and moment at the end-effector, enabling rapid adaptation to heterogeneous tissue properties and dynamic environments. We benchmark the proposed explicit force controller against a state-of-the-art integral adaptive admittance controller, demonstrating a significant reduction in phase lag from 121 degrees to 5.76 degrees at force tracking frequencies exceeding typical respiratory rates. In a second set of benchmarking experiments, compared to the admittance-based method, the explicit force controller achieves a reduction in average force tracking error of 77.7 % when the phantom is moving toward the transducer, and 72.6 % when the phantom is moving away. We integrate the explicit force and moment controller into a six-DoF haptic framework that renders physically-grounded interaction forces to the operator while the robot autonomously regulates contact force and optimizes probe alignment based on acoustic coupling. Validation across static and dynamic scans, as well as under external perturbations, shows that the system consistently maintains target force and moment profiles, aligns the probe with local surface normals, and adapts to changing contact conditions. Experimental results demonstrate that explicit force-based control improves ultrasound image quality, as quantified by confidence maps, compared to a manual haptic scan. These findings support the use of explicit force and moment control as an effective approach for robotic ultrasound imaging.

Human-centered robotics seeks to extend human skills, intuition, and decision-making into dangerous, remote, and inaccessible environments, and not to replace them. This vision is exemplified by OceanOneK, Stanford University's humanoid underwater robot, which combines autonomous physical interaction capabilities with intuitive haptic and stereo-vision interfaces. Through this integration, human experts can perceive, manipulate, and interact with remote environments as if physically present.OceanOneK employs compliant control, whole-body interaction, and skill learning from human demonstrations to perform delicate manipulation tasks in complex underwater settings. During archaeological expeditions in the Mediterranean Sea, the robot enabled experts to explore fragile historical sites and interact with submerged artifacts at depths beyond normal human reach. These missions demonstrated the powerful synergy between robotic autonomy and human expertise.The talk will also present Stanford's OpenSAI framework, built around Simulation and Active Interfaces. OpenSAI provides an integrated environment for modeling, simulation, control, and interactive operation of robotic systems, supporting whole-body control, haptic interaction, and human-in-the-loop operation. It serves as a bridge between simulation, real-time control, and remote embodied interaction, enabling the development and validation of systems such as OceanOneK.Beyond underwater exploration, these technologies have broad application in resource discovery, remote healthcare, infrastructure construction and maintenance, disaster response, and space operations. Human-centered robotic systems can expand human presence and capability in environments that are hazardous or otherwise inaccessible. This plenary will present the scientific foundations, technological advances, and future directions of human-centered robotics, illustrating how OceanOneK, haptic interfaces, skill learning, compliant control, and OpenSAI can extend human reach and preserve human expertise while opening new frontiers for science, industry, and society.

Generating sequences of human-like motions for humanoid robots presents challenges in collecting and analyzing reference human motions, synthesizing new motions based on these reference motions, and mapping the generated motion onto humanoid robots. To address these issues, we introduce SynSculptor, a humanoid motion analysis and editing framework that leverages postural synergies for training-free human-like motion scripting. To analyze human motion, we collect 3+ hours of motion capture data across 20 individuals where a real-time operational space controller mimics human motion on a simulated humanoid robot. The major postural synergies are extracted using principal component analysis (PCA) for velocity trajectories segmented by changes in robot momentum, constructing a style-conditioned synergy library for free-space motion generation. To evaluate generated motions using the synergy library, the foot-sliding ratio and proposed metrics for motion smoothness involving total momentum and kinetic energy deviations are computed for each generated motion, and compared with reference motions. Finally, we leverage the synergies with a motion-language transformer, where the humanoid, during execution of motion tasks with its end-effectors, adapts its posture based on the chosen synergy. Supplementary material, code, and videos are available at https://rhea-mal.github.io/humanoidsynergies.io.

The integration of extra-robotic limbs or fingers to enhance and extend motor capabilities, particularly for grasping and manipulation, remains a major challenge. In contrast to the natural human hand, which achieves highly dexterous and adaptive grasping, the performance of current extra-robotic limbs or fingers is still markedly limited. Human hands can detect the onset of slip through tactile feedback originating from tactile receptors during the grasping process, enabling precise and automatic regulation of grip force. This grip force is scaled by the coefficient of friction between the contacting surface and the fingers. The frictional information is perceived by humans depending upon the slip happening between the finger and the object. This ability to perceive friction allows humans to apply just the right amount of force needed to maintain a secure grip, adjusting based on the weight of the object and the friction of the contact surface. Enhancing this capability in extra-robotic limbs or fingers used by humans is challenging. To address this challenge, this paper introduces a novel approach to communicate frictional information to users through encoded vibrotactile cues. These cues are conveyed on the onset of incipient slip, thus allowing the users to perceive the friction and ultimately use this information to increase the force to avoid dropping the object. In a 2-alternative forced-choice protocol, participants gripped and lifted a glass under three different frictional conditions, applying a normal force of 3.5 N. After reaching this force, the glass was gradually released to induce slip. During this slipping phase, vibrations scaled according to the static coefficient of friction were presented to users, reflecting the frictional conditions. The results suggested an accuracy of $94.53\pm 3.05$ ( $\text{mean}\pm \text{SD}$ ) in perceiving frictional information upon lifting objects with varying friction. The results indicate the effectiveness of using vibrotactile feedback for sensory feedback, allowing users of extra-robotic limbs or fingers to perceive frictional information. This enables them to assess surface properties and adjust grip force according to the frictional conditions, enhancing their ability to grasp and manipulate objects more effectively.

The integration of extra-robotic limbs/fingers to enhance and expand motor skills, particularly for grasping and manipulation, possesses significant challenges. The grasping performance of existing limbs/fingers is far inferior to that of human hands. Human hands can detect onset of slip through tactile feedback originating from tactile receptors during the grasping process, enabling precise and automatic regulation of grip force. The frictional information is perceived by humans depending upon slip happening between finger and object. Enhancing this capability in extra-robotic limbs or fingers used by humans is challenging. To address this challenge, this paper introduces novel approach to communicate frictional information to users through encoded vibrotactile cues. These cues are conveyed on onset of incipient slip thus allowing users to perceive friction and ultimately use this information to increase force to avoid dropping of object. In a 2-alternative forced-choice protocol, participants gripped and lifted a glass under three different frictional conditions, applying a normal force of 3.5 N. After reaching this force, glass was gradually released to induce slip. During this slipping phase, vibrations scaled according to static coefficient of friction were presented to users, reflecting frictional conditions. The results suggested an accuracy of 94.53 p/m 3.05 (mean p/mSD) in perceiving frictional information upon lifting objects with varying friction. The results indicate effectiveness of using vibrotactile feedback for sensory feedback, allowing users of extra-robotic limbs or fingers to perceive frictional information. This enables them to assess surface properties and adjust grip force according to frictional conditions, enhancing their ability to grasp, manipulate objects more effectively.

This paper presents GD-Prox, a Haptic-Glove Dual-Proxy framework for remote dexterous robot hand telemanipulation. The proposed method extends the Dual-Proxy approach to glove-hand systems, addressing inherent actuation constraints and enabling stable bilateral interaction over lowbandwidth networks without direct task-space force transmission. Independent local controllers on the glove and robothand sides maintain responsive behavior under significant communication delays and reduced update rates. Experimental validation using intermittent press-release and sustained pinch-grasp tasks confirms the effectiveness of the approach in maintaining stable bilateral coordination under latency. The framework establishes a robust foundation for teleoperating robot hands with limited degrees of freedom, paving the way for future extensions to inter-finger coordination and dynamic object manipulation.

Exploiting the promise of recent advances in imitation learning for mobile manipulation will require the collection of large numbers of human-guided demonstrations. This paper proposes an open-source design for an inexpensive, robust, and flexible mobile manipulator that can support arbitrary arms, enabling a wide range of real-world household mobile manipulation tasks. Crucially, our design uses powered casters to enable the mobile base to be fully holonomic, able to control all planar degrees of freedom independently and simultaneously. This feature makes the base more maneuverable and simplifies many mobile manipulation tasks, eliminating the kinematic constraints that create complex and time-consuming motions in nonholonomic bases. We equip our robot with an intuitive mobile phone teleoperation interface to enable easy data acquisition for imitation learning. In our experiments, we use this interface to collect data and show that the resulting learned policies can successfully perform a variety of common household mobile manipulation tasks.

In this paper, we investigate the strategies used by a skilled ultrasound physician and explore the functionality of a robotic ultrasound scan acquisition system. We develop a framework to record an expert radiologist's probe motion, probe force, audio narration, and generated ultrasound images. Then, we optimize the acquired scan data to remove redundant sections and use the processed data to perform an autonomous survey scan with the robotic system. Finally, the radiologist performs a full detailed scan of the abdomen through a haptic user interface capable of long-distance remote interaction. We compare images generated at similar points in each scan and identify the same anatomy and pathologies, thus demonstrating equivalent diagnostic performance between the skilled ultrasound physician and robotic system.

This paper presents a highly compact paired-parallel mechanical architecture that aims for haptic transparency: accurate motion in free space and high force capability in contact. Low inertia and high stiffness of parallel manipulators are key properties to ensure force-motion fidelity in haptic applications. The proposed 7-DOF architecture combines two Delta-like translators in-parallel to a helical handle, to generate its translational and rotational motions with low coupling. The modeling of the architecture is simply written from the models of the Delta robots and the handle linkage. It shows that performances of such paired-parallel architecture derive from its components. A haptic device is designed based on this compact architecture. A prototype validates its mobility and the feasibility of the assembly.

This is to inform you that the following sentence was inadvertently omitted from the final version

Even the most robust autonomous behaviors can fail. The goal of this research is to both recover and collect data from failures, during autonomous task execution, so they can be prevented in the future. We propose haptic intervention for real-time failure recovery and data collection. Elly is a system that allows for seamless transitions between autonomous robot behaviors and human intervention while collecting sensory information from the human's recovery strategy. The system and our design choices were experimentally validated on a single arm task -- installing a lightbulb in a socket -- and a bimanual task -- screwing a cap on a bottle -- using two 7-DOF manipulators equipped 4-finger grippers. In these examples, Elly achieved over 80% task completion during a total of 40 runs.

In this paper, we develop the analytical framework for a novel Wireless signal-based Sensing capability for Robotics (WSR) by leveraging a robots’ mobility in 3D space. It allows robots to primarily measure relative direction, or Angle-of-Arrival (AOA), to other robots, while operating in non-line-of-sight unmapped environments and without requiring external infrastructure. We do so by capturing all of the paths that a wireless signal traverses as it travels from a transmitting to a receiving robot in the team, which we term as an AOA profile. The key intuition behind our approach is to enable a robot to emulate antenna arrays as it moves freely in 2D and 3D space. The small differences in the phase of the wireless signals are thus processed with knowledge of robots’ local displacement to obtain the profile, via a method akin to Synthetic Aperture Radar (SAR). The main contribution of this work is the development of (i) a framework to accommodate arbitrary 2D and 3D motion, as well as continuous mobility of both signal transmitting and receiving robots, while computing AOA profiles between them and (ii) a Cramer–Rao Bound analysis, based on antenna array theory, that provides a lower bound on the variance in AOA estimation as a function of the geometry of robot motion. This is a critical distinction with previous work on SAR-based methods that restrict robot mobility to prescribed motion patterns, do not generalize to the full 3D space, and require transmitting robots to be stationary during data acquisition periods. We show that allowing robots to use their full mobility in 3D space while performing SAR results in more accurate AOA profiles and thus better AOA estimation. We formally characterize this observation as the informativeness of the robots’ motion, a computable quantity for which we derive a closed form. All analytical developments are substantiated by extensive simulation and hardware experiments on air/ground robot platforms using 5 GHz WiFi. Our experimental results bolster our analytical findings, demonstrating that 3D motion provides enhanced and consistent accuracy, with a total AOA error of less than 10 ◦ for 95% of trials. We also analytically characterize the impact of displacement estimation errors on the measured AOA and validate this theory empirically using robot displacements obtained using an off-the-shelf Intel Tracking Camera T265. Finally, we demonstrate the performance of our system on a multi-robot task where a heterogeneous air/ground pair of robots continuously measure AOA profiles over a WiFi link to achieve dynamic rendezvous in an unmapped, 300 m 2 environment with occlusions.

We present a new paradigm for performing remote haptic-robot interactive operations. The new paradigm is anchored on an architecture that combines local autonomy with a high-level exchange strategy of reference input. This represents a departure from the conventional reliance on direct exchange of low-level control signals in a global feedback control system. The new approach establishes two local autonomous controllers acting on the robot and the haptic device, interfaced at a higher level via a dual-proxy model. The dual proxy is a passive bridge between the local autonomous controllers. It generates appropriate motion and force reference inputs that are consistent with the task physical interactions and the levels of autonomy. Its model is adjusted online with respect to exchanged position, contact, and environment geometry information. A key component in this methodology is the perception algorithm on the robot side, the force-space particle filter, designed to reliably estimate in real time the environment contact geometry. The series of simulations and physical experimental validations of the approach demonstrate the transparency and high fidelity in haptic-robot interaction and its inherent robustness to communication delays.

Programming paths for robotic welding conventionally requires precise positioning of workpieces, detailed 3D models and/or tedious teach pendant programming. A new method is introduced in this paper that enables an operator to teach the weld path to the robot through a haptic-visual interface. The operator teaches the path by guiding the tool tip to contact on the workpiece surface with force feedback through the haptic device, and drawing exploratory paths that intersect the edge to be welded as well as adjoining surfaces. Tool-tip positions in contact with the workpiece are recorded. A RANSAC-type algorithm is used to automatically estimate a piecewise parametric curve along the edge as well as geometric parameters of the adjoining surfaces. The required tool trajectory for the robot to weld along the workpiece edge is automatically generated. Experiments performed in simulation and on a physical KUKA IIWA7 robot demonstrate that the developed method can successfully detect workpiece edges within a maximum deviation of 1mm. Furthermore, the method is intuitive, and requires no knowledge of robot programming for an operator to program multi-segment weld paths quickly.

The promise of oceanic discovery has intrigued scientists and explorers, whether to study underwater ecology and climate change, or to uncover natural resources and historic secrets buried deep at archaeological sites. This quest to explore the oceans requires expert human access, but much of the oceans is inaccessible to humans. Reaching these depths is imperative for understanding the ecology, maintaining, and repairing underwater structures, and working in archaeological sites over this immensely unknown part of our planet. This challenge demands human - level abilities at depths where humans cannot or should not be. Ocean One was conceived to create a robotic diver with a high degree of autonomy for physical interaction with the environment while connected to a human expert through an intuitive interface. The robot was deployed in an expedition in the Mediterranean to King Louis XIV’s flagship Lune, lying off the coast of Toulon at ninety - one meters. The discussion focuses on the development of a new prototype, OceanOneK, with the ability to reach 1000 meters. Distancing humans physically from dangerous and unreachable spaces while connecting their skills, intuition, and experience to the task promises to fundamentally alter remote work. These development show how human - robot collaboration induced synergy can expand our abilities to reach new resources, build and maintain infrastructure, and perform disaster prevention and recovery operations - be it deep in oceans and mines, at mountain tops, or in space.

For decades, robots have been able to reliably follow precise trajectories making them ideal tools for assembly lines and other structured environments. However, pre-programmed motions fail under uncertainty and are unsafe around humans, making them inadequate for unstructured environments. This paper presents a framework to generate safe, robust, and generalizable robot behaviors for contact tasks where compliance plays a key role. First, we collect task data from haptic demonstrations. Then, we segment the data into a sequence of compliant primitives. Finally, we extract the key parameters required for a robot to perform each of the primitive actions using interpretable, model-based controllers. This method was experimentally validated on a steel bolting task using a 7-DOF Franka Panda robot. By recombining the primitives, we were also able to screw a cap onto containers of different sizes, placed in arbitrary configurations, using two different 7-DOF manipulators. The results show that our method generates position and orientation invariant, robot-agnostic controllers.

We present a comprehensive formulation to the problem of controlling a high-dimensional robotic system involving complex tasks subject to a variety of constraints, obstacles, balance, and contact challenges. Using intuitive and natural representations, the approach is initiated by establishing individual objectives for a task and its constraints. Simple independent controllers using artificial potential fields are then designed for each objective to reach goals while enforcing the constraints. Dynamically consistent projections in nullspaces associated with task and constraint representations are employed to deliver a coherent whole-body robot control. In multi-link multi-contact tasks, contact forces produce both resulting and internal forces. Internal forces play a critical role in robot balance and stability, achieved in this framework through modeling and controlling virtual linkages that explicitly describe the relationship between active/passive contact force, resultant force, controlled/uncontrolled internal force for multi-link multi-contact underactuated robots. Control of contacts with the environment involves material considerations such as friction and geometric constraints. Potential barriers direct the selection of contact forces ensuring stability and balance. This approach of dynamic projection and the Virtual Linkage Model addresses robot underactuation. In addition, the framework introduces a coordinate completion mechanism to establish a generalized coordinates representation of the task, removing redundancy and maintaining the full operational space dynamics description. This enables task-space dynamic control based on the relevant inertial properties. We present the experimental validation on a physical humanoid platform.

Collaborative Welding and Joint Sealing Robots With Haptic Feedback Cynthia Brosque, Elena Galbally, Yuxiao Chen, Ruta Joshi, Oussama Khatib and Martin Fischer Pages 1-8 (2021 Proceedings of the 38th ISARC, Dubai, UAE, ISBN 978-952-69524-1-3, ISSN 2413-5844) Abstract: Due to their unstructured and dynamic nature, construction sites present many challenges for robotic automation of tasks. Integrating human-robot collaboration (HRC) is critical for task success and implementation feasibility. This is particularly important for contact-rich tasks and other complex scenarios which require a level of reasoning that cannot be accomplished by a fully autonomous robot. Currently, many solutions rely on precise teleoperation that requires one operator per robot. Alternatively, one operator may oversee several semi-autonomous robots. However, the operators do not have the sensory feedback needed to adequately leverage their expertise and craftsmanship. Haptic interfaces allow for intuitive human-robot collaboration by providing rich contact feedback. This paper presents two human-robot collaboration solutions for welding and joint sealing through the use of a haptic device. Our approach allows for seamless transitions between autonomous robot capabilities and human intervention with rich contact feedback. Additionally, this work opens the door to intuitive programming of new tasks through haptic human demonstration. Keywords: Robotics; Construction; Haptics; Human-Robot Collaboration; Robotic Manipulation; Tactile Feedback DOI: https://doi.org/10.22260/ISARC2021/0003 Download fulltext Download BibTex Download Endnote (RIS) TeX Import to Mendeley