Robot systems for teleoperation commonly use a spring-like force pulling the follower robot towards the leader's position to track their movements. With this control strategy, the tracking accuracy deteriorates when the follower' stiffness is low, but high stiffness poses a danger to objects or people in the follower robot's environment. To address this trade-off between tracking accuracy and safety, we propose an alternative intention assimilation control (IAC) strategy where the robot's tracking accuracy can be ensured without high stiffness. Different from traditional approaches, which transmit the leader's current position to the follower, this new controller estimates the leader's target position and transmits it to the follower. With this strategy, the follower impedance can be changed on-the-fly to continuously reflect the user's desired impedance or modulated automatically to fulfill the task requirements. Our controller was validated on two 7 degree-of-freedom manipulators, yielding high tracking accuracy with varying impedance. Four experiments were conducted to compare teleoperation with IAC to tele-impedance control (TIC) during free tracking, interaction with a balloon, during peg insertion, and table polishing with force feedback. The results show that IAC increases tracking accuracy, improves task completion rate and reduces completion time. IAC enables the robot to accurately replicate the user's movement while giving them freedom to modulate the impedance according to their intention, providing an unprecedented level of control of the follower's position and its impedance during unilateral and bilateral teleoperation.

Abstract Back-ground Beneficial rehabilitation interventions for severely impaired stroke patients are limited. Owing to practical constraints in the routine clinical use of electroencephalogram (EEG)-based brain-computer interfaces, this study investigates the feasibility of using a more practical electromyography (EMG) to detect movement intention in severe stroke subjects without visible movement. Currently, no large-scale studies provide strong evidence in favour of EMG-based human-machine interaction for closed-loop control of robotic assistance for severe stroke. Objective To screen severely impaired stroke subjects without active wrist extension for the presence of residual EMG activity. Methods High-density surface EMG was recorded from the wrist extensor muscles of 100 severely impaired stroke survivors while they repeatedly attempted wrist extension. EMG activity during “Rest” and “Move” states was compared, and subjects showing statistically greater muscle activity during Move than Rest were classified as having residual EMG. A modified Hodges detector combined with the probability difference-sum ratio (PDSR) was used for classification, with a threshold of 0.73 identifying subjects with residual EMG. Results Of the 100 subjects without active wrist extension (Muscle power: MRC < 2), 64 exhibited residual EMG activity, supporting the feasibility of EMG for movement intention detection. Among these, 35 demonstrated consistent muscle activity (Detection probability > 0.2); representing suitable candidates for EMG-driven robot-assisted therapy. Conclusions A substantial proportion of severely impaired stroke subjects without active movement could benefit from a simpler EMG-based interface for robot-assisted therapy. Distinct neural mechanisms (intact voluntary drive or abnormal co-activation) may contribute to the residual muscle activity, which should be considered while designing control strategies.

Robotic-assisted procedures offer enhanced precision, but while fully autonomous systems are limited in task knowledge, difficulties in modeling unstructured environments, and generalisation abilities, fully manual teleoperated systems also face challenges such as delay, stability, and reduced sensory information. To address these, we developed an interactive control strategy that assists the human operator by predicting their motion plan at both high and low levels. At the high level, a surgeme recognition system is employed through a Transformer-based real-time gesture classification model to dynamically adapt to the operator's actions, while at the low level, a Confidence-based Intention Assimilation Controller adjusts robot actions based on user intent and shared control paradigms. The system is built around a robotic suturing task, supported by sensors that capture the kinematics of the robot and task dynamics. Experiments across users with varying skill levels demonstrated the effectiveness of the proposed approach, showing statistically significant improvements in task completion time and user satisfaction compared to traditional teleoperation.

Abstract Humans often collaborate under asymmetric information, for example when two people carry a table and only one knows the destination. They coordinate without speech using cues from movement kinematics, interaction forces, and object states. Characterizing this sensorimotor communication is difficult because these signals both execute the task and convey information, whose meaning is context-dependent. Here, we investigated a virtual table-carrying task where one partner knew the target while the other inferred it from visuo-haptic feedback. Participants flexibly adapted kinematic and haptic cues across contexts to convey intention. We introduce an explainable machine-learning framework that decodes intent from ongoing multimodal signals and quantifies where individual features are informative. Incorporating the decoded signals into a drift-diffusion model accurately predicted the uninformed partner’s target choices and decision times. Together, our framework explains how humans communicate through action and offers principles for collaborative robots to infer and express intent through physical interaction.

Achieving human-level dexterity in contact-rich, tool-mediated manipulation remains a significant challenge due to visual occlusion and the underdetermined nature of haptic sensing. This paper introduces a parameterized Equilibrium Manifold (EM) as a unified representation for tool-mediated interaction, and develops a closed-loop framework that integrates haptic estimation, online planning, and adaptive stiffness control. We establish a physical-geometric duality using an adaptive manipulation potential incorporating a differentiable contact model, which induces the manifold's geometric structure and ensures that complex physical interactions are encapsulated as continuous operations on the EM. Within this framework, we reformulate haptic estimation as a manifold parameter estimation problem. Specifically, a hybrid inference strategy (haptic SLAM) is employed in which discrete object shapes are classified via particle filtering, while the continuous object pose is estimated using analytical gradients for efficient optimization. By continuously updating the parameters of the manipulation potential, the framework dynamically reshapes the induced EM to guide online trajectory replanning and implement uncertainty-aware impedance control, thereby closing the perception-action loop. The system is validated through simulation and over 260 real-world screw-loosening trials. Experimental results demonstrate robust identification and manipulation success in standard scenarios while maintaining accurate tracking. Furthermore, ablation studies confirm that haptic SLAM and uncertainty-aware stiffness modulation outperform fixed impedance baselines, effectively preventing jamming during tight tolerance interactions.

The speed of voluntary movements varies systematically, with some individuals moving consistently faster than others across different actions. These variations, conceptualized as vigor, reflect a time-effort-accuracy tradeoff in motor planning. How do two mechanically coupled partners with different individual vigors collaborate, e.g. to move a table together? Here, we show that such dyads coordinate goal-directed movements with minimal interaction force, exhibiting a dyadic vigor with similar characteristics as individual vigor. The emerging dyadic motor plan is strongly influenced by the slower partner, whose vigor predicts dyadic vigor, with effects lasting beyond practice. Computational modeling with stochastic optimal control reveals the critical role of partners' movement timing uncertainty and vigor in shaping coordination, allowing us to predict dyadic movements from individual behavior across diverse conditions. These findings shed light on the mechanisms underlying human collaboration and may be used in applications ranging from physical training and rehabilitation to collaborative robotics for manufacturing.

Supernumerary robotic limbs (SLs) have the potential to transform a wide range of human activities, yet their usability remains limited by key technical challenges, particularly in ensuring safety and achieving versatile control. Here, we address the critical problem of maintaining balance in the human-SLs system, a prerequisite for safe and comfortable augmentation tasks. Unlike previous approaches that developed SLs specifically for stability support, we propose a general framework for preserving balance with SLs designed for generic use. Our hierarchical three-layer architecture consists of: (i) a prediction layer that estimates human trunk and center of mass (CoM) dynamics, (ii) a planning layer that generates optimal CoM trajectories to counteract trunk movements and computes the corresponding SL control inputs, and (iii) a control layer that executes these inputs on the SL hardware. We evaluated the framework with ten participants performing forward and lateral bending tasks. The results show a clear reduction in stance instability, demonstrating the framework's effectiveness in enhancing balance. This work paves the path towards safe and versatile human-SLs interactions. [This paper has been submitted for publication to IEEE.]

This article presents a learning-based robust control approach for electro-ribbon artificial muscle systems. The complex and coupled nonlinear electromechanical dynamics of these actuators make accurate modeling difficult. To address this challenge, we employ a neural network to approximate the actuator’s internal dynamics, with weight parameters set during an initial learning phase via gradient descent of a Lyapunov function. Feedback linearization combined with slidingmode control is then used to compensate for model estimation errors and achieve precise trajectory tracking, while an integral action mitigates steadystate uncertainties. A Lyapunov-function-based analysis establishes bounded estimation errors during the learning phase, and uniform ultimate boundedness of the closed-loop tracking errors. Experimental results under various operating conditions validate the novel control scheme, exhibiting improved tracking performance and robustness compared to conventional PID control.

The coordination of the multiple degrees-of-freedom of the human body may be simplified by muscle synergies, motor modules which can be flexibly combined to achieve various goals. Studies investigating adaptation to novel relationships between muscle activity and task outcomes found that altering the recruitment of such modules is faster than the learning of their structures de novo. However, how learning new synergy recruitments or new synergy structures may occur remains unclear. While trial-by-trial learning of novel sensorimotor tasks has been successfully modeled at the level of task variables, few models accounted for the redundancy of the motor system, particularly at the muscular level. However, these models either did not consider a modular architecture of the motor system, or assumed a priori knowledge of the sensorimotor task. Here, we present a computational model for the generation of redundant muscle activity where explicitly defined modules, implemented as spatial muscle synergies, can be updated together with their recruitment coefficients through an error-based learning process dependent on a forward model of the sensorimotor task, which is not assumed to be known a priori. Our model can qualitatively reproduce the experimental observations of slower learning and larger changes in the structure of the muscle activity under sensorimotor tasks that require the learning of novel patterns of muscle activity, providing further insights into the modular organization of the human motor system.

Estimating physical properties is critical for safe and efficient autonomous robotic manipulation, particularly during contact-rich interactions. In such settings, vision and tactile sensing provide complementary information about object geometry, pose, inertia, stiffness, and contact dynamics, such as stick-slip behavior. However, these properties are only indirectly observable and cannot always be modeled precisely (e.g., deformation in non-rigid objects coupled with nonlinear contact friction), making the estimation problem inherently complex and requiring sustained exploitation of visuo-tactile sensory information during action. Existing visuo-tactile perception frameworks have primarily emphasized forceful sensor fusion or static cross-modal alignment, with limited consideration of how uncertainty and beliefs about object properties evolve over time. Inspired by human multi-sensory perception and active inference, we propose the Cross-Modal Latent Filter (CMLF) to learn a structured, causal latent state-space of physical object properties. CMLF supports bidirectional transfer of cross-modal priors between vision and touch and integrates sensory evidence through a Bayesian inference process that evolves over time. Real-world robotic experiments demonstrate that CMLF improves the efficiency and robustness of latent physical properties estimation under uncertainty compared to baseline approaches. Beyond performance gains, the model exhibits perceptual coupling phenomena analogous to those observed in humans, including susceptibility to cross-modal illusions and similar trajectories in learning cross-sensory associations. Together, these results constitutes a significant step toward generalizable, robust and physically consistent cross-modal integration for robotic multi-sensory perception.

Synchronization often emerges spontaneously among interacting people, yielding practical benefits for tasks such as sports, physical rehabilitation, and collaborative manufacturing, and fostering a sense of unity and trust. Although visual interaction is typically considered the primary channel for achieving synchronization, it is constrained by sensorimotor delays, occlusions, and limited attentional resources. Here, we tested whether haptic communication can induce or enhance synchronization in a group motor task. Six quartets performed oscillatory wrist flexion/extension while being connected via virtual elastic bands; we compared haptic only, visual only, combined haptic&visual, and no feedback conditions. Even weak haptic coupling induced group synchronization, and combining haptic with visual feedback yielded higher and more stable synchronization than either modality alone. Haptic feedback also increased the frequency at which groups coordinated, and movement smoothness rose with frequency. These findings lay the groundwork for designing haptic interaction protocols for collaborative group environments.

From rough-and-tumble play to carrying objects together, children frequently engage in physical interactions. Adults use such interactions to exchange motion plans and enhance joint performance; but whether children possess this haptic communication ability, and how it relates to sensorimotor development, is not known. To address these questions, we developed a dedicated paradigm for school-aged children and conducted a study with 56 children (28 dyads, 5-17 years) using a gamified wrist-tracking task while mechanically coupled via a dual robotic interface. We found that tracking error decreased with age and was further reduced when children were connected to a partner. As in adults, connecting to a more-skilled partner improved performance, whereas connecting to a less-skilled partner did not significantly degrade it. These results indicate that school-aged children can exploit haptic communication during cooperative action despite their still-developing sensorimotor skills.

To enable robots to perform human-like dexterous manipulation, it is essential to understand how mechanical compliance, multi-modal sensing, and purposeful interaction jointly shape tactile perception. In this study, we use a dedicated modular e-Skin with interchangeable mechanical compliance and multi-modal sensing to systematically investigate how sensing embodiment and interaction strategies influence robotic perception of objects. Leveraging a curated set of soft wave objects with controlled viscoelastic and surface properties, we explore a rich set of palpation primitives that vary in indentation depth, frequency, and directionality. In addition, we propose the latent filter, an unsupervised, action-conditioned deep state-space model of the sophisticated interaction dynamics, and infer causal mechanical properties into a structured latent space. This provides in-depth, interpretable representation of how embodiment and interaction determine and influence perception. Our investigation demonstrates that multi-modal sensing outperforms unimodal sensing, emphasizing complex interaction between the environment and the mechanical properties of e-Skin.

Abstract Supernumerary robotic limbs (SLs) could extend human motor capabilities beyond the natural body, yet most demonstrations have been limited to simple, quasi-static tasks or direct teleoperation under visual control. Whether humans can safely and intuitively collaborate with autonomous SLs during complex, dynamic tasks remains unknown. We studied this question in a demanding assembly task requiring coordinated transport and handovers of a long object, which could not be completed by a single person. We developed a reconfigurable backpack platform mounting up to four high-payload robotic arms, together with safety-aware motion planning that generates human-compatible trajectories from task states without training data. Despite the SLs operating out of sight, participants adapted quickly, completed the task reliably, and reported low cognitive demand alongside high perceived safety and predictability. Kinematic and force analyses showed that control design governed both movement fluency and whole-body stability: faster limb motions yielded smoother interactions and shorter completion times, and a mirrored coordination strategy–in which one robotic limb counterbalanced the other–further shortened duration and reduced variability in left-right ground reaction force differences during co-manipulation. These results show that humans can integrate autonomous wearable limbs as out-of-sight partners in dynamic whole-body collaboration, and delineate behavioral and control principles underpinning fluency, stability, and usability in human augmentation.

Tactile exploration enables robots to acquire rich mechanical information about objects through physical interaction and enhances their perception and manipulation in unstructured environments. However, existing tactile object recognition methods are limited by closed-set assumptions, recognizing only predefined categories and relying heavily on labeled data. This limits their ability to handle unseen objects. Here we propose an open-set tactile recognition approach that broadens the ability of robots to identify known and novel objects. The approach leverages the intrinsic mechanical properties of objects, estimated online through haptic interaction. It integrates supervised learning for recognizing known objects with unsupervised clustering for characterizing novel ones. The inclusion of new objects is enabled by regression-based mechanical property estimation combined with distance-based online clustering. This allows robots to generalize effectively and extract reliable features from unseen objects, improving perception in dynamic environments. Validation on a 20 object dataset with diverse mechanical characteristics shows the efficacy of the approach, achieving a 96.02 +/- 1.69% recognition rate for 12 known objects and detecting 8 novel objects with a 90.79 +/- 5.45% accuracy. After integrating the newly identified objects into the robot's knowledge base, the approach achieves an Adjusted Rand Index of 0.701 +/- 0.096, confirming effective clustering. These results show the potential of the proposed method to advance open-set tactile perception and support more adaptive robot interaction in real-world settings.

Early limb motor development progresses from spontaneous to increasingly complex movements. Restricted movement variety in infancy can indicate neuromotor impairment associated with cerebral palsy (CP) risk. The General Movement Optimality Score (GMOS) used in clinic detects abnormal movement patterns, but requires highly trained physiotherapists, limiting accessibility. Objective biomechanical measures may provide quantitative predictors of CP risk, reducing clinical burden while offering novel physiotherapeutic targets. This study investigated whether variation in lower limb kinematics and muscle forces during spontaneous kicking is reduced in infants with lower GMOS scores, indicating higher CP risk. We also examined whether variation correlates with gestational and corrected age, reflecting the transition to more complex movements. Twenty-six Infants aged 0-3 months underwent wired electromagnetic motion capture (Polhemus Liberty System) with musculoskeletal modelling to estimate kinematics and muscle force waveforms, alongside GMOS scoring. Waveform variation was quantified using functional principal component analysis and correlated with GMOS scores and age, while partial least squares discriminant analysis (PLS-DA) identified biomechanical predictors of GMOS indications of 'normal' versus 'poor repertoire' (PR) movements. Variation in knee flexion, hip flexion and abduction kinematics, and rectus femoris, pectineus, biceps femoris and gemelli forces significantly correlated with GMOS scores. Gestational age correlated with hip abduction, ankle dorsiflexion, and gemelli and medial gastrocnemius forces. PLS-DA achieved 85% accuracy distinguishing normal from poor repertoire movements, with knee and hip kinematics and biceps femoris force as key discriminators. These findings demonstrate that lower limb kinematic and neuromuscular variation aligns with clinical motor assessments, supporting their potential as objective biomechanical markers of CP risk.

The acquisition of handwriting is a challenging task for children, requiring consistent exercise throughout primary school. Many children struggle to develop fluent handwriting, which in turn may impact their academic success and affect their self-esteem. The emergence of new technologies supporting handwriting exercise and allowing active interactions with educators, caregivers, and/or peers has led researchers to explore the potential use of these new tools. Studies show that haptic technology and exploiting experts' trajectory feedback may improve students' performance better than one-way instruction. In the present study, we explore the use of bidirectional haptic feedback to support handwriting exercises. Twelve child and caregiver dyads were asked to copy single letters or letter pairs in cursive handwriting while being physically coupled or not. Results show improvement for the less skilled participant in the dyad, especially in handwriting fluency, when physically coupled.

Perceiving the physical properties of different surfaces/textures via tactile sensing has been a long-standing problem in robotics. Most prior work has been limited to discriminative models that classify textures into a fixed set of categories. However, to enable seamless and efficient autonomous manipulation, robots must infer physical properties as structured, continuous variables rather than as discrete class labels. In this work, we present a novel deep state-space model (DSSM) to learn and infer key causal textural properties in an unsupervised manner. Using variational inference to solve the DSSM, our proposed Latent Filter allows robotic systems to perceive textures in a continuous and generalizable manner. In addition, we explore a novel interaction approach: Tacser (Tactile Enhancer), to further enhance tactile sensing through vibrations induced by high-frequency micro-movements and thereby improve perception. We evaluated our approach against state-of-the-art techniques and performed extensive ablation studies to demonstrate its effectiveness. This work advances tactile-based texture perception, providing a generalizable and comprehensive framework for robotics.

When a human controls a robot directly or via teleoperation, incomplete information and unpredictable environmental conditions can lead to conflicts between their plans. Differential game theory (GT) offers a framework for optimal robotic assistance, but existing methods for identifying the human model require a shared plan. This article introduces an approach to deal with a diverging human plan by leveraging a neuromechanical model of their viscoelasticity to directly estimate their motion intent during movement. The viscoelastic gains can then be integrated into a GT framework to compute optimal contribution of the robot to the common motor task. We evaluated the proposed method in experiments, comparing it with fixed impedance control and nonoptimal variable impedance control. Results demonstrate stable interactions even in the presence of conflicting motion plans, and superior performance relative to these two alternative methods.