Inchworm-like robots have shown promise in achieving multimodal movements, yet the integration of untethered locomotion and manipulation capabilities remains a persistent challenge in soft robotics. This paper presents a novel soft robot that addresses this challenge through an anisotropic magnetization distribution, achieved via 3D printing, enabling precise control over its three-dimensional structure and magnetic properties. This unique design allows for magnetic field-controlled locomotion with asymmetric time-varying postures, achieving a maximum stride length of 9.2 mm under an 80 mT field. The robot employs active friction manipulation between its feet, enabling an inchworm-like gait with alternating forefoot and rearfoot fixation. Additionally, the specific magnetization design permits independent control of grippers under the same magnetic field, facilitating complex manipulation tasks. The robot's potential in biomedical applications is demonstrated through a multi-region targeted drug delivery experiment. This integrated locomotion and manipulation platform shows promise for applications in confined spaces and represents a step forward in expanding the capabilities of soft robots for potential medical and industrial tasks.
This paper presents the analysis and integration of two forms of fusion for multi-robot multi-sensor target estimation, namely observation fusion and belief fusion. While the observation fusion is a traditional sensor fusion where the observation likelihood is fused within the loop of recursive Bayesian estimation (RBE), the belief fusion fuses independently updated beliefs outside the RBE loop. The comparative analysis of the two fusions show that observation fusion is suited for multimodal sensors within a single robot whereas multiple robots can each update their belief more effectively through belief fusion. Numerical results validate the strength of each fusion approach and show that the effectiveness of the optimally integrated system.
Gesture recognition plays an increasingly important role in fields such as human-computer inter-action and rehabilitation robots and therapy. However, most current gesture recognition systems are based on a single type of sensor, which limits the feature sources for gesture recognition. Therefore, this paper proposes a time-frequency and visual feature fusion multimodal recognition framework (TFVF -CNN). It combines the deep features extracted from a small amount of image data with the time-frequency fea-tures of surface electromyogram (sEMG) signals to improve the accuracy of gesture recognition based on sEMG. Additionally, this paper introduces a multi-modal dataset (MEVD) containing raw sEMG signal data, sEMG signal time-frequency data, and image data, providing a reliable data source for experiments. Finally, the experimental results show that gesture recognition accuracy with multimodal fusion features and time-frequency features based on sEMG signals is 93.88 % and 95.9 %, respectively. Furthermore, we verified the performance of gesture recognition under partial image occlusion and found that TFVF -CNN still exhibits superior performance.
The traditional program control method limits the flexibility of human-computer interaction, especially in robot interaction systems that come into direct contact with the human body. sEMG signal is a relatively flexible control signal, however, sEMG signal sensors may experience positional changes due to human motion, which can affect signal acquisition. To address this issue, this paper proposes a DANN method based on adversarial thinking to solve the problem of non-ideal sEMG gesture recognition on SeNic-Main dataset. Through adversarial thinking, 8 gesture actions under electrode offset were transferred and classified, achieving a maximum improvement of 9%. The results indicate that DANN can effectively solve the problem of non ideal sEMG gesture recognition.
Currently, the rapid development of artificial in-telligence (AI) provides a sufficient technological basis for the development of autonomous medical robots, and the implemen-tation of autonomous control can produce more stable surgical results while reducing the surgeon's workload. Meanwhile, human-machine interface (HMI) has received much research attention due to its intuitive and natural control, and HMI has been widely used in medical rehabilitation and other fields. In order to fully combine the advantages of AI and HMI, reduce the surgeon's burden, and decrease the difficulty of surgical uptake, this paper proposes a cooperative suturing scheme based on AI and multi-source information HMI. On one side, the computer vision recognises the wound information and controls the robot to complete the automatic needle insertion operation, and on the other side, the human operator performs the needle extraction operation using the multi-source infor-mation HMI. We conducted 10 automatic needle insertion and extraction experiments using the cooperative suturing platform. The average error in the horizontal direction of the insertion point is 1.552mm and the average error in the horizontal direction of the extraction point is 2.011mm. The average completion time is 70.81s. The results show that the cooperative suturing platform has the ability to assist in-line continuous suturing.
Maneuverability is a critical factor for UAV s to survive in high-intensity combat scenarios. Traditional UAV s typically operate in a stable motion state to ensure flight safety. However, in highly contested environments, special missions require high mobility while maintaining safety. Tilt-rotor UAV s, which combine the characteristics of fixed-wing and multi-rotor UAVs, can perform extreme maneuvers, such as high angles of attack and post-stall maneuvers, to rapidly change posture and complete tasks like positioning and defense. Therefore, integrating and optimizing safety and maneuverability metrics is necessary for enabling UAV s to achieve extreme maneuverability. In this paper, extreme maneuver planning and control for tilt-rotor UAV s are realized through a method based on third-order Bezier curve planning of body angular velocity. Experimental results demonstrate tilt-rotor UAVs can perform extreme super-maneuverable flips while maintaining overall process control.
Micro-motion augmentation of macro-motion continuum robot can largely enhance its micro-operation capability during surgery. However, most existing micro-motion continuum robots have non-compact mechanical structure or non-smooth micro tip motion, which leads to complicated control in micro lumens. This paper proposes a micro-motion augmentation method by rotating eccentrically arranged fibres with eccentric lumens for cable-driven continuum robots. Two thermal drawn polymer fibres with an eccentric lumen on each are inserted into its eccentric lumens of the continuum robot. The micro-motion mechanism relies on small changes in the neutral axis of the continuum robot due to the rotation of the eccentrically arranged fibres. A kinematic model considering variable neural axis is built to describe the micro motion, and a simulation is conducted to investigate the deformation characteristics. The resolution and micro motion with respect to different structural parameters are also analysed. Detailed experiments are performed to validate the effectiveness of the proposed method, and results indicate that the repeatability of micro-motion is 10.1 mu m (4.0 %).
This paper presents a low-cost, light-weight, six-degree-of-freedom force feedback manipulator, aiming to conduct the master-slave control of a surgical robot system with continuum structure. Firstly, considering the practical workspace requirements of the continuum minimally invasive surgical robot, a basic series configuration is proposed that can provide three degrees of freedom for force feedback and achieve gravity balance through mechanical counterweighting. Secondly, a force feedback bidirectional master-slave control system based on a dynamic model is constructed, which can realize incremental master-slave control, variable ratio control, and force feedback functions. In the end, experiments are conducted to verify the position accuracy, gravity balance effect, and force feedback performance of the master manipulator. The experimental results show that the presented force feedback master manipulator has a position resolution of less than 0.4mm and can provide a maximum feedback force of 75.7N. The overall cost of the system is less than 60 US dollars.
Binocular endoscopy is widely used in minimally invasive surgery due to the easy acquisition of three-dimensional (3D) information. However, there are still limitations such as reduced precision in specific depth scenarios. To address the deficiency, we introduce a novel binocular endoscope reconstruction method with a variable baseline, which is founded upon the utilization of a multi-arm concentric-tube robot. First, the poses of two endoscopes with variable baselines can be obtained by modified ORB-SLAM3 algorithm. Then, the 3D structure can be reconstructed using the improved semi-global matching (SGM) algorithm based on two endoscopic images with known relative poses. Finally, to quantitatively calculate the reconstruction error, the experiment employs a method of co-reconstruction with a chessboard to indirectly measure the error. The result shows that the proposed method can effectively improve the accuracy of 3D reconstruction while maintaining real-time performance.
Accurate contact force estimation is crucial for interactions between the robot arm and its environment. Presently, contact force estimation based on current information often suffers from biases caused by unmodeled errors, noise, and various other factors. In contrast, tactile sensors information effectively complements current information, enhancing the estimation of contact forces in robot arm interactions. To improve the accuracy of force estimation without force/torque sensors, we propose an information fusion method utilizing Kalman Filter (KF) to incorporate motor current with the tactile sensor for a nursing robot arm with Differential Modular Joints (DMJ). The coupling dynamic model of the DMJ and the contact force estimation model of the tactile sensor are established respectively. Subsequently, a linear KF fusion method is introduced based on these two methods. The experimental results indicate that contact force can be reliably estimated without force/torque sensors, significantly enhancing the accuracy of force estimation.
The under-train inspection robot operates inside the inspection pit under the metro train, precisely collecting point cloud and image data of critical under-train components to analyze the health of the train. The robot employs laser Simultaneous Localization and Mapping (SLAM) for autonomous localization. The inspection pit is a shared environment for both humans and robots. During operation, the environment is dynamic with possible worker collaboration. Personnel movement generates dynamic point clouds, which affect SLAM systems that assume a static environment. This reduces precision and affects operational safety and inspection effectiveness. To address this challenge, this paper introduces a dynamic point cloud segmentation system, suitable for the low-cost hybrid solid-state LiDAR used by the robot. The system utilizes appearance and motion features in 2D Bird's Eye View (BEV) to separate dynamic point clouds from input data. It also integrates spatial density clustering results to enhance segmentation and improve the accuracy of dynamic point recognition. Experiments in a simulated inspection pit environment demonstrate the system's effectiveness. The results indicate that the system effectively recognizes and segments dynamic point clouds. This capability allows for generating scan sequences that filter out dynamic human figures, thus minimizing SLAM interference from dynamic point clouds.
Locomotion is one of the most fundamental and crucial motor abilities in both humans and animals. Compared to other quadrupedal mammals, primates exhibit unique quadrupedal characteristics, such as diagonal gait and compliant walking. Significant progress has been made in past studies to understand the relationship between behaviors and neural activities in primates. However, these studies often limit the behavior to constrained environments and focus only on the upper limbs. Here, we trained a macaque to walk freely on a treadmill at four different speeds (0.5, 1.0, 1.5, 2.0 km/h), Intracortical signals and videos were recorded at the same time. Our results indicate that the primary motor cortex (M1) exhibits higher neuronal firing rates during the swing phase of gait, suggesting its critical role in gait encoding. Furthermore, we utilized only eight neural units to decode gait speeds and employed mutual information for feature selection to improve the decoding accuracies. Results indicated that four gait speeds can be reliably classified with an accuracy of 82.1%. The firing rates of neurons from 100 ms to 300 ms before the forelimb raise exhibited the most variations during task execution and contributed the most to the decoder model. This study provides new insights into the cortical control of quadrupedal locomotion. The proposed cortical speed decoder can be applied in brain-computer-interface (BCI) to control neurorobotics such as exoskeletons and prostheses.
In various fields such as robotics, rehabilitation, biomechanics, human-machine interfaces, and clinical research, human motion prediction has extensive applications. A hybrid networks model based on Convolutional Neural Network-Long Short-Term Memory-Attention (CNN-LSTM-Attention) was employed in this paper to extract features from the high-dimensional space and time series of surface electromyography (sEMG) data for continuous prediction of knee angles. Five able-bodied subjects participated in the study, walking at a self-selected pace while six surface electromyography (sEMG) signals from muscles involved in knee flexion and extension, along with knee angles, were synchronously recorded. The performance was assessed using the root mean square error (RMSE), Pearson Correlation Coefficient (r), and coefficient of determination (R-2) between estimated and measured knee joint angles. Compared to the CNN, LSTM, and CNN-LSTM algorithms, the adopted method estimated knee angles more accurately, achieving an average RMSE, rho, and R-2 of 3.09 +/- 0.90, 0.9911 +/- 0.0052, and 0.9817 +/- 0.0108, respectively. As the advance time increased, the accuracy of knee joint angle estimation decreased, particularly when the advance time reached 100 ms, at which point the RMSE is 4.83 degress. In addition, the computing time of the prediction model was less than 1 ms, which was shorter than the advance time 100 ms it could predict, thereby facilitating real-time computation in practical environments. Compared to previous studies, the proposed method excelled at extracting features directly from raw signals, avoiding the need for handcrafted feature extraction. Consequently, the model effectively achieved continuous and accurate predictions of joint motion angles and demonstrated potential to reduce latency in the human-machine interface.
In environments with limited visual feedback, effective teleoperation can be challenging. This paper proposes a teleoperation framework that combines wristband-based haptic feedback with a posture estimation system to help operators complete tasks more efficiently. The system provides force information from the robot's end-effector via the haptic wristband, supplementing the perception that visual feedback alone cannot provide. Two types of haptic feedback were tested, and the single-motor vibration method was selected. Experimental tasks involving cutting and forking fruit demonstrated improved perception accuracy and faster reaction times, reducing collision forces. The wristband-based haptic feedback device shows great potential for improving teleoperation in complex scenarios with limited visual feedback and has been open-sourced: https://github.com/ClangWU/forceband.git.
Motion capture of quadrupedal animals can provide expert demonstration for the locomotion control of quadrupedal robots, whereas pure reinforcement learning using rewards based on human intuition often yield unnatural behaviors. This paper proposes a two-phase training framework to take advantage of such expert data for rapid style learning and flexible skill reuse. Generative adversarial imitation learning of gait styles and skill exploration via maximization of mutual information between skills and state transitions are conducted in parallel with reinforcement learning in the first pre-training phase. In the second phase, fine-tuned policy network for a customized downstream task is trained to realize skill reuse, even for a task not demonstrated in the expert data. The experiments on a physical robot have shown successful simulation-to-reality transfer, and skills learned from walking, trotting and turning of real animals on flat ground can yield robust behaviors like crawling on flat ground and climbing up stairs of the robot.
Gesture recognition is crucial in fields such as exoskeleton and prosthetic control. A key factor affecting its practical effectiveness is how it can sustain high-quality gesture recognition performance under everyday external interferences. To address this challenge, this paper introduces a Force Myography (FMG) armband with adjustable sensor positioning, employing a cross-stretch design to ensure uniform force distribution across the armband. A human-machine coupling model was developed to analyze the impact of sensor position variations on data quality. Subsequently, an adjacent feature fusion and reconstruction method is proposed to map interfered signals to normal signals, thereby enhancing gesture recognition performance under interference. Five subjects participated in the experimental evaluation, undergoing tests both under interference and non-interference conditions. The results indicate that the uniform distribution of sensors along the arm improves data quality, achieving an accuracy exceeding 96.6%. The adjacent feature fusion and reconstruction method significantly enhances gesture recognition accuracy under interference, reaching 92.1%, and maintains a 96.2% effectiveness under non-interference. These results demonstrate the effectiveness of the proposed method, advancing the practical application of gesture recognition.
Laboratory automation technology has significantly progressed in various fields, including life sciences, medicine, and materials science. An upgraded mesenchymal stem cell biomanufacturing method has been introduced based on the Automated Cell Manufacturing (Aceman) platform. This method automates quality control requirements and enhances production efficiency in large-scale cell preparation, thus meeting the high demands for both quantity and quality of MSCs in clinical trials and therapeutic applications. Aceman improves the consistency and standardization of the production process and helps meet regulatory and market approval requirements. The article elaborates on the critical steps of the automated cell culture process, including cell seeding, passage, harvesting, and observation. It demonstrates the efficiency and effectiveness of the automated system compared to traditional manual operations through experimental results. The application of this technology is expected to significantly enhance the translation of cell-based therapies from bench to bedside, offering a promising direction for the future of cell therapy and biomanufacturing.
Eliminating the drift along the gravity direction is crucial for Simultaneous Localization and Mapping (SLAM) systems in long-term running. In this paper, a novel ground-constrain-based SLAM method (i.e., BAG-SLAM) is proposed to address the drift issue. BAG-SLAM fuses the ground extraction, LiDAR bundle adjustment (BA), and loop-closure detection methods into the back-end. In the back-end, the extracted ground planes of every keyframe are as constrains of robot poses for factor graph optimization. To deal with the ground distortion of scene, LiDAR-BA-based factors are used to refine the ground structure, correct the parameters of ground plane and remove the outliers. Experimental results on several public and self-collected datasets show that the proposed method effectively reduces the pose errors along the gravity direction. BAG-SLAM realizes the more accurate localization and more consistent map construction.
Access to specialized medical care, particularly in rural or underserved areas, remains a significant challenge, with limited availability of healthcare professionals and diagnostic equipment. Teleoperated robotic systems offer a solution by enabling remote procedures, such as ultrasound examinations, where doctors can operate robotic arms from distant locations. In response to these healthcare challenges, we propose a real-time teleoperation system for controlling a UR5 robot arm using a Geomagic Touch haptic device. This system maps the operator's hand movements to the robot arm, ensuring precise control and providing real-time haptic feedback, which is critical for delicate medical tasks. A force-torque sensor at the robot's end-effector measures the force applied during operation, relaying this information back to the haptic device to simulate tactile sensations. Synchronization between the devices at a frequency of 500 Hz ensures smooth and responsive control. Additionally, a Kalman filter is applied to reduce noise in the force data, enhancing system accuracy. Experimental results demonstrate the system's effectiveness with minimal delay and robust force feedback. This work represents a step forward in bridging the gap between remote healthcare providers and patients, offering a scalable solution for teleoperated medical procedures. All the codes and implementation details for this work can be found in the following repository: https://github.com/primpunn/Master.git.
