Central airway obstruction (CAO) can disrupt normal breathing and pose significant risks, with treatments often involving surgical removal. Accurate image segmentation is crucial for identifying the target region to assist surgeons or robotic surgery system during operations. Although manual annotation is the gold standard, it is time-consuming and subjective, making a robust automated segmentation algorithm desirable. In this paper, we propose an automated deep learning framework for CAO segmentation. As a benchmark study, we build a custom CAO phantom model and acquire endoscopy videos. We then establish inter-rater variability. To assess the effectiveness of the proposed method, a 4-fold cross-validation is performed with The Dice score as an evaluation metric. The proposed CAO segmentation method yields mean binary Dice scores of 0.95 and 0.89 against annotations from two human raters, respectively, compared to an inter-rater variability Dice score of 0.88. This indicates that the proposed framework can provide robust CAO segmentation from endoscopic video.

Steerable needles are minimally invasive devices that enable novel medical procedures by following curved paths to avoid critical anatomical obstacles. We introduce a new start pose robustness metric for steerable needle motion plans. A steerable needle deployment typically consists of a physician manually placing a steerable needle at a precomputed start pose on the surface of tissue and handing off control to a robot, which then autonomously steers the needle through the tissue to the target. The handoff between humans and robots is critical for procedure success, as even small deviations from a planned start pose change the steerable needle's reachable workspace. Our metric is based on a novel geometric method to efficiently compute how far the physician can deviate from the planned start pose in both position and orientation such that the steerable needle can still reach the target. We evaluate our metric through simulation in liver and lung scenarios. Our evaluation shows that our metric can be applied to plans computed by different steerable needle motion planners and that it can be used to efficiently select plans with large safe start regions.

Tendon-driven continuum robots have been gaining popularity in medical applications due to their ability to curve around complex anatomical structures, potentially reducing the invasiveness of surgery. However, accurate modeling is required to plan and control the movements of these flexible robots. Physics-based models have limitations due to unmodeled effects, leading to mismatches between model prediction and actual robot shape. Recently proposed learning-based methods have been shown to overcome some of these limitations but do not account for hysteresis, a significant source of error for these robots. To overcome these challenges, we propose a novel deep decoder neural network that predicts the complete shape of tendon-driven robots using point clouds as the shape representation, conditioned on prior configurations to account for hysteresis. We evaluate our method on a physical tendon-driven robot and show that our network model accurately predicts the robot's shape, significantly outperforming a state-of-the-art physics-based model and a learning-based model that does not account for hysteresis.

Prior models of continuously flexible robots typically assume uniform stiffness, and in this paper we relax this assumption. Geometrically varying stiffness profiles provide additional design freedom to influence the motions and workspaces of continuum robots. These results are timely, because with recent rapid advancements in multimaterial additive manufacturing techniques, it is now straightforward to create more complex stiffness profiles in robots. The key insight of this paper is to project forces and moments applied to the robot onto its center of stiffness (i.e. the Young’s modulus-weighted center of each cross section). We show how the center of stiffness can be thought of as analogous to a “precurved backbone” in a robot with uniform stiffness. This analogy enables a large body of prior work in Cosserat Rod modeling of such robots to be applied directly to those with stiffness variations. We experimentally validate this approach using multimaterial, soft, tendon-actuated robots. Lastly, to illustrate how these results can be used in practice, we investigate how stiffness variation can improve performance in a neurosurgical task.

Concentric push-pull robots delivered through flexible endoscopes work best if their laser-cut transmission tubes have high axial stiffness, high torsional stiffness, and low bending stiffness. This paper simultaneously addresses all three output stiffness values in the transmission design problem, explicitly considering axial stiffness, whereas prior work on laser-cut tube design has focused on the bending/torsional stiffness ratio. We demonstrate an inherent trade-off present in existing laser-cut patterns: it is difficult to simultaneously achieve high axial stiffness and low bending stiffness because these properties are very tightly correlated. To break this correlation and design all three stiffness independently, we propose a new type of laser material removal pattern that leverages local stiffness asymmetry ( E I x ≠ E I y ) in discrete bending segments separated by segments of solid tube. These discrete asymmetric segments are then rifled down the tube to achieve global stiffness symmetry. We parameterize the design and provide a study of the properties through finite-element analysis. We also consider the effect of interference between the tubes when the discrete segments are not aligned. Results show that our discrete asymmetric segment concept can achieve high axial stiffness and torsional stiffness better than previously suggested laser patterns while maintaining equally low bending stiffness. We also experimentally validated the proposed design's properties and actuation performance with professionally manufactured prototype Nitinol tubes for use in an endoscopic robot system.

Conventional soft robots are designed with constant, passive stiffness properties, based on desired motion capabilities. The ability to encode two fundamentally different stiffness characteristics promises to enable a single robot to be optimized for multiple divergent tasks simultaneously and this has been previously proposed with a variety of approaches including jamming-based designs. In this paper, we propose phase-changing metallic spines of various geometries to independently control specific directional stiffness parameters of soft robots, changing how they respond to their actuation inputs and external loads. We fabricate spine-like structures using a low melting point alloy (LMPA), enabling us to switch on and off the effects of the stiff metal structure of the overall robot's stiffness during use. Changing soft robot morphology in this manner will enable these robots to adapt to environments and tasks that require divergent motion and force/moment application capabilities.

For transendoscopic concentric tube robots to have high accuracy with respect to endoscope-derived information, they must be calibrated with respect to the endoscope. We propose to accomplish this by defining rays from a monocular endoscope to a set of robot tip locations. Minimizing the error between the rays and tip positions provides a means of calibration. This is a first step toward enabling the robot to use endoscope-derived information in future applications that require accuracy, such as image guidance or automation.

Continuum robots navigate narrow, winding passageways while safely and compliantly interacting with their environments. Sensing the robot's shape under these conditions is often done indirectly, using a few coarsely distributed (e.g., strain or position) sensors combined with the robot's mechanics-based model. More recently, given high-fidelity shape data, external interaction loads along the robot have been estimated by solving an inverse problem on the mechanics model of the robot. In this article, we argue that since shape and force are fundamentally coupled, they should be estimated simultaneously using a statistically principled approach. We accomplish this by applying continuous-time batch estimation directly to the arclength domain. A general continuum robot model serves as a statistical prior that is fused with discrete, noisy measurements taken along the robot's backbone. The result is a continuous posterior containing both shape and load functions of arclength, as well as their uncertainties. We first test the approach with a Cosserat rod, i.e., the underlying modeling framework that is the basis for a variety of continuum robots. We verify our approach numerically using distributed loads with various sensor combinations. Next, we experimentally validate shape and external load errors for highly concentrated force distributions (point loads). Finally, we apply the approach to a tendon-actuated continuum robot demonstrating applicability to more complex actuated robots.

Goal: We present a new framework for in vivo image guidance evaluation and provide a case study on robotic partial nephrectomy. Methods: This framework (called the “bystander protocol”) involves two surgeons, one who solely performs the therapeutic process without image guidance, and another who solely periodically collects data to evaluate image guidance. This isolates the evaluation from the therapy, so that in-development image guidance systems can be tested without risk of negatively impacting the standard of care. We provide a case study applying this protocol in clinical cases during robotic partial nephrectomy surgery. Results: The bystander protocol was performed successfully in 6 patient cases. We find average lesion centroid localization error with our IGS system to be 6.5 mm in vivo compared to our prior result of 3.0 mm in phantoms. Conclusions : The bystander protocol is a safe, effective method for testing in-development image guidance systems in human subjects.

Understanding elastic instability has been a recent focus of concentric tube robot research. Modeling advances have enabled prediction of when instabilities will occur and produced metrics for the stability of the robot during use. In this paper, we show how these metrics can be used to resolve redundancy to avoid elastic instability, opening the door for the practical use of higher curvature designs than have previously been possible. We demonstrate the effectiveness of the approach using a three-tube robot that is stabilized by redundancy resolution when following trajectories that would otherwise result in elastic instabilities. We also show that it is stabilized when teleoperated in ways that otherwise produce elastic instabilities. Lastly, we show that the redundancy resolution framework presented here can be applied to other control objectives useful for surgical robots, such as maximizing or minimizing compliance in desired directions.

Soft robots have garnered great interest in recent years due to their ability to navigate complex environments and enhance safety during unplanned collisions. However, their softness typically limits the forces they can apply and payloads they can carry, compared to traditional rigid-link robots. In this paper we seek to create a hybrid manipulator that can switch between a state in which it acts as a soft robot, and a state in which it has a series of selectively stiffenable links. The latter state, accomplished by solidifying chambers of low melting point metal alloy within the robot, is in some ways analogous to a traditional rigid-link manipulator. It also has the added benefit that each “link” can be set to a desired straight or curved shape before solidification and re-shaped when desired. Thermoelectric heat pumps enable local heating and cooling of the alloy, and tendons running along the robot enable actuation. Using a simple two-link prototype, we illustrate how alloy melting and solidification can be used to modify the robot's workspace and payload capacity.

Epilepsy affects more than 50 million people world- wide, afflicting patients with debilitating seizures. While antiepileptic drugs are available, 20-40% of patients remain medically refractory, leaving surgical interven- tion as the remaining option [1]. Hippocampal resec- tion is the gold standard surgical therapy, while laser interstitial thermal therapy (LITT) offers a minimally invasive alternative. Current LITT interventions involve delivery of thermal energy via a straight laser probe through a burr hole in the back of the skull under the guidance of magnetic resonance imaging (MRI). LITT has been associated with lower seizure freedom rates which we hypothesize is related to the use of straight line trajectories in a naturally curved structure. Our previous work proposed a percutaneous approach whereby a helically precurved needle is deployed through the foramen ovale to ablate along the hippocampal midline gripper capable of directly grasping needles. (c) De- maximizing tissue coverage [2]. This approach demands a safe, compact, and accurate actuation system capable of operating within the MRI scanner.

Mechanical metamaterials are microscale patterned structures that are designed to have specific mechanical properties at a macro-scale that are atypical of natural materials. Robotic manipulators composed of these materials can exhibit deformation and motion capabilities that can be customized and easily fabricated. However, as of now, the motion capability of such manipulators are encoded in their physical composition and cannot be changed. This paper presents multimodal metamaterial-based robot prototypes which can switch between the behaviors found in two different metamaterials. Two such robots are explored, a bending/shearing robot and a bending/twisting robot. The robot design is described in detail, including how the robots toggle between behavior modes via mechanical actuation of a sliding rod insert. Multi-modal robots are compared to their single-mode equivalents to characterize their capabilities. The single-mode behaviors are largely preserved in the multi-modal innovations. The multi-modal prototypes also demonstrate variable rigidity. We discuss the feasibility of using robots of this design as part of a robotic surgical system.