Suturing is valuable in many surgical procedures, yet is challenging to implement in mm-scale instruments due to their small end effectors with limited grasp force capabilities. To address this, in this paper, we present a new miniature needle grasper design and use it to demonstrate needle-andthread suturing using concentric tube robots (CTR) for the first time. We compare the new grasper's performance with two conventional endoscopic forceps graspers experimentally. Our design achieved a maximum needle-throwing force of 7.05 N, more than twice that of existing graspers, while maintaining a stable grasp throughout piercing motions. We then apply this technique to the practical surgical application of internal tracheal stent fixation. Our experiments demonstrate that a single suture can increase fixation strength by over an order of magnitude. These results demonstrate the feasibility of enhancing current minimally invasive procedures with thrown sutures within confined anatomical sites.

PURPOSE: Natural orifice surgeries minimize the need for incisions and reduce recovery time compared to open surgery; however, they require higher expertise due to visualization and orientation challenges. To enable reliable scene understanding for surgeon guidance and automation, we propose a perception pipeline that generates semantically informed 3D reconstructions. METHODS: We bring learning-based segmentation, depth estimation, and 3D reconstruction modules together. Through segmentation, we delineate tumor and prostate lobe borders, and through depth estimation and real-time SLAM-based reconstruction, we generate dense 3D point clouds from monocular videos. By propagating 2D labels into 3D space, we create real-time segmented maps of the surgical scenes. Additionally, we use registration with robot poses to solve the scale ambiguity of mapping from monocular images and allow the use of semantically informed real-time reconstructions in robotic surgeries. RESULTS: We achieve sub-millimeter reconstruction accuracy based on average one-sided Chamfer distances, average pose registration RMSE of 0.9 mm, and an estimated scale within 2% of ground truth. Compared to offline Structure-from-Motion baseline, the proposed SLAM-based approach improves processing time while maintaining or improving reconstruction accuracy. Qualitative evaluations show robustness in challenging scenarios, including submerged prostate experiments and cadaver airway explorations. CONCLUSION: We present a modular perception pipeline, integrating semantic segmentation with real-time monocular SLAM for natural orifice surgeries. This pipeline offers a promising solution for scene understanding that can facilitate automation or surgeon guidance. Due to its plug-and-play design and demonstrated generalizability across anatomies and experimental conditions, this framework provides a scalable foundation for future clinical translation.

Concentric tube robots (CTRs) offer dexterous motion at millimeter scales, enabling minimally invasive procedures through natural orifices. This work presents a coordinated model-based resection planner and learning-based retraction network that work together to enable semi-autonomous tissue resection using a dual-arm transurethral concentric tube robot (the Virtuoso). The resection planner operates directly on segmented CT volumes of prostate phantoms, automatically generating tool trajectories for a three-phase median lobe resection workflow: left/median trough resection, right/median trough resection, and median blunt dissection. The retraction network, PushCVAE, trained on surgeon demonstrations, generates retractions according to the procedural phase. The procedure is executed under Level-3 (supervised) autonomy on a prostate phantom composed of hydrogel materials that replicate the mechanical and cutting properties of tissue. As a feasibility study, we demonstrate that our combined autonomous system achieves a 97.1% resection of the targeted volume of the median lobe. Our study establishes a foundation for image-guided autonomy in transurethral robotic surgery and represents a first step toward fully automated minimally-invasive prostate enucleation.

Reliable estimation of surgical needle 3D position and orientation is essential for autonomous robotic suturing, yet existing methods operate almost exclusively under stereoscopic vision. In monocular endoscopic settings, common in transendoscopic and intraluminal procedures, depth ambiguity and rotational symmetry render needle pose estimation inherently ill-posed, producing a multimodal distribution over feasible configurations, rather than a single, well-grounded estimate. We present PinPoint, a probabilistic variational inference framework that treats this ambiguity directly, maintaining a distribution of pose hypotheses rather than suppressing it. PinPoint combines monocular image observations with robot-grasp constraints through analytical geometric likelihoods with closed-form Jacobians. This framework enables efficient Gauss-Newton preconditioning in a Stein Variational Newton inference, where second-order particle transport deterministically moves particles toward high-probability regions while kernel-based repulsion preserves diversity in the multimodal structure. On real needle-tracking sequences, PinPoint reduces mean translational error by 80

PURPOSE:Techniques that minimize dissection of neurovascular structures and the pelvic floor during radical prostatectomy improve perioperative outcomes and functional recovery. Previous groups have tried to reduce dissection by performing a transurethral prostatectomy. However, the vesicourethral anastomosis could not be reliably performed because of the limited instrumentation. We sought to address this with a concentric tube robot (CTR) system designed specifically for performing a transurethral vesicourethral anastomosis after transurethral prostatectomy, in a series of validated phantoms. MATERIALS AND METHODS:We have constructed a CTR system specifically for transurethral surgery. The robot features needle-sized robotic arms that pass through a rigid transurethral endoscope and are composed of telescoping, curved, elastic tubes. By axially rotating these tubes and telescopically extending them, our robot provides surgeons with two small arms that can bend and elongate at the tip of a standard-sized endoscope. This enables suturing within the small lumen of the urethra. We evaluated the CTR in performing the vesicourethral anastomosis in a series of validated phantoms, after transurethral radical prostatectomy was performed manually with a fiber laser. Anastomosis success was evaluated using a leak test. Additionally, we evaluated the surgical time of prostate resection and the suturing time of vesicourethral anastomosis. RESULTS:We performed transurethral radical prostatectomy and subsequent vesicourethral anastomosis in 11 phantoms. A successful anastomosis was performed in 10 out of 11 (91%) experiments. The median time of resection was 19 minutes (IQR: 18-21 minutes). The median suturing time was 103 minutes (IQR: 91-115 minutes). CONCLUSIONS:We demonstrated the use of a CTR system to perform a transurethral vesicourethral anastomosis in a series of experiments using validated phantoms. Our CTR overcomes the main barrier for providing a natural-orifice approach to radical prostatectomy by enabling intraluminal completion of the vesicourethral anastomosis.

Tissue retraction is essential for safe and efficient surgical resection; yet, it remains one of the most repetitive and attention-intensive subtasks in minimally invasive surgery. Surgeons must constantly manipulate the tissue to maintain exposure of critical anatomy while simultaneously performing precise dissection. In this work, we present PushCVAE, a Conditional Variational Autoencoder framework that learns to generate soft-tissue, nonprehensile retraction actions directly conditioned on monocular endoscopic images. Given a singleview input, PushCVAE predicts a three-dimensional contact point and pushing trajectory that safely retracts tissue without requiring explicit depth reconstruction or geometric modeling. We evaluated PushCVAE in seven surgical procedures using the Virtuoso Endoscopy System [1], a dual-arm continuum robot: Central Airway Obstruction (CAO) removal and Benign Prostatic Hyperplasia (BPH) median lobe removal. Out of 55 total retractions across both models, PushCVAE provided adequate retraction tension in 81.8% of pushes and required surgeon intervention only one time. These results represent the first demonstration of autonomous, image-conditioned softtissue retraction during surgical resections, in both airway and prostate surgery. PushCVAE establishes a generalizable framework for learning safe, autonomous surgical subtasks from small-scale demonstrations, paving the way for adaptive, image-driven surgical assistance across anatomically diverse domains.

Monocular depth estimation (MDE) provides a useful tool for robotic perception, but its predictions are often uncertain and inaccurate in challenging environments such as surgical scenes where textureless surfaces, specular reflections, and occlusions are common. To address this, we propose ProbeMDE, a cost-aware active sensing framework that combines RGB images with sparse proprioceptive measurements for MDE. Our approach utilizes an ensemble of MDE models to predict dense depth maps conditioned on both RGB images and a sparse set of known depth measurements obtained via proprioception, where the robot has touched the environment in a known configuration. We quantify predictive uncertainty via the ensemble's variance and measure the gradient of the uncertainty with respect to candidate measurement locations. To prevent mode collapse while selecting maximally informative locations to propriocept (touch), we leverage Stein Variational Gradient Descent (SVGD) over this gradient map. We validate our method in both simulated and physical experiments on central airway obstruction surgical phantoms. Our results demonstrate that our approach outperforms baseline methods across standard depth estimation metrics, achieving higher accuracy while minimizing the number of required proprioceptive measurements.

Focal prostate treatment (aka “male lumpectomy”) has the potential to reduce invasiveness for prostate cancer patients. However, clinical deployment has been impeded by the difficulty of performing surgery through the urethra, which we hypothesize relates to both instrument dexterity and visualization limitations. To evaluate this hypothesis, in this paper we propose a system consisting of an endoscopic robot to enhance dexterity and an image-guidance display updated periodically during surgery based on MRI images. To evaluate the system, in this paper we compare four conditions: unaided manual resection using an endoscope, robot-aided surgery without image guidance, image guidance without the robot, and both robot and image guidance together. We find that while the robot and image guidance improve performance individually, the combination of the two provides the greatest improvement.

Central airway obstruction (CAO) can disrupt normal breathing and pose significant risks, with treatments often involving surgical removal. Accurate image segmentation is crucial for identifying the target region to assist surgeons or robotic surgery system during operations. Although manual annotation is the gold standard, it is time-consuming and subjective, making a robust automated segmentation algorithm desirable. In this paper, we propose an automated deep learning framework for CAO segmentation. As a benchmark study, we build a custom CAO phantom model and acquire endoscopy videos. We then establish inter-rater variability. To assess the effectiveness of the proposed method, a 4-fold cross-validation is performed with The Dice score as an evaluation metric. The proposed CAO segmentation method yields mean binary Dice scores of 0.95 and 0.89 against annotations from two human raters, respectively, compared to an inter-rater variability Dice score of 0.88. This indicates that the proposed framework can provide robust CAO segmentation from endoscopic video.

Surgical automation requires precise guidance and understanding of the scene. Current methods in the literature rely on bulky depth cameras to create maps of the anatomy; however, this does not translate well to space-limited clinical applications. Monocular cameras are small and allow minimally invasive surgeries in tight spaces, but additional processing is required to generate 3D scene understanding. We propose a 3D mapping pipeline that uses only RGB images to create segmented point clouds of the target anatomy. To ensure the most accurate reconstruction, we compare different structure from motion algorithms' performance on mapping the central airway obstructions, and test the pipeline on a downstream task of tumor resection. In several metrics, including post-procedure percentage tissue charring, our pipeline performs comparably to RGB-D cameras and, in some cases, even surpasses their downstream task performance. These promising results demonstrate that automation guidance can be achieved in minimally invasive procedures with monocular cameras. This study is a step toward the complete autonomy of surgical robots.

Concentric tube robots delivered through endoscopes have thus far been deployed approximately straight ahead of the endoscope's tip, which requires relatively low curvatures and strains, and a single fixed view angle. However, in tight spaces in the body (such as the interior of the uterus), it is often desirable to work close beside the endoscope tip, approximately perpendicular to the endoscope axis. Doing this requires two advancements: (1) a way to angle the camera, ideally without physically moving it, to prevent collisions with anatomy or other tools in the constrained space, and (2) the ability to reach points approximately perpendicular to the endoscope axis, that are close to its tip. We address the first challenge by integrating a variable view angle endoscope designed for arthroscopy. We address the second with highly curved concentric tube robots, designed to undergo higher strains than have previously been reported in the literature. We experimentally demonstrate working sideways from the endoscope tip with such a system by tracing the periphery of simulated uterine fibroids at multiple angles. We also demonstrate the use of electrosurgery to cut around the periphery of a simulated lesion made from animal tissues, while working in a direction approximately perpendicular to the endoscope axis.

We seek to enable transurethral focal resection of prostate tumors via enhanced robotic dexterity, combined with MRI image guidance. Our approach is designed to reduce invasiveness compared to traditional transabdominal radical prostatectomy, with the goal of reducing rates of impotence, incontinence, and other complications. MRI guidance is useful in focal resections since many prostate tumors look optically no different from the surrounding prostate, but can be seen in preoperative MRI images. To guide these procedures in a practical clinical setting, we envision using the new generation of low-field scanners emerging on the market, in conjunction with endoscope-deployed concentric tube robots. These scanners would provide periodic intraoperative imaging to which one would register the preoperative high-field images in which the tumor is visualized. In this paper, we conduct a feasibility study to explore whether our robot and image guidance system can help guide the resection of tumors that are invisible optically. In our experiments, surgeons used our robot with periodic highfield MRI image updates, to resect an optically invisible tumor in an anthropomorphic prostate phantom. We show how the availability of imaging information informed both the direction and depth with which surgeons chose to resect the tumor.

For transendoscopic concentric tube robots to have high accuracy with respect to endoscope-derived information, they must be calibrated with respect to the endoscope. We propose to accomplish this by defining rays from a monocular endoscope to a set of robot tip locations. Minimizing the error between the rays and tip positions provides a means of calibration. This is a first step toward enabling the robot to use endoscope-derived information in future applications that require accuracy, such as image guidance or automation.

As surgical robotics are made progressively smaller, and their actuation systems simplified, the opportunity arises to re-evaluate how we integrate them into operating room workflows. Over the past few years, several research groups have shown that robots can be made so small and light that they can become hand-held tools, in contrast to the prevailing commercial paradigm of surgical robots being large multi-arm floor-mounted systems that must be remotely teleoperated. This hand-held paradigm enables robots to fit much more seamlessly into existing clinical workflows, and as such, these new robots need to be paired with similarly compact user interfaces. It also gives rise to a new area of user interface research, exploring how the surgeon can simultaneously control the position and orientation of the overall system, while also simultaneously controlling small robotic manipulators that maneuver dexterously at the tip. In this paper, we compare an onboard user interface mounted directly to the robotic platform against the traditional offboard user interface positioned away from the robot. In the latter, the surgeon positions the robot, and a support arm holds it in place while the surgeon operates the manipulators using the offboard surgeon console. The surgeon can move back and forth between the robot and the console as often as desired. Three experiments were conducted, and results show that the onboard interface enables statistically significantly faster performance in a point-touching task performed in a virtual environment.

This conference presentation was prepared for SPIE Medical Imaging, 2023.

Towards reducing the invasiveness of radical prostatectomy, we have designed a robotic system for performing it transurethrally. Suturing to attach the urethra to the bladder (i.e. anastomosis) after prostate removal is the most challenging part of the procedure, and has previously been demonstrated robotically only in synthetic phantoms. In this paper, we present initial experiments in biological tissues using ex vivo squid tissue embedded in an anthropomorphic phantom made using 3D printing and silicone casting. We successfully performed running sutures with our robotic system, fastening the urethra and the bladder to one another.

In this paper, we present a study on the viability of fabricating Concentric Tube Robots using Multi Jet Fusion (MJF) of Nylon-12, a type of elastic polymer commonly used in additive manufacturing. We note that Nylon-12 was already evaluated for the purpose of building CTRs in prior work, but fabrication was performed with Selective Laser Sintering (SLS), which produced unsatisfactory results. Our study is the first study to evaluate the suitability of MJF to 3Dprint CTRs.

Some of the earliest clinical motivations for concentric tube robots (CTRs) involved procedures which are often accomplished via electrosurgery [1], [2]. However, the development of electrosurgery in physical prototypes was initially left to future work, as early research focused on mechanics-based models and model-based control methods [3], [4]. In recent years, monopolar electrosurgery has been delivered through CTRs in physical prototypes [5], [6]. Since the ground is attached elsewhere on the patient’s body, monopolar tools typically simply require an exposed metal tip to cut. Bipolar tools, in contrast, carry two electrodes, are traditionally made in the shape of a forceps with each jaw containing one of the electrodes. Bipolar electrosurgery has several general advantages over monopolar, in applications where it can be used, including more localized heating and lower voltages, which lead to a lower risk of injury to the patient [7]. Motivated by these advantages, a bipolar electrosurgery forceps has been designed for delivery through for CTRs [8]. In this paper we propose an alternate approach in which the CTR itself acts as one of the active electrodes, with the other delivered through the CTR’s central lumen. We demonstrate teleoperated tissue cutting using a concentric tube robot equipped with this approach to electrosurgery.