Communication latency in long-distance telerobotic surgery poses a critical safety risk, particularly in high-precision procedures like retinal surgery where tool overshoots can cause irreversible patient injury. This paper introduces the Latency-Aware Telemonitoring for Injection in Ophthalmic Surgery (LATIOS) framework, which enhances safety by adaptively scaling the surgical robot's velocity along the critical axis of tool insertion. Our core contribution is a control algorithm that dynamically modulates the velocity scaling factor based on two real-time, coupled variables: the tool-tip-to-retina distance, estimated via a non-contact, shadow-based method, and the measured communication delay. We validated this system in a transatlantic user study where six participants in North America teleoperated a surgical robot in Europe to perform a series of simulated retinal punctuation tasks. The results demonstrate that LATIOS provides a statistically significant reduction in applied punctuation forces compared to constant control (p = 0.022). This objective safety improvement is achieved through a deliberate safety-efficiency trade-off, with the system enforcing a more cautious pace under high-latency conditions. Our work presents a robust, context-aware safety framework that addresses a key barrier to the clinical adoption of long-distance telerobotic surgery.
Age-related macular degeneration (AMD) and diabetic macular edema (DME) are leading causes of vision loss, and optical coherence tomography (OCT) is the standard modality for detecting and monitoring the subtle lesions that drive treatment decisions. Most deep-learning segmentation work for OCT is validated only in-domain, leaving generalization to clinical data collected under different acquisition protocols largely untested. This work develops and systematically ablates four lesion-segmentation pipelines – 2D and 3D variants for AMD and DME – reaching Dice scores of 0.76 to 0.82 with strong volumetric and surface calibration (r vol, r surf greater than or equal to 0.97 across all four pipelines) on an in-domain validation set. The ablation process establishes a full-volume, calibration-aware adoption standard that catches mechanisms an ordinary slice-level evaluation would keep, and identifies ensemble composition as the most consistent driver of improvement. To test generalization, the models are evaluated on OLIVES, an external clinical cohort with no lesion-level ground truth, using a proxy-metric framework built around biomarker AUROC, central subfield thickness (CST) correlation, and longitudinal concordance. Predictions track clinical biomarkers outside the training distribution, though less strongly than in-domain – evidence for, not validation of, automated lesion-burden tracking as a clinical tool.
In the realm of ophthalmic imaging, accurate vascular segmentation is paramount for diagnosing and managing various eye diseases. Contemporary deep learning-based vascular segmentation models rival human accuracy but still face substantial challenges in accurately segmenting minuscule blood vessels in neural network applications. In this paper, we propose a novel network (KaLDeX) for vascular segmentation leveraging a Kalman filter based linear deformable cross attention (LDCA) module integrated within a UNet++ framework. Our approach is based on two key components: Kalman filter (KF) based linear deformable convolution (LD) and cross-attention (CA) modules. The LD module is designed to adaptively adjust the focus on thin vessels that might be overlooked in standard convolution. The CA module improves the global understanding of vascular structures by aggregating the detailed features from the LD module with the high-level features from the UNet++ architecture. Finally, we adopt a topological loss function based on persistent homology to constrain the topological continuity of the segmentation. The proposed method is evaluated on retinal fundus image datasets (DRIVE, CHASE BD1, and STARE) as well as the 3 mm and 6 mm of the OCTA-500 dataset, achieving an average accuracy (ACC) of 97.25%, 97.77%, 97.85%, 98.89%, and 98.21%, respectively.
As surgical AI transitions from pixel-level detection to complex reasoning, Scene Graphs (SGs) offer the structured, relational representations necessary to decode dynamic surgical environments. This PRISMA-ScR-guided scoping review systematically maps the evolving landscape of SG research in surgery, analyzing 52 primary studies to chart applications and methodological shifts. Our analysis reveals rapid growth, yet uncovers a critical 'data divide': internal-view research (e.g., triplet recognition from endoscopic video) accounts for 79% of studies and predominantly uses real-world 2D video, while external-view operating room modeling relies heavily on simulated data. Methodologically, we identify a decisive shift from foundational graph neural networks to specialized foundation models and generative AI, which together now account for approximately 50% of research in 2025. Crucially, our synthesis suggests that Scene Graphs are evolving from simple descriptors into essential 'neuro-symbolic guardrails', providing the structured, verifiable intermediate representation needed to prevent hallucinations in increasingly autonomous Surgical Foundation Models. Despite this promise, a major translational gap remains: none (0/52) of the reviewed studies have proceeded to prospective clinical validation. We conclude that bridging this gap requires moving beyond standard computer vision metrics; we therefore propose the 'Validation Trinity' - prioritizing Semantic Query Success, Latency-Aware Accuracy, and Safety-Critical Recall - as the necessary evaluation framework to bring graph-based surgical AI into clinical practice.
Retinal Surgery Robotics is a rapidly emerging field that offers enhanced precision by overcoming human tremors. A key trend of these robotic designs is toward more compact and lightweight structures for improved positioning accuracy and precise force delivery. However, this compactness sacrifices the robot's working volume, making it difficult for ophthalmic surgeons to intuitively assess if retinal targets are accessible by the surgical tool tip. This paper proposes a methodology for visualizing the actual accessible area in the microscopic view to provide surgeons with an intuitive visual guide of the tool's reach, reducing uncertainty and streamlining extraocular robotic maneuvers. We validated this method on a commercial phantom with a surgical robot system, achieving <=1.0 deg error for 83.3% of tested points across four retinal subareas and demonstrating its clinical potential.
In ophthalmic surgery, surgeons or robots manipulate a light probe and an instrument around two separated trocars following sclerotomy to achieve orbital control for eyeball pose adjustment and subsequent surgical tasks referring to microscope frames. However, current methods face significant challenges in directly extracting the eyeball pose from real-time microscope frames due to the limited microscope perspective and the darkened operating room (OR). This paper decomposes eyeball rotations only along the x and y axes. Then, a method of calculating eyeball poses using eyeball geometry and microscopic trocar positions is presented. This method is tested by simulation and a phantom system with current [2.0, 2.8] degree error, providing assistant intraoperative eyeball status in the dark OR with extended method discussions.
Exudative (wet) age-related macular degeneration (AMD) is a leading cause of vision loss in older adults, typically treated with intravitreal injections. Emerging therapies, such as subretinal injections of stem cells, gene therapy, small molecules and RPE cells require precise delivery to avoid damaging delicate retinal structures. Robotic systems can potentially offer the necessary precision for these procedures. This paper presents a novel approach for motion compensation in robotic subretinal injections, utilizing real time Optical Coherence Tomography (OCT). The proposed method leverages B5-scans, a rapid acquisition of small-volume OCT data, for dynamic tracking of retinal motion along the Z-axis, compensating for physiological movements such as breathing and heartbeat. Validation experiments on ex vivo porcine eyes revealed challenges in maintaining a consistent tool-to-retina distance, with deviations of up to 200 μm for 100 μm amplitude motions and over 80 μm for 25 μm amplitude motions over one minute. Subretinal injections faced additional difficulties, with phase shifts causing the needle to move off-target and inject into the vitreous. These results highlight the need for improved motion prediction and horizontal stability to enhance the accuracy and safety of robotic subretinal procedures.
Several robotic frameworks have been recently developed to assist ophthalmic surgeons in performing complex vitreoretinal procedures such as subretinal injection of advanced therapeutics. These surgical robots show promising capabilities; however, most of them have to limit their working volume to achieve maximum accuracy. Moreover, the visible area seen through the surgical microscope is limited and solely depends on the eye posture. If the eye posture, trocar position, and robot configuration are not correctly arranged, the instrument may not reach the target position, and the preparation will have to be redone. Therefore, this paper proposes the optimization framework of the eye tilting and the robot positioning to reach various target areas for different patients. Our method was validated with an adjustable phantom eye model, and the error of this workflow was 0.13 +/- 1.65 deg (rotational joint around Y axis), -1.40 +/- 1.13 deg (around X axis), and 1.80 +/- 1.51 mm (depth, Z). The potential error sources are also analyzed in the discussion section.
Recent advancements in foundation models, such as the Segment Anything Model (SAM), have significantly impacted medical image segmentation, especially in retinal imaging, where precise segmentation is vital for diagnosis. Despite this progress, current methods face critical challenges: 1) modality ambiguity in textual disease descriptions, 2) a continued reliance on manual prompting for SAM-based workflows, and 3) a lack of a unified framework, with most methods being modality- and task-specific. To overcome these hurdles, we propose CLIP-unified Auto-Prompt Segmentation (), a novel method for unified segmentation across diverse tasks and modalities in retinal imaging. Our approach begins by pre-training a CLIP-based image encoder on a large, multi-modal retinal dataset to handle data scarcity and distribution imbalance. We then leverage GroundingDINO to automatically generate spatial bounding box prompts by detecting local lesions. To unify tasks and resolve ambiguity, we use text prompts enhanced with a unique "modality signature" for each imaging modality. Ultimately, these automated textual and spatial prompts guide SAM to execute precise segmentation, creating a fully automated and unified pipeline. Extensive experiments on 12 diverse datasets across 11 critical segmentation categories show that CLAPS achieves performance on par with specialized expert models while surpassing existing benchmarks across most metrics, demonstrating its broad generalizability as a foundation model.
Capsulorhexis is challenging in cataract surgery, since the size, centering, and circularity of the capsule are important. Those indicators are closely related to the subsequent step of phacoemulsication and the postoperative position of the intraocular lens. It takes 3-5 years for a resident to practice, while the occurrence of decient capsulorhexis is still inevitable. This paper proposes a robotic system to automate Continuous Curvilinear Capsulorhexis (CCC) in cataract surgery. A typical ophthalmic microscope system and a triaxial force sensor are utilized to guide the robot system with a force-vision method. The constraint of a Remote Center of Motion (RCM) is designed to perform the surgery route. The experimental results on exvivo porcine eyes show our autonomous method can achieve a satisfactory 6mm capsule. With an average centering deviation below 7:6% and circularity of 0:993, the consistency of the capsulorhexis is comparable to a surgeon-made one.
Advancements in robot-assisted eye surgery have significantly enhanced precision in delicate procedures by enabling controlled movement of surgical instruments through a small entry point (trocar) placed on the outer layer of the eye (sclera). However, before these procedures can begin, the robot must first be prepared, draped, and calibrated. Then, it is positioned near the patient, where precise patient alignment is performed. Finally, the system ensures accurate alignment between the surgical instrument and the trocar, a critical step for safe and effective instrument insertion and surgical execution. Despite these advancements, integrating robotic systems into clinical practice remains challenging. The current manual preparation and alignment process is time-consuming and error-prone, emphasizing the need for a more automated and adaptive solution. Patient alignment time has been identified as a key factor differentiating robot-assisted and manual surgeries in overall procedure duration, with robot preparation and patient alignment extending surgery time by an average of 51 +/- 6 minutes compared to manual methods. To address these challenges, this paper introduces a multi-component robotic system combined with an image-guided patient alignment methodology designed to reduce preparation and alignment time in robotic eye surgery while maintaining surgical efficiency. The robotic system consists of a 2D-Cart carrying a 4-degree-of-freedom (DOF) frame robot holding a 5-DOF micro-manipulator. It utilizes an image-guided, multi-stage alignment approach, enabling the system to efficiently be localized and stabilized in an optimal position beside the surgical bed and ensuring precise and adaptive positioning relative to the patient in the surgical draped condition. The proposed end-to-end robotic system preparation and patient alignment are validated through a user study involving eight medical experts in a simulated surgical environment using a draped phantom eye. The results demonstrate successful robot-patient alignment in all cases, with the additional setup time significantly reduced to 272 +/- 84 seconds, highlighting the high precision of the system and clinical feasibility. To further assess the effectiveness of the image-guided alignment in real surgical settings, eye detection experiments are conducted on five patients under surgical draped conditions, achieving 90% eye detection accuracy and 75% iris detection accuracy. This study introduces a robust, semi-automated approach to robot-patient alignment, offering a more efficient and adaptable alternative to current manual methods by reducing setup time and enhancing surgical accuracy.INDEX TERMS Medical robots and systems, surgical robotics, planning, clinical integration in medical robotics.
Significant advancements in AI-driven multimodal medical image diagnosis have led to substantial improvements in ophthalmic disease identification in recent years. However, acquiring paired multimodal ophthalmic images remains prohibitively expensive. While fundus photography is simple and cost-effective, the limited availability of OCT data and inherent modality imbalance hinder further progress. Conventional approaches that rely solely on fundus or textual features often fail to capture fine-grained spatial information, as each imaging modality provides distinct cues about lesion predilection sites. In this study, we propose a novel unpaired multimodal framework that utilizes extensive OCT-derived spatial priors to dynamically identify predilection sites, enhancing fundus image-based disease recognition. Our approach bridges unpaired fundus and OCTs via extended disease text descriptions. Initially, we employ contrastive learning on a large corpus of unpaired OCT and fundus images while simultaneously learning the predilection sites matrix in the OCT latent space. Through extensive optimization, this matrix captures lesion localization patterns within the OCT feature space. During the fine-tuning or inference phase of the downstream classification task based solely on fundus images, where paired OCT data is unavailable, we eliminate OCT input and utilize the predilection sites matrix to assist in fundus image classification learning. Extensive experiments conducted on 9 diverse datasets across 28 critical categories demonstrate that our framework outperforms existing benchmarks.
Robotic platforms provide consistent and precise tool positioning that significantly enhances retinal microsurgery. Integrating such systems with intraoperative optical coherence tomography (iOCT) enables image-guided robotic interventions, allowing autonomous performance of advanced treatments, such as injecting therapeutic agents into the subretinal space. However, tissue deformations due to tool-tissue interactions constitute a significant challenge in autonomous iOCT-guided robotic subretinal injections. Such interactions impact correct needle positioning and procedure outcomes. This paper presents a novel method for autonomous subretinal injection under iOCT guidance that considers tissue deformations during the insertion procedure. The technique is achieved through real-time segmentation and 3D reconstruction of the surgical scene from densely sampled iOCT B-scans, which we refer to as B5-scans. Using B5-scans we monitor the position of the instrument relative to a virtual target layer between the ILM and RPE. Our experiments on ex-vivo porcine eyes demonstrate dynamic adjustment of the insertion depth and overall improved accuracy in needle positioning compared to prior autonomous insertion approaches. Compared to a 35% success rate in subretinal bleb generation with previous approaches, our method reliably created subretinal blebs in 90% our experiments. The source code and data used in this study are publicly available on GitHub.
Clear corneal incision, a challenging step in cataract surgery, is important to the overall quality of the surgery. New surgeons usually spend one full year trying to perfect their incision, but even after such rigorous training deficient incisions can still occur. This article proposes an improved autonomous robotic system for this self-sealing incision. An imaging system with a monocular camera is used to capture the surgical scene and determine the position of the scalpel. Kinematics with a remote center of motion is designed for a multiaxes robot with a force sensor and the new approach of force-vision fusion is five times faster to perform a similar incision. A four-stage initial route and a five parts surgical route are designed and evaluated. The experimental results on exvivo porcine eyes show the autonomous clear corneal incision has a stricter three-plane structure than a surgeon-made incision, which is closer to the ideal incision.
Recent advancements in age-related macular degeneration treatments necessitate precision delivery into the subretinal space, emphasizing minimally invasive procedures targeting the retinal pigment epithelium (RPE)-Bruch’s membrane complex without causing trauma. Even for skilled surgeons, the inherent hand tremors during manual surgery can jeopardize the safety of these critical interventions. This has fostered the evolution of robotic systems designed to prevent such tremors. These robots are enhanced by FBG sensors, which sense the small force interactions between the surgical instruments and retinal tissue. To enable the community to design algorithms taking advantage of such force feedback data, this paper focuses on the need to provide a specialized dataset, integrating optical coherence tomography (OCT) imaging together with the aforementioned force data. We introduce a unique dataset, integrating force sensing data synchronized with OCT B-scan images, derived from a sophisticated setup involving robotic assistance and OCT integrated microscopes. Furthermore, we present a neural network model for image-based force estimation to demonstrate the dataset’s applicability.
In ophthalmic surgeries, such as vitreoretinal operations, surgeons rely on imaging systems, primarily microscopes, for real-time instrument monitoring and motion planning. However, novice surgeons struggle to extract 3D instrument positions from 2D microscope frames, necessitating extensive trial-and-error experience with the background that additional imaging modalities such as iOCT remain inaccessible in most operating rooms. Targeting intraocular assessment within the current surgical setup, this paper presents an image-based pose estimation method to obtain real-time instrument tip positions in a standard 12mm-radius spherical eyeball model, which links floating instruments with on-the-retinal objects based on the intraocular shadowing principle. We validate this estimation method in a Unity simulator and verify its depth estimation capability using a specially designed eyeball phantom. Both simulator and phantom experiments demonstrate an average needle-tip estimation error within [1.0, 2.0] mm using only 2D microscope frames.
The utilization of longitudinal datasets for glaucoma progression prediction offers a compelling approach to support early therapeutic interventions. Predominant methodologies in this domain have primarily focused on the direct prediction of glaucoma stage labels from longitudinal datasets. However, such methods may not adequately encapsulate the nuanced developmental trajectory of the disease. To enhance the diagnostic acumen of medical practitioners, we propose a novel diffusion-based model to predict prospective images by extrapolating from existing longitudinal fundus images of patients. The methodology delineated in this study distinctively leverages sequences of images as inputs. Subsequently, a time-aligned mask is employed to select a specific year for image generation. During the training phase, the time-aligned mask resolves the issue of irregular temporal intervals in longitudinal image sequence sampling. Additionally, we utilize a strategy of randomly masking a frame in the sequence to establish the ground truth. This methodology aids the network in continuously acquiring knowledge regarding the internal relationships among the sequences throughout the learning phase. Moreover, the introduction of textual labels is instrumental in categorizing images generated within the sequence. The empirical findings from the conducted experiments indicate that our proposed model not only effectively generates longitudinal data but also significantly improves the precision of downstream classification tasks.
AI-based vascular segmentation is becoming increasingly common in enhancing the screening and treatment of ophthalmic diseases. Deep learning structures based on U-Net have achieved relatively good performance in vascular segmentation. However, small blood vessels and capillaries tend to be lost during segmentation when passed through the traditional U-Net downsampling module. To address this gap, this paper proposes a novel Kalman filter based Linear Deformable Diffusion (KLDD) model for retinal vessel segmentation. Our model employs a diffusion process that iteratively refines the segmentation, leveraging the flexible receptive fields of deformable convolutions in feature extraction modules to adapt to the detailed tubular vascular structures. More specifically, we first employ a feature extractor with linear deformable convolution to capture vascular structure information form the input images. To better optimize the coordinate positions of deformable convolution, we employ the Kalman filter to enhance the perception of vascular structures in linear deformable convolution. Subsequently, the features of the vascular structures extracted are utilized as a conditioning element within a diffusion model by the Cross-Attention Aggregation module (CAAM) and the Channel-wise Soft Attention module (CSAM). These aggregations are designed to enhance the diffusion model’s capability to generate vascular structures. Experiments are evaluated on retinal fundus image datasets (DRIVE, CHASE DB1) as well as the 3mm and 6mm of the OCTA-500 dataset, and the results show that the diffusion model proposed in this paper outperforms other methods.
Background and Objective: In the realm of ophthalmic imaging, accurate vascular segmentation is paramount for diagnosing and managing various eye diseases. Contemporary deep learning-based vascular segmentation models rival human accuracy but still face substantial challenges in accurately segmenting minuscule blood vessels in neural network applications. Due to the necessity of multiple downsampling operations in the CNN models, fine details from high-resolution images are inevitably lost. The objective of this study is to design a structure to capture the delicate and small blood vessels. Methods: To address these issues, we propose a novel network (KaLDeX) for vascular segmentation leveraging a Kalman filter based linear deformable cross attention (LDCA) module, integrated within a UNet++ framework. Our approach is based on two key components: Kalman filter (KF) based linear deformable convolution (LD) and cross-attention (CA) modules. The LD module is designed to adaptively adjust the focus on thin vessels that might be overlooked in standard convolution. The CA module improves the global understanding of vascular structures by aggregating the detailed features from the LD module with the high level features from the UNet++ architecture. Finally, we adopt a topological loss function based on persistent homology to constrain the topological continuity of the segmentation. Results: The proposed method is evaluated on retinal fundus image datasets (DRIVE, CHASE_BD1, and STARE) as well as the 3mm and 6mm of the OCTA-500 dataset, achieving an average accuracy (ACC) of 97.25