Laser interstitial thermal therapy (LITT) is a novel and effective treatment for malignant glioma patients who could not undergo surgical resection. Preoperative planning for better tumor coverage and less probe use is a critical step of LITT's success, which can be challenging for surgeons as different constraints need to be considered. Here we proposed an automatic algorithm to plan trajectory combination for LITT, extending previous studies on LITT that used only one probe for treatment. Furthermore, we combined iterative optimization with Lagrangian relaxation and thus achieved both mathematic optimality and solving efficiency, which existing ablation planning research may fail to balance. The planning algorithm was evaluated in 16 cases of tumors and was compared with surgeon's planning result. In all cases our planning result achieved an average ablation rate of 99.83% and satisfied multiple constraints, outperforming the manual planning in probe usage and extent of ablation. By solving the relaxation model, our proposed method proved to be capable of reducing the number of probes to minimum with an average time cost of 39.73 seconds.

Augmented Reality (AR) has been used to facilitate surgical guidance during External Ventricular Drain (EVD) surgery, reducing the risks of misplacement in manual operations. During this procedure, the key challenge is accurately estimating the spatial relationship between pre-operative images and actual patient anatomy in AR environment. This research proposes a novel framework utilizing Time of Flight (ToF) depth sensors integrated in commercially available AR Head Mounted Devices (HMD) for precise EVD surgical guidance. As previous studies have proven depth errors for ToF sensors, we first assessed their properties on AR-HMDs. Subsequently, a depth error model and patient-specific parameter identification method are introduced for accurate surface information. A tracking pipeline combining retro-reflective markers and point clouds is then proposed for accurate head tracking. The head surface is reconstructed using depth data for spatial registration, avoiding fixing tracking targets rigidly on the patient's skull. Firstly, $7.580\pm 1.488 mm$ depth value error was revealed on human skin, indicating the significance of depth correction. Our results showed that the error was reduced by over $85\%$ using proposed depth correction method on head phantoms in different materials. Meanwhile, the head surface reconstructed with corrected depth data achieved sub-millimetre accuracy. An experiment on sheep head revealed $0.79 mm$ reconstruction error. Furthermore, a user study was conducted for the performance in simulated EVD surgery, where five surgeons performed nine k-wire injections on a head phantom with virtual guidance. Results of this study revealed $2.09 \pm 0.16 mm$ translational accuracy and $2.97\pm 0.91$ degree orientational accuracy.

The rapid development of neurosurgery robots and navigation systems has significantly enhanced surgical precision and safety.Neurosurgery robots,based on stereotactic technology,achieve submillimeter-level positioning through multi-modal image fusion and non-invasive registration techniques.Equipped with high-degree-of-freedom robotic arms and integrated sensors,they precisely perform complex procedures such as stereo-electroencephalography(SEEG)electrode implantation,deep brain stimulation(DBS)device placement,and magnetic resonance-guided laser interstitial thermal therapy(MRgLITT).Navigation systems employ optical or electromagnetic tracking technologies combined with multi-modal imaging to assist surgeons in tracking important structures and target lesion tissue precisely.Future advancements may autonomously generate personalized surgical plans to reduce human error,while flexible robotic arms and novel endoscope-holding robots could expand endoscopic applications.Furthermore,the integration of multiple technologies will broaden neurosurgical applications.These innovations not only minimize complication risks and shorten treatment periods,but also propel the field toward intelligent,minimally invasive neurosurgery.

Ultrasound-guided biopsy is widely used for tumor diagnosis, yet interpreting spatial information from ultrasound images remains challenging. Augmented reality (AR) enables in-situ overlay of imaging and guidance cues, potentially enhancing spatial awareness, improving image interpretation, and facilitating needle navigation. However, its impact on biopsy performance remains inconclusive in previous studies. This research investigates how various factors influence the performance of AR navigation in ultrasound-guided biopsy, aiming to identify crucial considerations for system design. A navigation system was first developed to deliver in-situ ultrasound images and biopsy needle along with various visual cues. Thirty-one participants, representing a range of ultrasound expertise, were recruited to perform 1860 biopsy simulations under varying conditions of visualization content and operational methods. Metrics, including biopsy accuracy, task duration, success rate, and subjective feedback, were recorded for analysis. In brief, experienced interventionists showed minimal benefit (4.39 mm vs. 3.14 mm, 10.64s vs. 9.23s) and reported increased cognitive load during familiar in-plane procedures when overloaded with virtual cues. In contrast, they demonstrated significant performance gains (8.01 mm vs. 2.73 mm, 13.32s vs. 6.77s) and improved user experience in less familiar out-of-plane tasks, even outperforming their in-plane results. These findings highlight the need for user- and task-specific system design considering the procedural expertise.

Objective. Accurate prediction of thermal damage extent is essential for effective and precise thermal therapy, especially in brain laser interstitial thermal therapy (LITT). Immediate postoperative contrast-enhanced T1-weighted imaging (CE-T1WI) is the primary method for clinically assessing in vivo thermal damage after image-guided LITT. CE-T1WI reveals a hyperintense enhancing rim surrounding the target lesion, which serves as a key radiological marker for evaluating the thermal damage extent. Although widely used in clinical practice, traditional thermal damage models rely on empirical parameters from in vitro experiments, which can lead to inaccurate predictions of thermal damage in vivo. Additionally, these models predict only two tissue states (damaged or undamaged), failing to capture three tissue states observed on post-CE-T1WI images, highlighting the need for improved thermal damage prediction methods. Approach. This study proposes a novel convolutional long short-term memory-based model that utilizes intraoperative temperature distribution history data measured by magnetic resonance temperature imaging (MRTI) during LITT to predict the enhancing rim on post-CE-T1WI images. This method was implemented and evaluated on retrospective data from 56 patients underwent brain LITT. Main results. The proposed model effectively predicts the enhancing rim on postoperative images, achieving an average dice similarity coefficient of 0.82 ( +/- 0.063) on the test dataset. Furthermore, it generates real-time predicted thermal damage area variation trends that closely resemble those of the traditional thermal damage model, suggesting potential for real-time prediction of thermal damage extent. Significance. This method could provide a valuable tool for visualizing and assessing intraoperative thermal damage extent.

Closed Reduction and Internal Fixation (CRIF) is the preferred treatment for closed long bone shaft fractures, where frequent X-rays imaging reduce efficiency and increase radiation exposure. Intra-operative ultrasound offers real-time, radiation-free bone imaging, but challenges orthopedic surgeons. To address this, we developed an Augmented Reality (AR) navigation system with in-situ ultrasound visualization, combining on-device infrared tool tracking via an AR headset’s depth sensor with remote rendering for accurate, low-latency images. A standard operating procedure for surgical application was also proposed. Clinical trials with six patients showed significant reduction in fluoroscopy dose and operation time.

BACKGROUND:Identifying the elbow flexion-extension (FE) motion axis is crucial for the precise placement of an elbow external fixator. Traditional x-ray fluoroscopy has high radiation exposure, low accuracy, poor repeatability, and is greatly influenced by individual experience. PURPOSE:This study aimed to assess the feasibility and accuracy of a novel image-free method for elbow external fixation surgery based on computer-assisted navigation. METHODS:An optical positioning system was employed to track the motion trajectory of the forearm relative to the upper arm. This system works by emitting infrared signals detected by cameras to determine the spatial positions of markers attached to the limbs. The 3D principal components analysis (PCA) was used to determine the orientation of the elbow's rotation axis by analyzing the trajectory data. The 2D circle fitting method involved fitting a circle to the projected data points on a plane perpendicular to the identified axis, ensuring precise calculation of the rotation plane and axis. A custom graphical user interface (GUI) was developed to visualize the kinematics data and guide the placement of an axis pin for the external fixator. RESULTS:The kinematics data of elbow FE movements showed high repeatability with RMS-radius within 1 mm, RMS-coplane less than 2 mm, Deviation-center no more than 2 mm, and deviation angle within 1.5°. CONCLUSIONS:The proposed image-free method improves the accuracy and repeatability of elbow external fixation surgery, increasing feasibility and security.

Mastoidectomy is critical in acoustic neuroma surgery, where precise planning of the bone milling area is essential for surgical navigation. The complexity of representing the irregular volumetric area and the presence of high-risk structures (e.g., blood vessels and nerves) complicate this task. In order to determine the bone area to mill using preoperative CT images automatically, we propose an automated planning method using evolutionary multi-objective optimization for safer and more efficient milling plans. High-resolution segmentation of the adjacent risk structures is performed on preoperative CT images with a template-based approach. The maximum milling area is defined based on constraints from the risk structures and tool dimensions. Deformation fields are used to simplify the volumetric area into limited continuous parameters suitable for optimization. Finally, a multi-objective optimization algorithm is used to achieve a Pareto-optimal design. Compared with manual planning on six volumes, our method reduced the potential damage to the scala vestibuli by 29.8%, improved the milling boundary smoothness by 78.3%, and increased target accessibility by 26.4%. Assessment by surgeons confirmed the clinical feasibility of the generated plans. In summary, this study presents a parameterization approach to irregular volumetric regions, enabling automated milling area planning through optimization techniques that ensure safety and feasibility. This method is also adaptable to various volumetric planning scenarios.

The use of Augmented Reality (AR) for navigation purposes has shown beneficial in assisting physicians during the performance of surgical procedures. These applications commonly require knowing the pose of surgical tools and patients to provide visual information that surgeons can use during the task performance. Existing medical-grade tracking systems use infrared cameras placed inside the Operating Room (OR) to identify retro-reflective markers attached to objects of interest and compute their pose. Some commercially available AR Head-Mounted Displays (HMDs) use similar cameras for self-localization, hand tracking, and estimating the objects' depth. This work presents a framework that uses the built-in cameras of AR HMDs to enable accurate tracking of retro-reflective markers, such as those used in surgical procedures, without the need to integrate any additional components. This framework is also capable of simultaneously tracking multiple tools. Our results show that the tracking and detection of the markers can be achieved with an accuracy of 0.09 +- 0.06 mm on lateral translation, 0.42 +- 0.32 mm on longitudinal translation, and 0.80 +- 0.39 deg for rotations around the vertical axis. Furthermore, to showcase the relevance of the proposed framework, we evaluate the system's performance in the context of surgical procedures. This use case was designed to replicate the scenarios of k-wire insertions in orthopedic procedures. For evaluation, two surgeons and one biomedical researcher were provided with visual navigation, each performing 21 injections. Results from this use case provide comparable accuracy to those reported in the literature for AR-based navigation procedures.

Laser interstitial thermal therapy (LITT) is popular for treating brain tumours and epilepsy. The strict control of tissue thermal damage extent is crucial for LITT. Temperature prediction is useful for predicting thermal damage extent. Accurately predicting in vivo brain tissue temperature is challenging due to the temperature dependence and the individual variations in tissue properties. Considering these factors is essential for improving the temperature prediction accuracy. Objective . To present a method for predicting patient-specific tissue temperature distribution within a target lesion area in the brain during LITT. Approach . A magnetic resonance temperature imaging (MRTI) data-driven estimation model was constructed and combined with a modified Pennes bioheat transfer equation (PBHE) to predict patient-specific temperature distribution. In the PBHE for temperature prediction, the individual specificity and temperature dependence of thermal tissue properties and blood perfusion, as well as the individual specificity of optical tissue properties were considered. Only MRTI data during one laser irradiation were required in the method. This enables the prediction of patient-specific temperature distribution and the resulting thermal damage region for subsequent ablations. Main results . Patient-specific temperature prediction was evaluated based on clinical data acquired during LITT in the brain, using intraoperative MRTI data as the reference standard. Our method significantly improved the prediction performance of temperature distribution and thermal damage region. The average root mean square error was decreased by 69.54%, the average intraclass correlation coefficient was increased by 37.5%, the average Dice similarity coefficient was increased by 43.14% for thermal damage region prediction. Significance . The proposed method can predict temperature distribution and thermal damage region at an individual patient level during LITT, providing a promising approach to assist in patient-specific treatment planning for LITT in the brain.

Objective: Robot-assisted drilling or milling is used widely in surgeries concerning bones, which is restricted in precision and safety due to tool bit runout. This paper aims to achieve real-time accurate runout status monitor, to support safer surgeries. Methods: First, a rigid tool runout model is constructed. An orthogonal dual laser tracking structure and relevant calibration methods are then proposed. An algorithm is finally proposed to realize tool tip tracking under eccentrical load with the tool model and tracking structure. Results: The experiment under no load indicated a high accordance between the model and the measurement. Relative errors of 8.11% and 14.96% were presented in accuracy tests on x and y , respectively, while 5.5 m and 6.2 m absolute error were presented for precision. Moreover, the method was able to differentiate different stages during drilling under eccentrical loads. The calculation efficiency reaches 1736 fps . Conclusion: The proposed method can provide real-time accurate surgical drilling tool position status, proving the potential to improve the safety of surgeries.

Computer-aided detection (CADe) for endoscopy can help physicians to locate and identify lesions better, but there are still many false positives (FP) when processing cross-domain data. This paper proposes SE-SORT, a plug-and-play tracker, designed to be seamlessly integrated as a post-processing plugin into existing CADe systems to enhance the adaptability to cross-domain data. The proposed tracker adds trajectory initialization thresholds into the tracking association strategy, reducing the impact of high confidence FPs on the matching process. Experiments show that the modified tracker effectively reduces the impact without significantly affecting the processing speed. This allows the detector of CADe system to tolerate lower detection confidence thresholds, thus improving the overall accuracy on cross-domain data. Compared to existing SORT trackers, the proposed tracker exhibits better accuracy and higher efficiency in endoscopic lesion detection and tracking. This work will help to improve the generalization and expanding the clinical application scope of related works on endoscopic real-time CADe.

Objective: External Ventricular Drainage (EVD) is a widely used procedure in neurosurgery that is restricted in accuracy and reproducibility due to free-hand operation. Augmented reality (AR) improves punctuation success rate by superimposing virtual paths on the operation area. However, the effectiveness of surgical guidance is affected by tracking accuracy. This paper aims to achieve accurate and stable head tracking during EVD surgery. Methods: We propose a dynamic inside-out tracking method combining retro-reflective markers and point clouds. First, built-in infrared depth sensor of HoloLens 2 is used to identify markers pasted on patient's head for coarse registration of preoperative images and intraoperative patient. Real-time 3D point clouds and point-to-plane ICP registration are then used to further improve tracking accuracy and stability. Meanwhile, we calibrate and correct the depth distortion of the HoloLens 2 depth sensor on different materials, improving the accuracy of point cloud-based tracking methods. Results: The root mean square error (RMSE) of preoperative registration is less than 1.6mm; average RMSE of intraoperative head tracking is less than 1.28mm. Meanwhile, average angular tracking jitter is reduced by more than 40% when integrating point clouds. The proposed method can achieve 37.7fps tracking. Conclusion: The retro-reflective marker and point cloud hybrid tracking method in this paper can achieve high-precision real-time head tracking, providing the potential for accurate visual guidance in EVD surgery.

Miniaturized manipulators are invaluable for microsurgery; however, they often face limitations in precision, load capacity, and Degrees of Freedom (DOFs) due to size and weight constraints. In this article, we introduce a 6-DOF mini manipulator designed to address these challenges. Powered by cost-effective stepper linear motion modules, the proposed device incorporates a dual-plane mechanism for 4-DOF XY motion. We also innovatively integrated a compliant 2-DOF rod drive system for Z-axis rotation and translation. The manipulator's forward/inverse and remote center of motion kinematics models are provided. Furthermore, we developed a microsurgical platform equipped with microvision sensing and remote-control functions. This manipulator, which weighs less than 72 g, enables precise 6-DOF control for tools with diameters less than 0.5 mm. It delivers more than 3 N of thrust force along the compliant rod drive axis, achieving at least 10 mu m motion resolution and a 9 N maximum output force per DOF. Rigorous testing was performed, demonstrating the system's ability to successfully perform puncture and drilling operations on porcine skin and bone. Additionally, we showed its effectiveness in ocular surgeries, demonstrating its potential for use as a cost-effective, portable device in minimally invasive procedures.

PURPOSE:Biopsies play a crucial role in determining the classification and staging of tumors. Ultrasound is frequently used in this procedure to provide real-time anatomical information. Using augmented reality (AR), surgeons can visualize ultrasound data and spatial navigation information seamlessly integrated with real tissues. This innovation facilitates faster and more precise biopsy operations.METHODS:We have developed an augmented reality biopsy navigation system characterized by low display latency and high accuracy. Ultrasound data is initially read by an image capture card and streamed to Unity via net communication. In Unity, navigation information is rendered and transmitted to the HoloLens 2 device using holographic remoting. Concurrently, a retro-reflective tool tracking method is implemented on the HoloLens 2, enabling the simultaneous tracking of the ultrasound probe and biopsy needle. Distinct navigation information is provided during in-plane and out-of-plane punctuation. To evaluate the effectiveness of our system, we conducted a study involving ten participants, assessing puncture accuracy and biopsy time in comparison to traditional methods.RESULTS:Ultrasound image was streamed from the ultrasound device to augmented reality headset with 122.49±11.61ms latency, while only 16.22±11.25ms was taken after data acquisition from image capture card. Navigation accuracy reached 1.23±0.68mm in the image plane and 0.95±0.70mm outside the image plane, within a depth range of 200 millimeters. Remarkably, the utilization of our system led to 98% and 95% success rate in out-of-plane and in-plane biopsy, among ten participants with little ultrasound experience.CONCLUSION:To sum up, this paper introduces an AR-based ultrasound biopsy navigation system characterized by high navigation accuracy and minimal latency. The system provides distinct visualization contents during in-plane and out-of-plane operations according to their different characteristics. Use case study in this paper proved that our system can help young surgeons perform biopsy faster and more accurately.

Objective: Ultrasound (US) probe calibration is critical for the localized ultrasound system. Key points and surfaces are often used for calibration, whose accuracy is restricted by the ultrasound volume effect. The aim of this paper is to accurately calibrate the US probe under volume effect. Method: We present a model-guided ultrasound probe calibration method, to provide accurate calibration results under volume effect. First, we design a rotationally symmetric calibration phantom unit to provide image areas weekly affected by volume effect during continuous scanning. Second, US images from the uncalibrated probe are used to reconstruct the 3D image volume. Finally, we use image registration and point registration for super-resolution unification of US images, the model and the tracking device. Results: In multiple probe calibration experiments at different probe depths, the average calibration precision was 0.163 mm; in the needle tip tracking experiment, the average detection accuracy was 0.335 mm. Conclusion: Guided by the specially designed model, our method can realize precise and accurate ultrasound probe calibration under volume effect.

Cerebrovascular segmentation based on phase-contrast magnetic resonance angiography (PC-MRA) provides patient-specific intracranial vascular structures for neurosurgery planning. However, the vascular complex to-pology and spatial sparsity make the task challenging. Inspired by the computed tomography reconstruction, this paper proposes a Radon Projection Composition Network (RPC-Net) for cerebrovascular segmentation in PC-MRA, aiming to enhance distribution probability of vessels and fully obtain the vascular topological informa-tion. Multi-directional Radon projections of the images are introduced and a two-stream network is used to learn the features of the 3D images and projections. The projection domain features are remapped to the 3D image domain by filtered back-projection transform to obtain the image-projection joint features for predicting vessel voxels. A four-fold cross-validation experiment was performed on a local dataset containing 128 PC-MRA scans. The average Dice similarity coefficient, precision and recall of the RPC-Net achieved 86.12%, 85.91% and 86.50%, respectively, while the average completeness and validity of the vessel structure were 85.50% and 92.38%, respectively. The proposed method outperformed the existing methods, especially with significant improvement on the extraction of small and low-intensity vessels. Moreover, the applicability of the segmen-tation for electrode trajectory planning was also validated. The results demonstrate that the RPC-Net realizes an accurate and complete cerebrovascular segmentation and has potential applications in assisting neurosurgery preoperative planning.

In recent years, increasing interest has been revealed in medical Augmented Reality (AR), particularly in medical education, preoperative planning, and intra-operative guidance. However, there lacks a platform to effectively integrate each essential part of surgical AR, including visualization, interaction, tracking, and medical image processing, hindering new beginners or surgeons from easily building up any comprehensive AR applications. Therefore, we introduce 3D Slicer-AR-Bridge, a novel framework taking advantage of open-source medical image processing software 3D Slicer and advanced AR developing toolkits, to enable fast generation of surgical AR applications with high performance, high extensibility, and transparent data flow. The bridge is first built upon a seamless medical data sharing interface supporting volumes, segmentations, models, and annotations, between 3D Slicer and Unity, together with corresponding visualization and interaction methods. Sensor resources from AR devices are then synchronized to the bridge during visualization using holographic remoting to enable in-situ display with high render quality. Meanwhile, all resources are highly accessible in 3D Slicer, while a general user interface is provided to operate all the elements, enabling the creation of an in-situ AR application with little coding, which we believe will greatly benefit the community and clinical research. The performance tests in our paper indicate that 3D Slicer-AR-Bridge can do data exchange efficiently. At the end of our paper, we provide two different applications built with the bridge with little modification to the codes, to exemplify how the framework can help the researchers in surgical AR.

颅内病变磁共振监控激光间质热消融(LITT)是一种微创手术治疗方法,与传统开颅手术相比具有并发症发生率低、手术创伤更小的特点.LITT是用激光发出的热量消融目标靶点病变组织.在磁共振监控LITT手术中,使用导航定位机器人微创置入光纤,将激光能量准确输送到消融靶点,利用磁共振实时监测,精准控制消融区域内的温度场变化,利用预测模型可视化消融区的覆盖情况,实时调整光纤的位置以及激光能量,从而选择性地消融病变组织.该技术可以避开颅内关键结构与重要功能区,获得准确的消融区覆盖,实现安全精准的适形消融.颅内病灶激光消融治疗具有良好的有效性和安全性,为颅内病变治疗提供了一种微创的新途径.