This study aimed to develop a high-precision and high-speed registration method to support clinicians during minimally invasive therapies such as high-intensity focused ultrasound (HIFU) therapy. In these procedures, accurate localization and treatment of pathological regions within a restricted surgical field require a high level of expertise. Therefore, real-time and highly accurate intraoperative navigation is essential. To address this need, we proposed a method of extracting blood vessel cross-sections from intraoperative ultrasound images, quantifying their planar dispersion and structural complexity, and investigating geometric parameters that contribute to improved registration accuracy. Additionally, we examined suitable point cloud generation methods for point-based registration to enhance real-time performance. The feasibility of the proposed method was further validated under simulated clinical condition such as natural respiration. Our method achieved the HIFU-required accuracy of less than 5 mm and demonstrated processing within 0.2 seconds, which is significantly faster than conventional approaches. These findings suggest the potential applicability of our method as a surgeon assistant system in ultrasound-guided therapy.

OBJECTIVE:Morphological changes such as angulation and torsion of the trigeminal nerve have been reported to cause trigeminal neuralgia (TN). The authors sought to quantify and objectively evaluate the morphological changes of the trigeminal nerve and to elucidate the cause of TN. METHODS:The authors retrospectively analyzed the cases of patients with primary TN who had undergone microvascular decompression at a single facility between January 2016 and December 2022 and had both single-artery compression and a good postoperative outcome. The authors performed segmentation of the trigeminal nerve by using the patients' pre- and postoperative high-resolution MR images, and they then created a 3D model. The centerline of the trigeminal nerve was obtained using volume skeletonization, and the authors created multiple cross-sectional images by reslicing the 3D model perpendicular to the centerline. The parameters analyzed were as follows: the 1) centerline length; 2) centerline curvature; 3) centerline torsion; 4) cross-sectional area; 5) cross-sectional flattening ratio; and 6) cross-sectional long-axis angle. Comparisons were made for each parameter between the affected and unaffected side and between preoperative and postoperative trigeminal nerve findings. RESULTS:After exclusions, 70 of the 127 patients who underwent microvascular decompression during the study period were included in the analysis. In the preoperative images, the trigeminal nerve on the affected side had a significantly longer centerline length (p = 0.0003), greater curvature (p = 0.0012), smaller cross-sectional area (p < 0.0001), and greater flattening ratio (p = 0.0059) than the unaffected side. On the affected side, the preoperative trigeminal nerve had a significantly longer centerline length (p < 0.0001), greater curvature (p = 0.0028), and smaller cross-sectional area (p < 0.0001) compared to the postoperative trigeminal nerve. CONCLUSIONS:It is possible to analyze the morphological changes of the trigeminal nerve by using this method. In the preoperative trigeminal nerve on the affected side, the centerline is long and curved, and the cross-sectional area is small and flat. Further analyses may help clarify the pathophysiology, aid in diagnoses, and predict the efficacy of treatment.

This study verified the ability of the existing VoxelMorph model to predict intermediate frames from consecutive ultrasound image pairs. We systematically evaluated the effect of the time interval between image pairs on prediction accuracy, varying the time interval from 0.4 to 2.4 s. As a result, prediction accuracy tended to decrease as the time interval increased. For minute time intervals, a statistically significant difference in NCC values was observed between inspiration and expiration phases, while no significant difference was observed for other time intervals. Qualitative evaluation confirmed that the predicted intermediate frames showed high similarity to the ground truth frames. This suggests that the existing registration model is effective for spatio-temporal interpolation in dynamic ultrasound images, which can lead to the acquisition of high-volume-rate 4D data regardless of imaging equipment specifications.

Accurate three-dimensional (3D) segmentation of hepatic vascular networks is crucial for supporting ultrasound-mediated theranostics for liver diseases. Despite advancements in deep learning techniques, accurate segmentation remains challenging due to ultrasound image quality issues, including intensity and contrast fluctuations. This study introduces intensity transformation-based data augmentation methods to improve deep convolutional neural network-based segmentation of hepatic vascular networks. We employed a 3D U-Net, which leverages spatial contextual information, as the baseline. To address intensity and contrast fluctuations and improve 3D U-Net performance, we implemented data augmentation using high-contrast intensity transformation with S-shaped tone curves and low-contrast intensity transformation with Gamma and inverse S-shaped tone curves. We conducted validation experiments on 78 ultrasound volumes to evaluate the effect of both geometric and intensity transformation-based data augmentations. We found that high-contrast intensity transformation-based data augmentation decreased segmentation accuracy, while low-contrast intensity transformation-based data augmentation significantly improved Recall and Dice. Additionally, combining geometric and low-contrast intensity transformation-based data augmentations, through an OR operation on their results, further enhanced segmentation accuracy, achieving improvements of 9.7% in Recall and 3.3% in Dice. This study demonstrated the effectiveness of low-contrast intensity transformation-based data augmentation in improving volumetric segmentation of hepatic vascular networks from ultrasound volumes.

To realize real-time image registration between preoperative three-dimensional ultrasound images and an intraoperative two-dimensional (2D) ultrasound image, accurate image extraction of vascular networks at a high frame rate is necessary. To apply this approach to liver surgery, we attempted to modify the parameters of the Mask region-based convolutional neural network (R-CNN) deep learning model to extract liver blood vessels. The acquired 2D ultrasound images of the liver were divided into training and evaluation datasets, and a model was built using the training dataset. We modified the components of feature extraction, region proposal, and mask detection in the fundamental architecture of the Mask R-CNN. Finally, the model constructed was compared with the conventional Mask R-CNN using the hold-out method. The results revealed improvements of the dice similarity coefficient and the computational time. The findings suggest that the modified Mask R-CNN can be employed for highly accurate real-time detection of liver blood vessels.

Augmented reality technologies provide transformative solutions in various surgical fields. Our research focuses on the use of an advanced augmented reality system that projects 3D holographic images directly into surgical footage, potentially improving the surgeon’s orientation to the surgical field and lowering the cognitive load. We created a novel system that combines exoscopic surgical footage from the “ORBEYE” and displays both the surgical field and 3D holograms on a single screen. This setup enables surgeons to use the system without using head-mounted displays, instead viewing the integrated images on a 3D monitor. Thirteen surgeons and surgical assistants completed tasks with 2D and 3D graphical surgical guides. The NASA Task Load Index was used to assess mental, physical, and temporal demands. The use of 3D graphical surgical guides significantly improved performance metrics in cochlear implant surgeries by lowering mental, physical, temporal, and frustration levels. However, for Bonebridge implantation, the 2D graphical surgical guide performed better overall (p = 0.045). Participants found the augmented reality system’s video latency to be imperceptible, measuring 0.13 ± 0.01 s. This advanced augmented reality system significantly improves the efficiency and precision of cochlear implant surgeries by lowering cognitive load and improving spatial orientation.

Ultrasound imaging is an essential tool in anesthesiology, particularly for ultrasound-guided peripheral nerve blocks (US-PNBs). However, challenges such as speckle noise, acoustic shadows, and variability in nerve appearance complicate the accurate localization of nerve tissues. To address this issue, this study introduces a deep convolutional neural network (DCNN), specifically Scaled-YOLOv4, and investigates an appropriate network model and input image scaling for nerve detection on ultrasound images. Utilizing two datasets, a public dataset and an original dataset, we evaluated the effects of model scale and input image size on detection performance. Our findings reveal that smaller input images and larger model scales significantly improve detection accuracy. The optimal configuration of model size and input image size not only achieved high detection accuracy but also demonstrated real-time processing capabilities.

Shape from focus (SFF) is a 3D shape measurement technique using a monocular camera.SFF also provides an all-in-focus image from multiple defocus images.In SFF, the pixelwise focused camera position and pixel data are determined from the degree of focus (focus measure).We previously reported that the optimal window size in the focus measure computation for accurate 3D shape measurement depends on the local texture frequency of objects.In this study, we validate the performance of the all-in-focus image reconstruction.We performed two tests: image quality validation using human tissue slides and a feasibility test using animal tissue.First, the image quality was validated using the similarity between reference images and reconstructed images of five human tissue slides.Second, the surface texture of an animal tissue was reconstructed as a feasibility test.The experimental results show that the similarity of the reconstructed images was significantly high, and the precise texture of the tissue 3D surface could be clearly observed.From the experiments, we demonstrate the reconstruction performance of the all-in-focus image by SFF both quantitatively and qualitatively.

Tunable stiffness mechanisms can increase the noninvasiveness and stability of organ manipulation in laparoscopic liver resection. We have developed an organ-grasping device using beam-shaped tunable stiffness mechanism. Increasing the change ratio of stiffness will improve the performance of the device by offering high flexibility when adhering to the liver surface and high rigidity during the manipulation of the liver; however, optimal design of the beam has not been investigated. In this study, we investigate the wavy structure shape of the device that enhances the change in the ratio of stiffness. To increase the stiffness in a high-stiffness state, we used principal stress lines in the device to design the edge curve of the wavy shape material in the beams. We also investigated the arrangement of the wavy shape to decrease the stiffness in a low-stiffness state. Simulation using finite element method showed that the change ratio of stiffness was improved up to 13.0 by the new wavy shape arranged with the uniformly thick bottom of the waves.

The purpose of this study is to develop a support software to place ultrasound transducer, which produces local acoustic radiation force, for active induction of a micro object, representing a thin catheter, through blood vessel network. First, based on the image analysis of 3D blood vessel network, which was obtained using echography, the system calculates the possible area of the ultrasound transducer to be able to induce a micro object to desired path in blood vessel network. Meanwhile, due to the shape of body surface and the position of the ribs, which was obtained by an optical position sensor, the common area with the body surface was derived. Then, considering the position and the movable area of the robot, which grasps and moves the ultrasound transducer, the system indicates the candidate commands for the robot. To verify the system performance, we have conducted a simulation for active induction experiment of thin catheter through blood vessel network of a normal volunteer subject. As the result, we confirmed the variation of the thin catheter induction according to both body posture and breathing condition.

Most surgical tools are made of metal or soft rubber, the stiffnesses of which may not be suitable for some surgeries. There is a need for stiffness-tunable materials for use in surgical tools or a mechanism for tuning the stiffness of a material without a chemical or thermal reaction. We have proposed a stiffness-tunable beam-shaped material with pneumatic access for stiffness control. We have also shown its application in grasping a soft organ stably during laparoscopic liver surgery. Its stiffness became 5.47 times greater by inducing -80 kPa negative pressure inside the device. The beam structure had a wavy composition of hard and soft rubbers. In this paper, we evaluate the finite element method (FEM) performance to determine the optimal design of the stiffness-tunable beam. FEM resulted in a 11.3-fold improvement of stiffness tuning for wavy shapes, that is, sequentially aligned pillars with an adequate pitch, and increased the stiffness, one of the shaping parameters, 11.8-fold.

Virtual fluoroscopy is a surgical navigation technique that is often used in orthopedic surgery. In virtual fluoroscopy, a surgeon draws tool insertion lines on intra-operative X-ray images as a surgical plan, and the position of an actual surgical tool tracked by a position sensor is also drawn on the X-ray images. The surgeon can intuitively monitor the positioning error between the plan and the tool without continuous X-ray imaging; therefore, X-ray exposure is significantly reduced compared with that in conventional fluoroscopic procedures. However, a large navigation error is often reported even when the surgeon places the tool on the planned line. To address this issue, we investigated the reason for the error by using a mock virtual fluoroscopy system. From the results of the mock operation study, we revealed that the tool positioning error depends on the convergence angle around the plan path rather than the camera rotation angle.

Brain structure segmentation on magnetic resonance (MR) images is important for various clinical applications. It has been automatically performed by using fully convolutional networks. However, it suffers from the class imbalance problem. To address this problem, we investigated how loss weighting strategies work for brain structure segmentation tasks with different class imbalance situations on MR images. In this study, we adopted segmentation tasks of the cerebrum, cerebellum, brainstem, and blood vessels from MR cisternography and angiography images as the target segmentation tasks. We used a U-net architecture with cross-entropy and Dice loss functions as a baseline and evaluated the effect of the following loss weighting strategies: inverse frequency weighting, median inverse frequency weighting, focal weighting, distance map-based weighting, and distance penalty term-based weighting. In the experiments, the Dice loss function with focal weighting showed the best performance and had a high average Dice score of 92.8% in the binary-class segmentation tasks, while the cross-entropy loss functions with distance map-based weighting achieved the Dice score of up to 93.1% in the multi-class segmentation tasks. The results suggested that the distance map-based and the focal weightings could boost the performance of cross-entropy and Dice loss functions in class imbalanced segmentation tasks, respectively.

PURPOSE:In recent years, fully convolutional networks (FCNs) have been applied to various medical image segmentation tasks. However, it is difficult to generate a large amount of high-quality annotation data to train FCNs for medical image segmentation. Thus, it is desired to achieve high segmentation performances even from incomplete training data. We aim to evaluate performance of FCNs to clean noises and interpolate labels from noisy and sparsely given label images.METHODS:To evaluate the label cleaning and propagation performance of FCNs, we used 2D and 3D FCNs to perform volumetric brain segmentation from magnetic resonance image volumes, based on network training on incomplete training datasets from noisy and sparse annotation.RESULTS:The experimental results using pseudo-incomplete training data showed that both 2D and 3D FCNs could provide improved segmentation results from the incomplete training data, especially by using three orthogonal annotation images for network training.CONCLUSION:This paper presented a validation for label cleaning and propagation based on FCNs. FCNs might have the potential to achieve improved segmentation performances even from sparse annotation data including possible noises by manual annotation, which can be an important clue to more efficient annotation.

Respiratory monitoring is a significant issue to reduce patient risks and medical staff labor in postoperative care and epidemic infection, particularly after the COVID-19 pandemic. Oximetry is widely used for respiration monitoring in the clinic, but it sometimes fails to capture a low-functional respiratory condition even though a patient has breathing difficulty. Another approach is breathing-sound monitoring, but this is unstable due to the indirect measurement of lung volume. Kobayashi in our team is developing a sensor measuring temporal changes in lung volume with a displacement sensor attached across the sixth and eighth ribs. For processing these respiratory signals, we propose the combination of complex-valued wavelet transform and the correlation among spectrum sequences. We present the processing results and discuss its feasibility to detect a low-functional condition in respiration. The result for detecting low-functional respiration showed good performance with a sensitivity of 0.88 and specificity of 0.88 to 1 in its receiver operating characteristic (ROC) curve.

A fiducial planar marker is generally used for positional tracking by a single camera.The technology is widely applied to entertainment systems and to medical assistance such as surgical navigation systems.However, the tracking accuracy is not sufficient for precise navigation; therefore, fiducial marker tracking has not replaced position sensors such as infrared light tracking systems.In particular, attitude accuracy is important for tracking because a marker is attached to a convenient position of a tracked tool far from the tool's tip.Therefore, the attitude error generates a large positional error at the tip.To address this issue, we propose a method of improving fiducial planar marker tracking accuracy by sensor fusion.The proposed tracking system consists of a fiducial marker and a gyroscope, which provides angular velocity.To integrate the sensors, a sensor fusion filter based on a Kalman filter was designed.The feasibility and performance of the filter were validated experimentally by using a three-axis motorized rotational stage with potentiometers.The results showed that the root-mean-square error of attitude measurement was reduced by the proposed integration method.We confirmed that sensor fusion with a gyroscope is feasible for 3D tracking of a fiducial planar marker.

This paper reports the precision of shape-from-focus (SFF) imaging according to the texture frequencies and window sizes of a focus measure. SFF is one of various depth measurement techniques for optical imaging, such as microscopy and endoscopy. SFF measures the depth of an object according to focus measure, which is generally computed with a fixed window. The window size affects the performance of SFF and should be adjusted for the texture of an object. In this study, we investigated the precision difference of SFF in texture frequencies and by window size. Two experiments were performed: precision validation in texture frequencies with a fixed window size, and precision validation in various window sizes related to pixel-cycle lengths. The first experimental results showed that a smaller window size could not provide a correct focus measure, and the second results showed that a window size that is approximately equal to a pixel-cycle length of the texture could provide better precision. These findings could potentially contribute to determining the appropriate window size of focus measure operation in shape-from-focus reconstruction.

Three-dimensional (3D) shape acquisition has been widely introduced to enrich quantitative analysis with the combination of object shape and texture, for example, surface roughness evaluation in industry and gastrointestinal endoscopy in medicine. Shape from focus is a promising technique to measure substance surfaces in 3D space because no occlusion problem appears in principle, as does with stereo shape measurement, which is another commonly used option. We have been developing endoscopic shape measurement devices and shape reconstruction algorithms. In this paper, we propose a mechanism for driving an image sensor reciprocated for the shape from focus of 3D shape measurement in monocular endoscopy. It uses a stepping motor and a planar-end cam, which transforms the motor rotation to imaging sensor reciprocation, to implement the shape from focus of 3D shape measurement in endoscopy. We test and discuss the device in terms of its driving accuracy and application feasibility for endoscopic 3D shape measurement.