Accurate needle placement in spine interventions is critical for effective pain management, yet it depends on reliable identification of anatomical landmarks and careful trajectory planning. Conventional imaging guidance often relies both on CT and X-ray fluoroscopy, exposing patients and staff to high dose of radiation while providing limited real-time 3D feedback. We present an optical see-through augmented reality (OST-AR)-guided robotic system for spine procedures that provides in situ visualization of spinal structures to support needle trajectory planning. We integrate a cone-beam CT (CBCT)-derived 3D spine model which is co-registered with live ultrasound, enabling users to combine global anatomical context with local, real-time imaging. We evaluated the system in a phantom user study involving two representative spine procedures: facet joint injection and lumbar puncture. Sixteen participants performed insertions under two visualization conditions: conventional screen vs. AR. Results show that AR significantly reduces execution time and across-task placement error, while also improving usability, trust, and spatial understanding and lowering cognitive workload. These findings demonstrate the feasibility of AR-guided robotic ultrasound for spine interventions, highlighting its potential to enhance accuracy, efficiency, and user experience in image-guided procedures.

Purpose: Spinal instability is a widespread condition that causes pain, fatigue, and restricted mobility, profoundly affecting patients' quality of life. In clinical practice, the gold standard for diagnosis is dynamic X-ray imaging. However, X-ray provides only 2D motion information, while 3D modalities such as computed tomography (CT) or cone beam computed tomography (CBCT) cannot efficiently capture motion. Therefore, there is a need for a system capable of visualizing real-time 3D spinal motion while minimizing radiation exposure. Methods: We propose ultrasound as an auxiliary modality for 3D spine visualization. Due to acoustic limitations, ultrasound captures only the superficial spinal surface. Therefore, the partially compounded ultrasound volume is registered to preoperative 3D imaging. In this study, CBCT provides the neutral spine configuration, while robotic ultrasound acquisition is performed at maximal spinal bending. A kinematic model is applied to the CBCT-derived spine model for coarse registration, followed by ICP for fine registration, with kinematic parameters optimized based on the registration results. Real-time ultrasound motion tracking is then used to estimate continuous 3D spinal motion by interpolating between the neutral and maximally bent states. Results: The pipeline was evaluated on a bendable 3D-printed lumbar spine phantom. The registration error was 1.941 ± 0.199 mm and the interpolated spinal motion error was 2.01 ± 0.309 mm (median). Conclusion: The proposed robotic ultrasound framework enables radiation-reduced, real-time 3D visualization of spinal motion, offering a promising 3D alternative to conventional dynamic X-ray imaging for assessing spinal instability.

Recent robotics advancements have enabled novel applications in medicine, such as automating ultrasound acquisition to support sonographers and enable remote operation. However, a key challenge is the lack of haptic and perceptual transparency regarding the robot’s decisions and actions, which may undermine trust and user acceptance. In this study, we propose a multisensory feedback system to enhance physician–robot interaction during robotic ultrasound procedures. The system provides real-time feedback on the force exerted by the robot on the patient and was developed following a user-centered approach, informed by clinical domain expertise, to support usability and clinical relevance. In a simulated Extended Reality (XR) environment, we evaluated the impact of different feedback conditions—including two sonification strategies, one visual feedback method, and their combinations. A user study with 30 participants, including three practicing clinicians, was conducted to assess these modalities. Results showed that multisensory feedback significantly reduced cognitive load (NASA-TLX: A1+V vs.V, p<0.001; A2+V vs. V, p < 0.01), improved usability (SUS: A1+V vs. V, p < 0.001; A2+V vs. V, p < 0.001) and perceived ease of use (SEQ: A1+V vs. V, p < 0.01; A2+V vs. V, p < 0.01). While visual feedback alone yielded better performance in force monitoring, the multisensory conditions supported better focus on the primary task of image analysis. These findings suggest that multisensory feedback can enhance user experience and support interaction factors associated with trust formation, such as reduced workload and improved awareness of robot behavior, especially with further training and adaptation. Given the predominantly non-clinical sample and the use of abstract visual targets, these results should be read as an exploratory, preclinical validation of multisensory human–robot interaction rather than as evidence of clinical efficacy.

PurposeThis study compares two augmented reality (AR)-guided imaging workflows, one based on ultrasound shape completion and the other on cone-beam computed tomography (CBCT), for planning and executing lumbar needle interventions. The aim is to assess how imaging modality influences user performance, usability, and trust during AR-assisted spinal procedures.MethodsBoth imaging systems were integrated into an AR framework, enabling in situ visualization and trajectory guidance. The ultrasound-based workflow combined AR-guided robotic scanning, probabilistic shape completion, and AR visualization. The CBCT-based workflow used AR-assisted scan volume planning, CBCT acquisition, and AR visualization. A between-subject user study was conducted and evaluated in two phases: (1) planning and image acquisition, and (2) needle insertion.ResultsPlanning time was significantly shorter with the CBCT-based workflow, while SUS, SEQ, and NASA-TLX were comparable between modalities. In the needle insertion phase, the CBCT-based workflow yielded marginally faster insertion times, significantly lower overall placement error, and better subjective ratings with higher Trust. The ultrasound-based workflow achieved adequate accuracy for facet joint insertion, but showed larger errors for lumbar puncture, where reconstructions depended more heavily on shape completion.ConclusionThe findings indicate that both AR-guided imaging pipelines are viable for spinal intervention support. CBCT-based AR offers advantages in efficiency, precision, usability, and user confidence during insertion, whereas ultrasound-based AR provides adaptive, radiation-free imaging but is limited by shape completion in deeper spinal regions. These complementary characteristics motivate hybrid AR guidance that uses CBCT for global anatomical context and planning, augmented by ultrasound for adaptive intraoperative updates.

Robotic ultrasound (RUS) presents new opportunities for remote diagnostics but introduces challenges related to perceptual transparency and operator trust. To address this, we explore the use of multisensory feedback by combining visual and auditory cues within an Extended Reality (XR) interface to support physicians during robot-assisted ultrasound procedures. We developed a prototype system that visualizes probe force and provides real-time sonification using both musical and physics-based approaches. A user study in a simulated XR environment evaluated five feedback conditions across concurrent imaging, force monitoring, and patient interaction tasks. Initial findings suggest that audiovisual feedback may reduce cognitive load and enhance task coordination. This work offers early insights into the design of multimodal interfaces for clinical human–robot interaction.

Robotic ultrasound systems have the potential to improve medical diagnostics, but patient acceptance remains a key challenge. To address this, we propose a novel system that combines an AI-based virtual agent, powered by a large language model (LLM), with three mixed reality visualizations aimed at enhancing patient comfort and trust. The LLM enables the virtual assistant to engage in natural, conversational dialogue with patients, answering questions in any format and offering real-time reassurance, creating a more intelligent and reliable interaction. The virtual assistant is animated as controlling the ultrasound probe, giving the impression that the robot is guided by the assistant. The first visualization employs augmented reality (AR), allowing patients to see the real world and the robot with the virtual avatar superimposed. The second visualization is an augmented virtuality (AV) environment, where the real-world body part being scanned is visible, while a 3D Gaussian Splatting reconstruction of the room, excluding the robot, forms the virtual environment. The third is a fully immersive virtual reality (VR) experience, featuring the same 3D reconstruction but entirely virtual, where the patient sees a virtual representation of their body being scanned in a robot-free environment. In this case, the virtual ultrasound probe, mirrors the movement of the probe controlled by the robot, creating a synchronized experience as it touches and moves over the patient's virtual body. We conducted a comprehensive agent-guided robotic ultrasound study with all participants, comparing these visualizations against a standard robotic ultrasound procedure. Results showed significant improvements in patient trust, acceptance, and comfort. Based on these findings, we offer insights into designing future mixed reality visualizations and virtual agents to further enhance patient comfort and acceptance in autonomous medical procedures.

The advancement and maturity of large language models (LLMs) and robotics have unlocked vast potential for human-computer interaction, particularly in the field of robotic ultrasound. While existing research primarily focuses on either patient-robot or physician-robot interaction, the role of an intelligent virtual sonographer (IVS) bridging physician-robot-patient communication remains underexplored. This work introduces a conversational virtual agent in Extended Reality (XR) that facilitates real-time interaction between physicians, a robotic ultrasound system(RUS), and patients. The IVS agent communicates with physicians in a professional manner while offering empathetic explanations and reassurance to patients. Furthermore, it actively controls the RUS by executing physician commands and transparently relays these actions to the patient. By integrating LLM-powered dialogue with speech-to-text, text-to-speech, and robotic control, our system enhances the efficiency, clarity, and accessibility of robotic ultrasound acquisition. This work constitutes a first step toward understanding how IVS can bridge communication gaps in physician-robot-patient interaction, providing more control and therefore trust into physician-robot interaction while improving patient experience and acceptance of robotic ultrasound. The code is available at https://github.com/stytim/IVS .

DManaging indirect access in laparoscopy as a minimally invasive procedure poses challenges to physicians. In particular, an endoscope must be navigated to achieve adequate visualization of the surgical anatomy, while coping with unergonomic poses, tremor, and fatigue. Furthermore, the alignment of visual perception and physical movement, dictated by the endoscope's position relative to the monitor, can lead to hand-eye coordination challenges. We propose unified deployment of a robotic endoscope holder together with an augmented reality display to counteract the aforementioned challenges in laparoscopy. Our augmented reality system provides an interactive, stereoscopic, virtual monitor displaying an endoscopic stream. In addition, our method design enables direct control of the robotic endoscope holder. Our user study demonstrates the potential of the proposed method to significantly improve hand-eye coordination, while insights from our usability study for robotic control indicate promising trends, including high usability and low cognitive demand.

The use of Augmented Reality (AR) for navigation purposes has shown beneficial in assisting physicians during the performance of surgical procedures. These applications commonly require knowing the pose of surgical tools and patients to provide visual information that surgeons can use during the task performance. Existing medical-grade tracking systems use infrared cameras placed inside the Operating Room (OR) to identify retro-reflective markers attached to objects of interest and compute their pose. Some commercially available AR Head-Mounted Displays (HMDs) use similar cameras for self-localization, hand tracking, and estimating the objects' depth. This work presents a framework that uses the built-in cameras of AR HMDs to enable accurate tracking of retro-reflective markers, such as those used in surgical procedures, without the need to integrate any additional components. This framework is also capable of simultaneously tracking multiple tools. Our results show that the tracking and detection of the markers can be achieved with an accuracy of 0.09 +- 0.06 mm on lateral translation, 0.42 +- 0.32 mm on longitudinal translation, and 0.80 +- 0.39 deg for rotations around the vertical axis. Furthermore, to showcase the relevance of the proposed framework, we evaluate the system's performance in the context of surgical procedures. This use case was designed to replicate the scenarios of k-wire insertions in orthopedic procedures. For evaluation, two surgeons and one biomedical researcher were provided with visual navigation, each performing 21 injections. Results from this use case provide comparable accuracy to those reported in the literature for AR-based navigation procedures.

Root canal therapy (RCT) is a widely performed procedure in dentistry, with over 25 million individuals undergoing it annually. This procedure is carried out to address inflammation or infection within the root canal system of affected teeth. However, accurately aligning CT scan information with the patient's tooth has posed challenges, leading to errors in tool positioning and potential negative outcomes. To overcome these challenges, a mixed reality application is developed using an optical see-through head-mounted display (OST-HMD). The application incorporates visual cues, an augmented mirror, and dynamically updated multi-view CT slices to address depth perception issues and achieve accurate tooth localization, comprehensive canal exploration, and prevention of perforation during RCT. Through the preliminary experimental assessment, significant improvements in the accuracy of the procedure are observed. Specifically, with the system the accuracy in position was improved from 1.4 to 0.4 mm (more than a 70% gain) using an Optical Tracker (NDI) and from 2.8 to 2.4 mm using an HMD, thereby achieving submillimeter accuracy with NDI. 6 participants were enrolled in the user study. The result of the study suggests that the average displacement on the crown plane of 1.27 ± 0.83 cm, an average depth error of 0.90 ± 0.72 cm and an average angular deviation of 1.83 ± 0.83°. Our error analysis further highlights the impact of HMD spatial localization and head motion on the registration and calibration process. Through seamless integration of CT image information with the patient's tooth, our mixed reality application assists dentists in achieving precise tool placement. This advancement in technology has the potential to elevate the quality of root canal procedures, ensuring better accuracy and enhancing overall treatment outcomes.

The utilization of augmented reality (AR) in medical robotics offers significant advancements in enhancing procedural accuracy and patient safety. This paper investigates novel AR visualization techniques designed to depict in-contact force applied by a robotic ultrasound probe, aiming to optimize the control practitioners have over probe force for ultrasound procedures, thereby enhancing both image quality and patient comfort. We developed and evaluated four distinct AR visualization techniques through a comprehensive user study conducted in a clinical setting. The study assessed the efficiency and user experience associated with each technique. The findings revealed notable differences in user performance and preferences, indicating that specific visualizations significantly improve the precision of force application and could lead to better procedural outcomes. The results underscore the potential of AR visualizations to transform robotic-assisted medical procedures by improving the interface between clinicians and robotic systems. Moreover, these advancements foster a deeper trust and acceptance of robotic technologies among healthcare professionals and patients. This study not only highlights the immediate benefits of AR in enhancing robotic ultrasound but also sets the stage for further research into AR's expansive role in complex medical robotics scenarios.

目的 研究稀土对液相等离子体电解渗碳层组织结构和性能的影响.方法 将稀土 LaCl3·7H2O 和CeCl3·7H2O添加到电解液中,在17-4PH不锈钢表面制备有无稀土添加的液相等离子体电解渗碳层.通过扫描电子显微镜、金相显微镜、X射线衍射仪分析渗层的表面形貌、截面组织和相结构,利用维氏硬度计、洛氏硬度计和摩擦磨损试验机评价渗层的硬度、塑韧性和耐磨性.结果 渗碳层主要由碳化物、"膨胀"α相和少量铁氧化物组成,稀土LaCl3·7H2O和CeCl3·7H2O均可以促进等离子体电解渗碳层表面碳化物的生成,且稀土CeCl3·7H2O可以有效抑制渗层表面铁氧化物的生成.添加稀土LaCl3·7H2O和CeCl3·7H2O后,渗层表面多孔化合物层厚度由 20 μm分别减小至 15 μm和 8 μm,致密层+扩散层的厚度从 20 μm分别增加至46 μm和 45 μm.添加稀土LaCl3·7H2O和CeCl3·7H2O后,渗层的有效硬化层厚度可达 70 μm,是不加稀土时的 3倍以上,截面硬度呈梯度分布.添加稀土LaCl3·7H2O和CeCl3·7H2O后,渗层表面洛氏压痕附近的径向裂纹出现了明显的偏转.添加稀土LaCl3·7H2O可使摩擦前期摩擦因数显著降低至 0.14,磨痕宽度减至 534 μm,主要发生氧化磨损、化合物层剥落和磨粒磨损,而添加稀土CeCl3·7H2O可使摩擦因数一直维持在 0.21左右,磨痕宽度显著减少至226 μm,主要发生轻微的磨粒磨损.结论 稀土LaCl3·7H2O和CeCl3·7H2O均可以改善渗层表面质量,提高等离子体电解渗碳层的耐磨性,且稀土CeCl3·7H2O的效果更显著.

•20 differential bacterial taxa (mainly belonged to Firmicutes) were identified in AD.•Differential bacterial taxa were closely involved in amino acid metabolism.•Differential microbial metabolites were closely involved in amino acid metabolism.•Six potential metabolic biomarkers for diagnosing AD from HC were identified.

In the rapidly evolving field of computer vision, efficient and accurate annotation of 3D scenes plays a crucial role. While automation has streamlined this process, manual intervention is still essential for obtaining precise annotations. Existing annotation tools often lack intuitive interactions and efficient interfaces, particularly when it comes to annotating complex elements such as 3D bounding boxes, 6D human poses, and semantic relationships in a 3D scene. Therefore, it is often time-consuming and error-prone. Emerging technologies such as augmented reality (AR) and virtual reality (VR) have shown potential to provide an immersive and interactive environment for annotators to label objects and their relationships. However, the cost and accessibility of these technologies can be a barrier to their widespread adoption. This work introduces a novel tablet-based system that utilizes built-in motion tracking to facilitate an efficient and intuitive 3D scene annotation process. The system supports a variety of annotation tasks and leverages the tracking and mobility features of the tablet to enhance user interactions. Through a thorough user study investigating three distinct tasks - creating bounding boxes, adjusting human poses, and annotating scene relationships - we evaluate the effectiveness and usability of two interaction methods: touch-based interactions and hybrid interactions that utilize both touch and device motion tracking. Our results suggest that leveraging the tablet’s motion tracking feature could lead to more intuitive and efficient annotation processes. This work contributes to the understanding of tablet-based interaction and the potential it holds for annotating complex 3D scenes.

Navigated surgery enables physicians to perform complex tasks assisted by virtual representations of surgical tools and anatomical structures visualized using intraoperative medical images. Integrating Augmented Reality (AR) in these scenarios enriches the virtual information presented to the surgeon by utilizing a wide range of visualization techniques. In this work, we introduce a novel approach to conveying additional depth and shape information of the augmented content using dynamic visualization techniques. Compared to existing works, these techniques allow users to gather information not only from pictorial but also from kinetic depth cues.

ABSTRACT Digital Displays are integral to modern systems and serve as the primary visual human-computer interface. A physical monitor is a typical example for showing computer-generated information to a person – as long they are located directly in front of them. In an operating room, however, people may be unable to move freely or communicate only based on verbal communication due to noise pollution. Furthermore, mobile and connected editing and sharing of the same content on the go is currently not readily available. Therefore, we propose Augmented Reality Collaborative Medical Displays for duplicating any displayed content as a virtual monitor hovering in space in the real world. Our method replicates any user input on the virtual representation onto the original physical monitor. We put forward a use case in planning a surgical intervention of the abdomen with intra-operative CT imaging. Moreover, we demonstrate how a medical immersive teleconsultation system utilises our method for meaningful interactions. Finally, we evaluate our method with multiple display resolutions. Based on our observations from the use cases and quantitative evaluation, we believe our proposed concept facilitates the integration of future collaborative medical applications.

A new two-dimensional (2D) nickel(II) coordination polymer (Ni-CP) has been synthesized hydrothermally, named as [Ni(DDB)0.5(2,2′-bipy)(H2O)]·H2O (H4DDB = 1,4-di(3,5-dicarboylphenoxy) benzene and 2,2′-bipy = 2,2′-bipyridine), and characterized by element analysis, TGA, PXRD, SEM and XPS techniques. Structural analysis shows that the coordination polymer possesses 2D waved network thought DDB4- ligand in μ4 fashion. The Ag-loaded product (Ag@Ni-CP) has been prepared by light reduction method. Photocatalytic degradation performance of Ni-CP and Ag@Ni-CP were investigated, and the latter exhibits excellent degradation effect, especially for Rhodamine B and methylene blue with the high degradation rate of 99% rapidly in 60 min. Compared with the reported MOFs catalyst materials, Ag@Ni-CP exhibits higher and faster degradation performance. The photocatalytic mechanism was investigated by free radical trapping experiments. The electrochemical performance tests show that Ni-CP has good electrical capacity and cycle stability, which means it might be a good supercapacitor in the future.

田汉先生对中国戏剧事业有着不可磨灭的贡献.他的人生追求以及精神大量地凝注于其作品中,《关汉卿》便是他后期作品中的一抹亮色.本文试从三个方面分析该剧所体现的田汉先生的是非观念、人生追求和创作理念;并且我们也可以看到,剧中的"恶魔性"正是田汉先生将中西文化融合化用的结果.

Head-mounted loupes can increase the user's visual acuity to observe the details of an object. On the other hand, optical see-through head-mounted displays (OST-HMD) are able to provide virtual augmentations registered with real objects. In this article, we propose AR-Loupe, combining the advantages of loupes and OST-HMDs, to offer augmented reality in the user's magnified field-of-vision. Specifically, AR-Loupe integrates a commercial OST-HMD, Magic Leap One, and binocular Galilean magnifying loupes, with customized 3D-printed attachments. We model the combination of user's eye, screen of OST-HMD, and the optical loupe as a pinhole camera. The calibration of AR-Loupe involves interactive view segmentation and an adapted version of stereo single point active alignment method (Stereo-SPAAM). We conducted a two-phase multi-user study to evaluate AR-Loupe. The users were able to achieve sub-millimeter accuracy ( 0.82 mm) on average, which is significantly ( ) smaller compared to normal AR guidance ( 1.49 mm). The mean calibration time was 268.46 s. With the increased size of real objects through optical magnification and the registered augmentation, AR-Loupe can aid users in high-precision tasks with better visual acuity and higher accuracy.

Real-time 3D reconstruction using multiple RGBD cameras and their online transmission facilitates the adoption of mixed reality telepresence. However, such a system can only cover a limited volume, and increasing the number of RGBD cameras is unfeasible due to setup complexity and space constraints. To address this issue, we present the concept of Dynamic 3D View Sharing, which complements the views of a 3D reconstruction system by the dynamic view of the user's HMD. Here, we present a markerless calibration method integrating these two seamlessly into the mixed reality telepresence systems without disrupting the current workflow.