The in-situ endograft fenestration, a possible surgical option for the minimally invasive treatment of aneurysms with unfavorable anatomy, is today limited by difficulties in targeting the fenestration site and by the lack of a safe method to perforate the graft. In this work we suggest the use of: a 3D electromagnetic (EM) navigator, to accurately guide the endovascular instruments to the target, and a laser system, to selectively perforate the graft. More particularly we propose to integrate a laser fiber into a sensorized guidewire and we describe an EM sensorization strategy to accurately guide the laser tool. Finally we preliminary explore different laser irradiation conditions to achieve a successful endograft fenestration and we verify that the heating generated by the laser doesn’t damage the EM coils.
We present a longitudinal statistical shape/volumetric model of the cranium of infants and use it as prior information to support the design of cranial shape correction helmets. In addition, a logical approach based on natural brain growth is considered to derive necessary formulation to calculate the required treatment duration. The respective morphological analysis and statistical model will be integrated into the current haptic-based design pipeline of a company to produce an effective computer-assisted design system.
Respiratory motion is a limiting factor during cancer therapy. Although image tracking can facilitate compensation for this motion, system latencies will still reduce the accuracy of tracking-based treatments. We propose a novel approach for temporal prediction of the motion of anatomical targets in the liver, observed from ultrasound sequences. The method is based on an ensemble of six prediction models, including neural networks, which are trained on motion traces and images. Using leave-one-subject-out validation on 24 liver ultrasound 2D sequences from the Challenge on Liver Ultrasound Tracking, the best performance was achieved by the linear regression-based ensemble of all methods with an accuracy of 1.49 (2.39) mm for a latency of 300 (600) ms.
Statistical shape models (SSMs) play an important role in medical image analysis. A sufficiently large number of high quality datasets is needed in order to create a SSM containing all possible shape variations. However, the available datasets may contain corrupted or missing data due to the fact that clinical images are often captured incompletely or contain artifacts. In this work, we propose a weighted Robust Principal Component Analysis (WRPCA) method to create SSMs from incomplete or corrupted datasets. In particular, we introduce a weighting scheme into the conventional Robust Principal Component Analysis (RPCA) algorithm in order to discriminate unusable data from meaningful ones in the decomposition of the training data matrix more accurately. For evaluation, the proposed WRPCA is compared with conventional RPCA on both corrupted (63 CT datasets of the liver) and incomplete datasets (15 MRI datasets of the human foot). The results show a significant improvement in terms of reconstruction accuracy on both datasets.
Neurosurgery is one of the key areas for computer-assisted navigation. By providing an intuitive visualization and feedback to the operator, augmented reality has the potential to provide improved clinical workflows and patient outcomes. In this work, a mobile augmented reality system for surgical neuronavigation is presented, allowing for a direct projection of anatomical information on both planar and non-planar surfaces. The system consists of a tracked mobile laser projector, integrated within a navigation system, providing registered patient surface and image data. Using this setup, image data can be corrected for distortions and displayed accurately on the body surface in relation to the patient. We present the overall system with an efficient distortion correction, and evaluate the accuracy with a series of experiments for point and surface projection errors, as well as the general clinical applicability. The system is demonstrated for the projection of craniotomy incision lines and general image data directly onto the surface of a human skull model, where an average error of 1.04 mm shows a sufficient accuracy with respect to the clinical application.
There are many proven problems associated with traditional surgical planning methods for orthognathic surgery. To address these problems, we developed a computer-aided surgical simulation (CASS) system, the AnatomicAligner, to plan orthognathic surgery following our streamlined clinical protocol. The system includes six modules: image segmentation and three-dimensional (3D) reconstruction, registration and reorientation of models to neutral head posture, 3D cephalometric analysis, virtual osteotomy, surgical simulation, and surgical splint generation. The accuracy of the system was validated in a stepwise fashion: first to evaluate the accuracy of AnatomicAligner using 30 sets of patient data, then to evaluate the fitting of splints generated by AnatomicAligner using 10 sets of patient data. The industrial gold standard system, Mimics, was used as the reference. When comparing the results of segmentation, virtual osteotomy and transformation achieved with AnatomicAligner to the ones achieved with Mimics, the absolute deviation between the two systems was clinically insignificant. The average surface deviation between the two models after 3D model reconstruction in AnatomicAligner and Mimics was 0.3 mm with a standard deviation (SD) of 0.03 mm. All the average surface deviations between the two models after virtual osteotomy and transformations were smaller than 0.01 mm with a SD of 0.01 mm. In addition, the fitting of splints generated by AnatomicAligner was at least as good as the ones generated by Mimics. We successfully developed a CASS system, the AnatomicAligner, for planning orthognathic surgery following the streamlined planning protocol. The system has been proven accurate. AnatomicAligner will soon be available freely to the boarder clinical and research communities.
It is clinically important to accurately predict facial soft tissue changes following bone movements in orthognathic surgical planning. However, the current simulation methods are still problematic, especially in clinically critical regions, e.g., the nose, lips and chin. In this study, finite element method (FEM) simulation model with realistic tissue sliding effects was developed to increase the prediction accuracy in critical regions. First, the facial soft-tissue-change following bone movements was simulated using FEM with sliding effect with nodal force constraint. Subsequently, sliding effect with a nodal displacement constraint was implemented by reassigning the bone-soft tissue mapping and boundary condition for realistic sliding movement simulation. Our method has been quantitatively evaluated using 30 patient datasets. The FEM simulation method with the realistic sliding effects showed significant accuracy improvement in the whole face and the critical areas (i.e., lips, nose and chin) in comparison with the traditional FEM method.
Augmented reality (AR) is widely used in minimally invasive surgery (MIS), since it enhances the surgeon’s perception of spatial relationship by overlaying the invisible structures on the endoscopic images. Depth perception is the key problem in AR visualization. In this paper, we present a video-based AR system for aiding MIS of removing a tumor inside a kidney. We explore several different AR visualization techniques. They are transparent overlay, virtual window, random-dot mask and the ghosting method. We also introduce the depth-aware ghosting method to further enhance the depth perception of virtual structure which has complex spatial geometry. Both simulated and in vivo experiments were carried out to evaluate these AR visualization techniques. The experimental results demonstrate the feasibility of our AR system and AR visualization techniques. Finally, we conclude the characteristics of these AR visualization techniques.
We propose an automatic fast-registration augmented reality (AR) surgical navigation system for minimally invasive surgery. The system integrates a fast-registration technique with three-dimensional (3D) integral videography (IV) image overlay. The detailed anatomic information generated by IV technique is superimposed to the patient using 3D autostereoscopic images, which reproduce motion parallax with naked eyes. To reduce the patient-3D overlay image registration time and achieve automatic execution, we integrate a 3D image overlay system with a real-time patient tracking system, which utilizes particle filter algorithm and depth camera. Experimental results showed that the system can lower the registration time and the can reach up to 10 frames per second (fps). The 3D overlay image to the patient registration average error is 1.88 mm, with standard deviation of 0.72 mm. Further work for improvement of the depth camera acquisition accuracy and tracking algorithm makes this system more feasible and practical.
The recent success of convolutional neural networks in many computer vision tasks implies that their application could also be beneficial for vision tasks in cardiac electrophysiology procedures which are commonly carried out under guidance of C-arm fluoroscopy. Many efforts for catheter detection and reconstruction have been made, but especially robust detection of catheters in X-ray images in realtime is still not entirely solved. We propose two novel methods for (i) fully automatic electrophysiology catheter electrode detection in interventional X-ray images and (ii) single-view depth estimation of such electrodes based on convolutional neural networks. For (i), experiments on 24 different fluoroscopy sequences (1650 X-ray images) yielded a detection rate > 99 6.08 ± 4.66 mm.
Dynamic contrast-enhanced MRI (DCE-MRI) acquires T1-weighted MRI scans before and after injection of an MRI contrast agent such as gadolinium (Gd). Gadolinium causes the relaxation time to decrease, resulting in higher MR image intensities after injection followed by a gradual decrease in image intensities during wash out. Gd does not pass the intact blood–brain barrier (BBB), thus its dynamics can be used to quantify pathology associated with BBB leaks. In current clinical practice, it is suggested to use the same pulse sequence for pre-injection T1 calibration and Gd concentration calculation in the DCE image sequence based on the spoiled gradient recalled echo (SPGR) signal equation. A common method for T1 estimation is using variable flip angle (VFA). However, when the parameters such as the repetition time (TR) for image acquisition could be tuned differently for T1 estimation and DCE acquisition, the popular dcemriS4 software package that handles only a fixed TR often results in discrepancies in Gd concentration estimation. This paper reports a quick solution for calculating Gd concentrations when different TRs are used. First, the pre-injection T1 map is calculated by using the Levenberg-Marquardt algorithm with VFA acquisition, then, because the TR used for DCE acquisition is different from the VFA TR, the equilibrium magnetization is updated with the TR for DCE, and the Gd concentration is calculated thereafter. In the experiments, we first simulated Gd concentration curves for different tissue types and generated the corresponding VFA and DCE image sequences and then used the proposed method to reconstruct the concentration. Comparing with the original simulated data allows us to validate the accuracy of the proposed computation. Further, we tested performance of the method by simulating different amounts of Ktrans changes in a manually selected region of interest (ROI). The results showed that the new method can estimate Gd dynamics more accurately in the case where different TRs are used and be sensitive enough to detect slight Ktrans changes in DCE-MRI.
Periacetabular osteotomy (PAO) is an effective approach for surgical treatment of hip dysplasia in young adults. However, achieving an optimal acetabular reorientation during PAO is the most critical and challenging step. Routinely, the correct positioning of the acetabular fragment largely depends on the surgeons experience and is done under fluoroscopy to provide the surgeon with continuous live x-ray guidance. To address these challenges, we developed a computer assisted system. Our system starts with a fully automatic detection of the acetabular rim, which allows for quantifying the acetabular 3D morphology with parameters such as acetabular orientation, femoral head Extrusion Index (EI), Lateral Center Edge (LCE) angle, total and regional femoral head coverage (FHC) ratio for computer assisted diagnosis, planning and simulation of PAO. Intra-operative navigation is used to implement the pre-operative plan. Two validation studies were conducted on four sawbone models to evaluate the efficacy of the system intra-operatively and post-operatively. By comparing the pre-operatively planned situation with the intra-operatively achieved situation, average errors of \(0.6^\circ \pm 0.3^\circ \), \(0.3^\circ \pm 0.2^\circ \) and \(1.1^\circ \pm 1.1^\circ \) were found respectively along three motion directions (Flexion/Extension, Abduction/Adduction and External Rotation/Internal Rotation). In addition, by comparing the pre-operatively planned situation with the post-operative results, average errors of \(0.9^\circ \pm 0.3^\circ \) and \(0.9^\circ \pm 0.7^\circ \) were found for inclination and anteversion, respectively.
One of the remaining challenges in event-related fMRI is to discriminate between the vascular response and the neural activity in the BOLD signal. This discrimination is done by identifying the hemodynamic territories which differ in their underlying dynamics. In the literature, many approaches have been proposed to estimate these underlying dynamics, which is also known as Hemodynamic Response Function (HRF). However, most of the proposed approaches depend on a prior information regarding the shape of the parcels (territories) and their number. In this paper, we propose a novel approach which relies on the adaptive mean shift algorithm for the parcellation of the brain. A variational inference is used to estimate the unknown variables while the mean shift is embedded within a variational expectation maximization (VEM) framework to allow for estimating the parcellation and the HRF profiles without having any prior information about the number of the parcels or their shape. Results on synthetic data confirms the ability of the proposed approach to estimate accurate HRF estimates and number of parcels. It also manages to discriminate between voxels in different parcels especially at the borders between these parcels. In real data experiment, the proposed approach manages to recover HRF estimates close to the canonical shape in the bilateral occipital cortex.
Understanding the structure and function of the muscular system is important in many different medical scenarios, such as student and patient education, muscle training, and rehabilitation of patients with kinetic problems. The structure and function of muscles are difficult to learn as muscle movement is imperceptible using the traditional methods. In this paper, an interactive mixed reality system is proposed for facilitating the learning of the muscles of the upper extremities. The proposed system consists of two main components: an AR view, overlaying the virtual model of the arm on top of the video stream, and a VR view, providing a more detailed understanding of the muscles. The mixed reality view helps the user to mentally map the behaviour of muscles to his/her own body during different movements. A user study including twenty students was performed by a questionnaire framework. The results indicate that our system is useful for learning the structure and function of the muscle and can be a valuable supplement to established muscle learning paradigms.
This paper presents a novel approach for evaluating technical skills in Transoesophageal Echocardiography (TEE). Our core assumption is that operational competency can be objectively expressed by specific motion-based measures. TEE experiments were carried out with an augmented reality simulation platform involving both novice trainees and expert radiologists. Probe motion data were collected and used to formulate various kinematic parameters. Subsequent analysis showed that statistically significant differences exist among the two groups for the majority of the metrics investigated. Experts exhibited lower completion times and higher average velocity and acceleration, attributed to their refined ability for efficient and economical probe manipulation. In addition, their navigation pattern is characterised by increased smoothness and fluidity, evaluated through the measures of dimensionless jerk and spectral arc length. Utilised as inputs to well-known clustering algorithms, the derived metrics are capable of discriminating experience levels with high accuracy (>84
Augmented reality has been proposed as a solution to overcome some of the current shortcomings of image-guided neurosurgery. In particular, it has been used to merge patient images, surgical plans, and the surgical field of view into a comprehensive visualization. In this paper we explore the use of augmented reality for planning craniotomies in image-guided neurosurgery procedures for tumour resections. Our augmented reality image-guided neurosurgery system was brought into the operating room for 8 cases where the surgeon used augmented reality prior to tumour resection. We describe our initial results that suggest that augmented reality can play an important role in tailoring the size and shape of the craniotomy and for evaluating intra-operative surgical strategies. With continued development and validation, augmented reality guidance has the potential to improve the minimally invasiveness of image-guided neurosurgery through improved intraoperative surgical planning.
In this work, we propose a novel and powerful image analysis framework for hippocampal morphometry in early mild cognitive impairment (EMCI), an early prodromal stage of Alzheimer's disease (AD). We create a hippocampal surface atlas with subfield information, model each hippocampus using the SPHARM technique, and register it to the atlas to extract surface deformation signals. We propose a new alternative to standard random field theory (RFT) and permutation image analysis methods, Statistical Parametric Mapping (SPM) Distribution Analysis or SPM-DA, to perform statistical shape analysis and compare its performance with that of RFT methods on both simulated and real hippocampal surface data. The major strengths of our framework are twofold: (a) SPM-DA provides potentially more powerful algorithms than standard RFT methods for detecting weak signals, and (b) the framework embraces the important hippocampal subfield information for improved biological interpretation. We demonstrate the effectiveness of our method via an application to an AD cohort, where an SPM-DA method detects meaningful hippocampal shape differences in EMCI that are undetected by standard RFT methods.
We describe a method for evaluating facial expressivity in order to improve related clinical assessments of Parkinson’s Disease (PD). There is a controversial evidence in the literature that PD facial impairment can be detected on certain emotional expressions. This study aimed to investigate the feasibility of discriminative and quantitive measures of PD from the ability of a subject to express facial expressions. Video clips of 8 subjects (4 healthy controls and 4 with patients with PD) were recorded during daily sessions over several weeks. Observations covered emotion variation over one week for control subjects and six weeks for patients with PD. A statistical shape model was used to track facial expressions and to measure the amount of expressivity exhibited by each subject. The study suggests that measures of the amount of movement during happiness, disgust and anger expressions are the most discriminative, with PD patients exhibiting less movement than controls. This work demonstrates that it may be possible to measure day-to-day variations in symptoms of PD automatically.
Transpyloric tube insertion is a commonly intervention which consists in inserting a tube through the nose, the oesophagus and the stomach until the pylorus. This procedure can be done blindly if no licensed physician operator is available. Thus, the caregivers have to insert the tube without any visual feedbacks. The position of the tube has also to be conformed to avoid any issues. We proposed a guidance system for the insertion of this tube through an augmented reality application using an electromagnetic tracker. The trajectory and the orientation of the tube were visualized on a screen to monitor the tube insertion but also to conform the position of the tube tip inside the pylorus. This monitoring was improved by adding preoperative landmarks coming from a CT image or intraoperative landmarks obtained with ultrasound. Finally, the display was overlaid on a camera view to match the tube trajectory on the patient during the insertion so the clinicians directly visualized anatomical landmarks to guide themselves during the tube insertion. Preliminary results on one phantom showed the usefulness of our guidance system for blindly tube insertion. Further evaluations have to be realized on animals and on patients to validate these results.
