Patients with congenital heart disease often have cardiac anatomy that deviates significantly from normal, frequently requiring multiple heart surgeries. Image segmentation from a preoperative cardiovascular magnetic resonance (CMR) scan would enable creation of patient-specific 3D surface models of the heart, which have potential to improve surgical planning, enable surgical simulation, and allow automatic computation of quantitative metrics of heart function. However, there is no publicly available CMR dataset for whole-heart segmentation in patients with congenital heart disease. Here, we release the HVSMR-2.0 dataset, comprising 60 CMR scans alongside manual segmentation masks of the 4 cardiac chambers and 4 great vessels. The images showcase a wide range of heart defects and prior surgical interventions. The dataset also includes masks of required and optional extents of the great vessels, enabling fairer comparisons across algorithms. Detailed diagnoses for each subject are also provided. By releasing HVSMR-2.0, we aim to encourage development of robust segmentation algorithms and clinically relevant tools for congenital heart disease.

Introduction As the life expectancy of children with congenital heart disease (CHD) is rapidly increasing and the adult population with CHD is growing, there is an unmet need to improve clinical workflow and efficiency of analysis. Cardiovascular magnetic resonance (CMR) is a noninvasive imaging modality for monitoring patients with CHD. CMR exam is based on multiple breath-hold 2-dimensional (2D) cine acquisitions that should be precisely prescribed and is expert and institution dependent. Moreover, 2D cine images have relatively thick slices, which does not allow for isotropic delineation of ventricular structures. Thus, development of an isotropic 3D cine acquisition and automatic segmentation method is worthwhile to make CMR workflow straightforward and efficient, as the present work aims to establish. Methods Ninety-nine patients with many types of CHD were imaged using a non-angulated 3D cine CMR sequence covering the whole-heart and great vessels. Automatic supervised and semi-supervised deep-learning-based methods were developed for whole-heart segmentation of 3D cine images to separately delineate the cardiac structures, including both atria, both ventricles, aorta, pulmonary arteries, and superior and inferior vena cavae. The segmentation results derived from the two methods were compared with the manual segmentation in terms of Dice score, a degree of overlap agreement, and atrial and ventricular volume measurements. Results The semi-supervised method resulted in a better overlap agreement with the manual segmentation than the supervised method for all 8 structures (Dice score 83.23 ± 16.76% vs. 77.98 ± 19.64%; P -value ≤0.001). The mean difference error in atrial and ventricular volumetric measurements between manual segmentation and semi-supervised method was lower (bias ≤ 5.2 ml) than the supervised method (bias ≤ 10.1 ml). Discussion The proposed semi-supervised method is capable of cardiac segmentation and chamber volume quantification in a CHD population with wide anatomical variability. It accurately delineates the heart chambers and great vessels and can be used to accurately calculate ventricular and atrial volumes throughout the cardiac cycle. Such a segmentation method can reduce inter- and intra- observer variability and make CMR exams more standardized and efficient.

Training deep learning models that segment an image in one step typically requires a large collection of manually annotated images that captures the anatomical variability in a cohort. This poses challenges when anatomical variability is extreme but training data is limited, as when segmenting cardiac structures in patients with congenital heart disease (CHD). In this paper, we propose an iterative segmentation model and show that it can be accurately learned from a small dataset. Implemented as a recurrent neural network, the model evolves a segmentation over multiple steps, from a single user click until reaching an automatically determined stopping point. We develop a novel loss function that evaluates the entire sequence of output segmentations, and use it to learn model parameters. Segmentations evolve predictably according to growth dynamics encapsulated by training data, which consists of images, partially completed segmentations, and the recommended next step. The user can easily refine the final segmentation by examining those that are earlier or later in the output sequence. Using a dataset of 3D cardiac MR scans from patients with a wide range of CHD types, we show that our iterative model offers better generalization to patients with the most severe heart malformations.

An important factor for the development of spinal degeneration, pain and the outcome of spinal surgery is known to be the balance of the spine. It must be analyzed in an upright, standing position to ensure physiological loading conditions and visualize load-dependent deformations. Despite the complex 3D shape of the spine, this analysis is currently performed using 2D radiographs, as all frequently used 3D imaging techniques require the patient to be scanned in a prone position. To overcome this limitation, we propose a deep neural network to reconstruct the 3D spinal pose in an upright standing position, loaded naturally. Specifically, we propose a novel neural network architecture, which takes orthogonal 2D radiographs and infers the spine’s 3D posture using vertebral shape priors. In this work, we define vertebral shape priors using an atlas and a spine shape prior, incorporating both into our proposed network architecture. We validate our architecture on digitally reconstructed radiographs, achieving a 3D reconstruction Dice of 0.95, indicating an almost perfect 2D-to-3D domain translation. Validating the reconstruction accuracy of a 3D standing spine on real data is infeasible due to the lack of a valid ground truth. Hence, we design a novel experiment for this purpose, using an orientation invariant distance metric, to evaluate our model’s ability to synthesize full-3D, upright, and patient-specific spine models. We compare the synthesized spine shapes from clinical upright standing radiographs to the same patient’s 3D spinal posture in the prone position from CT.

Provide a summary of the challenge purpose. This should include a general introduction in the topic from both a biomedical as well as from a technical point of view and clearly state the envisioned technical and/or biomedical impact of the challenge. Congenital heart disease (CHD) encompassing These include: shape changes within a vessel or chamber, abnormal connectivity between cardiac structures, abnormal structure locations, duplicated structures, and/or missing structures. The heart of each CHD patient is different, with a unique combination of heart defects from birth, new atypical connections and implants from prior surgeries, and shape changes from long-term cardiac remodeling. surgical may appreciation of the true locations and sizes the surgical from to research for many years, and will encourage development of algorithms that bring automated analysis to a variety of new clinical applications involving extreme anatomical variability. Abstract Provide a summary of the challenge purpose. This should include a general introduction in the topic from both a biomedical as well as from a technical point of view and clearly state the envisioned technical and/or biomedical impact of the challenge.

Hypoplastic left heart syndrome (HLHS) is a severe congenital heart defect in which the right ventricle and associated tricuspid valve (TV) alone support the circulation. TV failure is thus associated with heart failure, and the outcome of TV valve repair are currently poor. 3D echocardiography (3DE) can generate high-quality images of the valve, but segmentation is necessary for precise modeling and quantification. There is currently no robust methodology for rapid TV segmentation, limiting the clinical application of these technologies to this challenging population. We utilized a Fully Convolutional Network (FCN) to segment tricuspid valves from transthoracic 3DE. We trained on 133 3DE image-segmentation pairs and validated on 28 images. We then assessed the effect of varying inputs to the FCN using Mean Boundary Distance (MBD) and Dice Similarity Coefficient (DSC). The FCN with the input of an annular curve achieved a median DSC of 0.86 [IQR: 0.81–0.88] and MBD of 0.35 [0.23–0.4] mm for the merged segmentation and an average DSC of 0.77 [0.73–0.81] and MBD of 0.6 [0.44–0.74] mm for individual TV leaflet segmentation. The addition of commissural landmarks improved individual leaflet segmentation accuracy to an MBD of 0.38 [0.3–0.46] mm. FCN-based segmentation of the tricuspid valve from transthoracic 3DE is feasible and accurate. The addition of an annular curve and commissural landmarks improved the quality of the segmentations with MBD and DSC within the range of human inter-user variability. Fast and accurate FCN-based segmentation of the tricuspid valve in HLHS may enable rapid modeling and quantification, which in the future may inform surgical planning. We are now working to deploy this network for public use.

We propose a new iterative segmentation model which can be accurately learned from a small dataset. A common approach is to train a model to directly segment an image, requiring a large collection of manually annotated images to capture the anatomical variability in a cohort. In contrast, we develop a segmentation model that recursively evolves a segmentation in several steps, and implement it as a recurrent neural network. We learn model parameters by optimizing the interme- diate steps of the evolution in addition to the final segmentation. To this end, we train our segmentation propagation model by presenting incom- plete and/or inaccurate input segmentations paired with a recommended next step. Our work aims to alleviate challenges in segmenting heart structures from cardiac MRI for patients with congenital heart disease (CHD), which encompasses a range of morphological deformations and topological changes. We demonstrate the advantages of this approach on a dataset of 20 images from CHD patients, learning a model that accurately segments individual heart chambers and great vessels. Com- pared to direct segmentation, the iterative method yields more accurate segmentation for patients with the most severe CHD malformations.

This book constitutes the refereed proceedings of two workshops held at the 19th International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2016, in Athens, Greece, in October 2016: the First International Workshop on Reconstruction and Analysis of Moving Body Organs, RAMBO 2016, and the First International Workshop on Whole-Heart and Great Vessel Segmentation from 3D Cardiovascular MRI in Congenital Heart Disease, HVSMR 2016. The 17 revised regular papers presented in this book were carefully reviewed and selected from a total of 21 submissions. The papers cover following topics: Registration; Reconstruction; Deep learning for heart segmentation; Discrete optimization and probabilistic intensity modeling; Atlas-based strategies; Random forests.

We present an interactive algorithm to segment the heart chambers and epicardial surfaces, including the great vessel walls, in pediatric cardiac MRI of congenital heart disease. Accurate whole-heart segmentation is necessary to create patient-specific 3D heart models for surgical planning in the presence of complex heart defects. Anatomical variability due to congenital defects precludes fully automatic atlas-based segmentation. Our interactive segmentation method exploits expert segmentations of a small set of short-axis slice regions to automatically delineate the remaining volume using patch-based segmentation. We also investigate the potential of active learning to automatically solicit user input in areas where segmentation error is likely to be high. Validation is performed on four subjects with double outlet right ventricle, a severe congenital heart defect. We show that strategies asking the user to manually segment regions of interest within short-axis slices yield higher accuracy with less user input than those querying entire short-axis slices.

Creating 3D Heart Models of Children with Congenital Heart Disease using Magnetic Resonance Imaging Danielle F. Pace, Polina Golland, David Annese, Tal Geva, Andrew J. Powell, and Mehdi H. Moghari Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, Cambridge, MA, United States, Department of Cardiology, Boston Children's Hospital, Boston, MA, United States, Department of Pediatrics, Harvard Medical School, Boston, MA, United States

Traditional deformable image registration imposes a uniform smoothness constraint on the deformation field. This is not appropriate when registering images visualizing organs that slide relative to each other, and therefore leads to registration inaccuracies. In this paper, we present a deformation field regularization term that is based on anisotropic diffusion and accommodates the deformation field discontinuities that are expected when considering sliding motion. The registration algorithm was assessed first using artificial images of geometric objects. In a second validation, monomodal chest images depicting both respiratory and cardiac motion were generated using an anatomically-realistic software phantom and then registered. Registration accuracy was assessed based on the distances between corresponding segmented organ surfaces. Compared to an established diffusive regularization approach, the anisotropic diffusive regularization gave deformation fields that represented more plausible image correspondences, while giving rise to similar transformed moving images and comparable registration accuracy.

An approach to study population differences in cerebral vasculature is proposed. This is done by (1) extending the concept of encoding cerebral blood vessel networks as spatial graphs and (2) quantifying graph similarity in a kernel-based discriminant classifier setup. We argue that augmenting graph vertices with information about their proximity to selected brain structures adds discriminative information and consequently leads to a more expressive encoding. Using graph-kernels then allows us to quantify graph similarity in a principled way. To demonstrate our approach, we assess the hypothesis that gender differences manifest as variations in the architecture of cerebral blood vessels, an observation that previously had only been tested and confirmed for the Circle of Willis. Our results strongly support this hypothesis, i.e, we can demonstrate non-trivial, statistically significant deviations from random gender classification in a cross-validation setup on 40 healthy patients.

Regularization is used in deformable image registration to encourage plausible displacement fields, and significantly impacts the derived correspondences. Sliding motion, such as that between the lungs and chest wall and between the abdominal organs, complicates registration because many regularizations are global smoothness constraints that produce errors at object boundaries. We present locally adaptive regularizations that handle sliding objects with locally planar and tubular geometries. These regularizations allow discontinuities to develop in the displacement field at sliding interfaces and increase the independence with which regions surrounding distinct geometric structures can behave. Validation is performed by registering inhale and exhale abdominal computed tomography (CT) images and artificial images of a sliding tube. The sliding registration methods produce more realistic correspondences that may better reflect the underlying physical motion, while performing as well as the diffusive regularization with respect to image match.

Recent contributions to the body of knowledge on traumatic brain injury (TBI) favor the view that multimodal neuroimaging using structural and functional magnetic resonance imaging (MRI and fMRI, respectively) as well as diffusion tensor imaging (DTI) has excellent potential to identify novel biomarkers and predictors of TBI outcome. This is particularly the case when such methods are appropriately combined with volumetric/morphometric analysis of brain structures and with the exploration of TBI-related changes in brain network properties at the level of the connectome. In this context, our present review summarizes recent developments on the roles of these two techniques in the search for novel structural neuroimaging biomarkers that have TBI outcome prognostication value. The themes being explored cover notable trends in this area of research, including (1) the role of advanced MRI processing methods in the analysis of structural pathology, (2) the use of brain connectomics and network analysis to identify outcome biomarkers, and (3) the application of multivariate statistics to predict outcome using neuroimaging metrics. The goal of the review is to draw the community's attention to these recent advances on TBI outcome prediction methods and to encourage the development of new methodologies whereby structural neuroimaging can be used to identify biomarkers of TBI outcome.

Standard image registration methods do not account for changes in image appearance. Hence, metamorphosis approaches have been developed which jointly estimate a space deformation and a change in image appearance to construct a spatio-temporal trajectory smoothly transforming a source to a target image. For standard metamorphosis, geometric changes are not explicitly modeled. We propose a geometric metamorphosis formulation, which explains changes in image appearance by a global deformation, a deformation of a geometric model, and an image composition model. This work is motivated by the clinical challenge of predicting the long-term effects of traumatic brain injuries based on time-series images. This work is also applicable to the quantification of tumor progression (e.g., estimating its infiltrating and displacing components) and predicting chronic blood perfusion changes after stroke. We demonstrate the utility of the method using simulated data as well as scans from a clinical traumatic brain injury patient.

Surgical procedures often have the unfortunate side-effect of causing the patient significant trauma while accessing the target site. Indeed, in some cases the trauma inflicted on the patient during access to the target greatly exceeds that caused by performing the therapy. Heart disease has traditionally been treated surgically using open chest techniques with the patient being placed "on pump" - i.e. their circulation being maintained by a cardio-pulmonary bypass or "heart-lung" machine. Recently, techniques have been developed for performing minimally invasive interventions on the heart, obviating the formerly invasive procedures. These new approaches rely on pre-operative images, combined with real-time images acquired during the procedure. Our approach is to register intra-operative images to the patient, and use a navigation system that combines intra-operative ultrasound with virtual models of instrumentation that has been introduced into the chamber through the heart wall. This paper illustrates the problems associated with traditional ultrasound guidance, and reviews the state of the art in real-time 3D cardiac ultrasound technology. In addition, it discusses the implementation of an image-guided intervention platform that integrates real-time ultrasound with a virtual reality environment, bringing together the pre-operative anatomy derived from MRI or CT, representations of tracked instrumentation inside the heart chamber, and the intra-operatively acquired ultrasound images.

Image-guided interventions rely on the common assumption that pre-operative information can depict intraoperative morphology with sufficient accuracy. Nevertheless, in the context of minimally invasive cardiac therapy delivery, this assumption loses ground; the heart is a soft-tissue organ prone to changes induced during access to the heart and especially intracardiac targets. In addition to its clinical value for cardiac interventional guidance and assistance with the image-and model-to-patient registration, here we show how ultrasound imaging may be used to estimate changes in the heart position and morphology of structures of interest at different stages in the procedure. Using a magnetically tracked 2D transesophageal echocardiography transducer, we acquired in vivo images of the heart at different stages during the procedural workflow of common minimally invasive cardiac procedures, including robot-assisted coronary artery bypass grafting, mitral valve replacement/repair, or model-enhanced US-guided intracardiac interventions, all in the coordinate system of the tracking system. Anatomical features of interest (mitral and aortic valves) used to register the pre-operative anatomical models to the intraoperative coordinate frame were identified from each dataset. This information allowed us to identify the global position of the heart and also characterize the valvular structures at various peri-operative stages, in terms of their orientation, size, and geometry. Based on these results, we can estimate the differences between the pre- and intra-operative anatomical features, their effect on the model-to-subject registration, and also identify the need to update or optimize any pre-operative surgical plan to better suit the intra-operative procedure workflow.

Objective We report our experience with ultrasound augmented reality (US-AR) guidance for mitral valve prosthesis (MVP) implantation in the pig using off-pump, closed, beating intracardiac access through the Guiraudon Universal Cardiac Introducer attached to the left atrial appendage. Methods Before testing US-AR guidance, a feasibility pilot study on nine pigs was performed using US alone. US-AR guidance, tested on a heart phantom, was subsequently used in three pigs (~65 kg) using a tracked transesophageal echocardiography probe, augmented with registration of a 3D computed tomography scan, and virtual representation of the MVP and clip-delivering tool (Clipper); three pigs were used to test feature-based registration. Results Navigation of the MVP was facilitated by the 3D anatomic display. AR displayed the MVP and the Clipper within the Atamai Viewer, with excellent accuracy for tool placement. Positioning the Clipper was hampered by the design of the MVP holder and Clipper. These limitations were well displayed by AR, which provided guidance for improved design of tools. Conclusions US-AR provided informative image guidance. It documented the flaws of the current implantation technology. This information could not be obtained by any other method of evaluation. These evaluations provided guidance for designing an integrated tool: combining an unobtrusive valve holder that allows the MVP to function properly as soon as positioned, and an anchoring system, with clips that can be released one at a time, and retracted if necessary, for optimal results. The portability of Real-time US-AR may prove to be the ideal practical image guidance system for all closed intracardiac interventions.

Anesthetic nerve blocks are a common therapy performed in hospitals around the world to alleviate acute and chronic pain. Tracking systems have shown considerable promise in other forms of therapy, but little has been done to apply this technology in the field of anesthesia. We are developing a guidance system for combining tracked needles with non-invasive ultrasound (US) and patient-specific geometric models. In experiments with phantoms two augmented reality (AR) guidance systems were compared to the exclusive use of US for lumbar facet injection therapy. Anesthetists and anesthesia residents were able to place needles within 0.57mm of the intended targets using our AR systems compared to 5.77 mm using US alone. A preliminary cadaver study demonstrated the system was able to accurately place radio opaque dye on targets. The combination of real time US with tracked tools and AR guidance has the potential to replace CT and fluoroscopic guidance, thus reducing radiation dose to patients and clinicians, as well as reducing health care costs.

Ultrasound is garnering significant interest as an imaging modality for surgical guidance, due to its affordability, real-time temporal resolution and ease of integration into the operating room. Minimally-invasive intracardiac surgery performed on the beating-heart prevents direct vision of the surgical target, and procedures such as mitral valve replacement and atrial septal defect closure would benefit from intraoperative ultrasound imaging. We propose that placing 4D ultrasound within an augmented reality environment, along with a patient-specific cardiac model and virtual representations of tracked surgical tools, will create a visually intuitive platform with sufficient image information to safely and accurately repair tissue within the beating heart. However, the quality of the imaging parameters, spatial calibration, temporal calibration and ECG-gating must be well characterized before any 4D ultrasound system can be used clinically to guide the treatment of moving structures. In this paper, we describe a comprehensive accuracy assessment framework that can be used to evaluate the performance of 4D ultrasound systems while imaging moving targets. We image a dynamic phantom that is comprised of a simple robot and a tracked phantom to which point-source, distance and spherical objects of known construction can be attached. We also follow our protocol to evaluate 4D ultrasound images generated in real-time by reconstructing ECG-gated 2D ultrasound images acquired from a tracked multiplanar transesophageal probe. Likewise, our evaluation framework allows any type of 4D ultrasound to be quantitatively assessed.