BACKGROUND:The reconstruction of a computed tomography (CT) image can be compromised by artifacts, which, in many cases, reduce the diagnostic value of the image. These artifacts often result from missing or corrupt regions in the projection data, for example, by truncation, metal, or limited angle acquisitions. PURPOSE:In this work, we introduce a novel deep learning-based framework, latent space reconstruction (LSR), which enables correction of various types of artifacts arising from missing or corrupted data. METHODS:First, we train a generative neural network on uncorrupted CT images. After training, we iteratively search for the point in the latent space of this network that best matches the compromised projection data we measured. Once an optimal point is found, forward-projection of the generated CT image can be used to inpaint the corrupted or incomplete regions of the measured raw data. RESULTS:We used LSR to correct for truncation and metal artifacts. For the truncation artifact correction, images corrected by LSR show effective artifact suppression within the field of measurement (FOM), alongside a substantial high-quality extension of the FOM compared to other methods. For the metal artifact correction, images corrected by LSR demonstrate effective artifact reduction, providing a clearer view of the surrounding tissues and anatomical details. CONCLUSIONS:The results indicate that LSR is effective in correcting metal and truncation artifacts. Furthermore, the versatility of LSR allows its application to various other types of artifacts resulting from missing or corrupt data.

BACKGROUND:Denoising low dose computed tomography (CT) images can have great advantages for the aim of minimizing the radiation risk of the patients, as it can help lower the effective dose to the patient while providing constant image quality. In recent years, deep denoising methods became a popular way to accomplish this task. Conventional deep denoising algorithms, however, cannot handle the correlation between neighboring pixels or voxels very well, because the noise structure in CT is a resultant of the global attenuation properties of the patient and because the receptive field of most denoising approaches is rather small. PURPOSE:The purpose of this study is to improve existing denoising networks, by providing them additional information about the image noise. METHODS:We here propose to generate N $N$ additional noise realizations by simulation, reconstruct them, and use these noise images as additional input into existing denoising networks. This noise augmentation is intended to guide the denoising process. The additional noise realizations are not only required during training, but also during inference. The rationale behind this noise-augmented deep denoising (NADD) is that CT image noise is strongly patient-specific and it is non-local since it depends on the attenuation of X-ray beams. NADD is architecture-agnostic and can thus be used to improve any previously proposed method. We demonstrate NADD using existing denoising networks that we slightly modified in their input layer in order to take the CT image that is to be denoised plus additional noise images as input. To do so, we modified three popular denoising networks, the CNN10, the ResNet, and the WGAN-VGG and apply them to clinical cases with 90% dose reduction. RESULTS:In all cases tested, the denoising networks strongly benefit from the noise augmentation. Noise artifacts that are being misinterpreted by the original networks as being anatomical structures, are correctly removed by the NADD version of the same networks. The more noise images are provided, the better the performance. CONCLUSIONS:Providing additional simulated noise realizations helps to significantly improve the performance of CT image denoising networks.

Learning-based methods for the restoration of computed tomography (CT) images promise very good image quality even in areas with insufficient data sampling and thus suggest enormous savings in measurement time. This work shows by means of restorations of sparse-view CT data that such methods must be evaluated thoroughly and in a task-specific manner, as details of the workpiece may not be exactly reconstructed. In addition, this work examines the influence of these methods on metrological specification measurements of CTs and the conclusions that can be drawn with regard to the objective specification of such algorithms.

Many methods have been developed to reduce radiation dose in computed tomography (CT) scans without sacrificing image quality. Recently, deep learning-based methods have shown promising results on the task of CT image denoising. However, they remain difficult to interpret, and thus safety concerns have been raised. In this work we develop a method to reconstruct the invariances of arbitrary denoising methods with an approach inspired by the optimization schemes commonly used to generate adversarial examples. We apply our method to one proof-of-principle algorithm as well as to two previously proposed denoising networks and show that it can successfully reconstruct their invariances.

BackgroundDigital subtraction angiography (DSA) is a fluoroscopy method primarily used for the diagnosis of cardiovascular diseases (CVDs). Deep learning-based DSA (DDSA) is developed to extract DSA-like images directly from fluoroscopic images, which helps in saving dose while improving image quality. It can also be applied where C-arm or patient motion is present and conventional DSA cannot be applied. However, due to the lack of clinical training data and unavoidable artifacts in DSA targets, current DDSA models still cannot satisfactorily display specific structures, nor can they predict noise-free images.PurposeIn this study, we propose a strategy for producing abundant synthetic DSA image pairs in which synthetic DSA targets are free of typical artifacts and noise commonly found in conventional DSA targets for DDSA model training.MethodsMore than 7,000 forward-projected computed tomography (CT) images and more than 25,000 synthetic vascular projection images were employed to create contrast-enhanced fluoroscopic images and corresponding DSA images, which were utilized as DSA image pairs for training of the DDSA networks. The CT projection images and vascular projection images were generated from eight whole-body CT scans and 1,584 3D vascular skeletons, respectively. All vessel skeletons were generated with stochastic Lindenmayer systems. We trained DDSA models on this synthetic dataset and compared them to the trainings on a clinical DSA dataset, which contains nearly 4,000 fluoroscopic x-ray images obtained from different models of C-arms.ResultsWe evaluated DDSA models on clinical fluoroscopic data of different anatomies, including the leg, abdomen, and heart. The results on leg data showed for different methods that training on synthetic data performed similarly and sometimes outperformed training on clinical data. The results on abdomen and cardiac data demonstrated that models trained on synthetic data were able to extract clearer DSA-like images than conventional DSA and models trained on clinical data. The models trained on synthetic data consistently outperformed their clinical data counterparts, achieving higher scores in the quantitative evaluation of peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) metrics for DDSA images, as well as accuracy, precision, and Dice scores for segmentation of the DDSA images.ConclusionsWe proposed an approach to train DDSA networks with synthetic DSA image pairs and extract DSA-like images from contrast-enhanced x-ray images directly. This is a potential tool to aid in diagnosis.

BackgroundDeep learning-based methods led to significant advancements in many areas of medical imaging, most of which are concerned with the reduction of artifacts caused by motion, scatter, or noise. However, with most neural networks being black boxes, they remain notoriously difficult to interpret, hindering their clinical implementation. In particular, it has been shown that networks exhibit invariances w.r.t. input features, that is, they learn to ignore certain information in the input data.PurposeTo improve the interpretability of deep learning-based low-dose CT image denoising networks.MethodsWe learn a complete data representation of low-dose input images using a conditional variational autoencoder (cVAE). In this representation, invariances of any given denoising network are then disentangled from the information it is not invariant to using a conditional invertible neural network (cINN). At test time, image-space invariances are generated by applying the inverse of the cINN and subsequent decoding using the cVAE. We propose two methods to analyze sampled invariances and to find those that correspond to alterations of anatomical structures.ResultsThe proposed method is applied to four popular deep learning-based low-dose CT image denoising networks. We find that the networks are not only invariant to noise amplitude and realizations, but also to anatomical structures.ConclusionsThe proposed method is capable of reconstructing and analyzing invariances of deep learning-based low-dose CT image denoising networks. This is an important step toward interpreting deep learning-based methods for medical imaging, which is essential for their clinical implementation.

Long lasting efforts have been made to reduce radiation dose and thus the potential radiation risk to the patient for computed tomography acquisitions without severe deterioration of image quality. To this end, numerous reconstruction and noise reduction algorithms have been developed, many of which are based on iterative reconstruction techniques, incorporating prior knowledge in the projection or image domain. Recently, deep learning-based methods became increasingly popular and a multitude of papers claim ever improving performance both quantitatively and qualitatively. In this work, we find that the lack of a common benchmark setup and flaws in the experimental setup of many publications hinder verifiability of those claims. We propose a benchmark setup to overcome those flaws and improve reproducibility and verifiability of experimental results in the field. In a comprehensive and fair evaluation of several deep learning-based low dose CT denoising algorithms, we find that most methods perform statistically similar and improvements over the past six years have been marginal at best.

BACKGROUND:Vascular diseases are often treated minimally invasively. The interventional material (stents, guidewires, etc.) used during such percutaneous interventions are visualized by some form of image guidance. Today, this image guidance is usually provided by 2D X-ray fluoroscopy, that is, a live 2D image. 3D X-ray fluoroscopy, that is, a live 3D image, could accelerate existing and enable new interventions. However, existing algorithms for the 3D reconstruction of interventional material require either too many X-ray projections and therefore dose, or are only capable of reconstructing single, curvilinear structures.PURPOSE:Using only two new X-ray projections per 3D reconstruction, we aim to reconstruct more complex arrangements of interventional material than was previously possible.METHODS:This is achieved by improving a previously presented deep learning-based reconstruction pipeline, which assumes that the X-ray images are acquired by a continuously rotating biplane system, in two ways: (a) separation of the reconstruction of different object types, (b) motion compensation using spatial transformer networks.RESULTS:Our pipeline achieves submillimeter accuracy on measured data of a stent and two guidewires inside an anthropomorphic phantom with respiratory motion. In an ablation study, we find that the aforementioned algorithmic changes improve our two figures of merit by 75 % (1.76 mm → 0.44 mm) and 59 % (1.15 mm → 0.47 mm) respectively. A comparison of our measured dose area product (DAP) rate to DAP rates of 2D fluoroscopy indicates a roughly similar dose burden.CONCLUSIONS:This dose efficiency combined with the ability to reconstruct complex arrangements of interventional material makes the presented algorithm a promising candidate to enable 3D fluoroscopy.

When domain knowledge is limited and experimentation is restricted by ethical, financial, or time constraints, practitioners turn to observational causal discovery methods to recover the causal structure, exploiting the statistical properties of their data. Because causal discovery without further assumptions is an ill-posed problem, each algorithm comes with its own set of usually untestable assumptions, some of which are hard to meet in real datasets. Motivated by these considerations, this paper extensively benchmarks the empirical performance of recent causal discovery methods on observational _iid_ data generated under different background conditions, allowing for violations of the critical assumptions required by each selected approach. Our experimental findings show that score matching-based methods demonstrate surprising performance in the false positive and false negative rate of the inferred graph in these challenging scenarios, and we provide theoretical insights into their performance. This work is also the first effort to benchmark the stability of causal discovery algorithms with respect to the values of their hyperparameters. Finally, we hope this paper will set a new standard for the evaluation of causal discovery methods and can serve as an accessible entry point for practitioners interested in the field, highlighting the empirical implications of different algorithm choices.

Since X-ray scattering is a major cause of artifacts, its correction is a crucial step in almost any CT application. Most existing approaches, however, are based on complex theoretical models that need to be tailored to that particular application. To perform scatter estimation in absence of such models, we propose the unsupervised deep scatter estimation (uDSE). Here, uDSE combines a scatter estimation network that operates in projection domain with a scatter correction layer and CT reconstruction layer. In that way scatter estimation can be trained using an unsupervised Wassersten GAN (WGAN) setup in which the parameters of the scatter estimation network are optimized such that the resulting scatter corrected reconstructions cannot be distinguished from samples of a true artifact-free reference set. To demonstrate the feasibility of the proposed approach, uDSE is evaluated for simulated CBCT scans. Applied to the corresponding test data, uDSE is able to remove most of the present scatter artifacts and yields similar CT value accuracy (mean error of 27.9 HU vs. 24.7 HU) as a state-of-the-art supervised scatter estimation approach. Thus, uDSE may be used in the future to learn scatter estimation in cases where labels are not available or cannot be generated with sufficient accuracy.

Purpose With the rising number of computed tomography (CT) examinations and the trend toward personalized medicine, patient-specific dose estimates are becoming more and more important in CT imaging. However, current approaches are often too slow or too inaccurate to be applied routinely. Therefore, we propose the so-called deep dose estimation (DDE) to provide highly accurate patient dose distributions in real time Methods To combine accuracy and computational performance, the DDE algorithm uses a deep convolutional neural network to predict patient dose distributions. To do so, a U-net like architecture is trained to reproduce Monte Carlo simulations from a two-channel input consisting of a CT reconstruction and a first-order dose estimate. Here, the corresponding training data were generated using CT simulations based on 45 whole-body patient scans. For each patient, simulations were performed for different anatomies (pelvis, abdomen, thorax, head), different tube voltages (80 kV, 100 kV, 120 kV), different scan trajectories (circle, spiral), and with and without bowtie filtration and tube current modulation. Similar simulations were performed using a second set of eight whole-body CT scans from the Visual Concept Extraction Challenge in Radiology (Visceral) project to generate testing data. Finally, the DDE algorithm was evaluated with respect to the generalization to different scan parameters and the accuracy of organ dose and effective dose estimates based on an external organ segmentation. Results DDE dose distributions were quantified in terms of the mean absolute percentage error (MAPE) and a gamma analysis with respect to the ground truth Monte Carlo simulation. Both measures indicate that DDE generalizes well to different scan parameters and different anatomical regions with a maximum MAPE of 6.3% and a minimum gamma passing rate of 91%. Evaluating the organ dose values for all organs listed in the International Commission on Radiological Protection (ICRP) recommendation, shows an average error of 3.1% and maximum error of 7.2% (bone surface). Conclusions The DDE algorithm provides an efficient approach to determine highly accurate dose distributions. Being able to process a whole-body CT scan in about 1.5 s, it provides a valuable alternative to Monte Carlo simulations on a graphics processing unit (GPU). Here, the main advantage of DDE is that it can be used on top of any existing Monte Carlo code such that real-time performance can be achieved without major adjustments. Thus, DDE opens up new options not only for dosimetry but also for scan and protocol optimization.

Today, 2D+T fluoroscopy is usually used for image guidance in interventional radiology. For challenging procedures, 4D (3D+T) image guidance would be advantageous. The difficulty in realizing X-ray-based 4D interventional guidance lies in the development of a very dose efficient reconstruction algorithm. To this end, we improve on a previously presented algorithm for the reconstruction of interventional tools. By incorporating temporal information into a 3D convolutional neural network, we reduce the number of X-ray projections that need to be acquired for the 3D reconstruction of guidewires from four to two, thereby halving dose and decreasing the demands put on imaging devices implementing the algorithm. In experiments with two moving guidewires in an anthropomorphic phantom, we observe little deviation of our 3D reconstructions from the ground truth.

Long lasting efforts have been made to reduce radiation dose and thus the potential radiation risk to the patient for CT acquisitions without severe deterioration of image quality. To this end, different reconstruction and noise reduction algorithms have been developed, many of which are based on iterative reconstruction techniques, incorporating prior knowledge in the image domain. Recently, deep learning-based methods have shown impressive performance, outperforming many of the previously proposed CT denoising approaches both visually and quantitatively. However, with most neural networks being black boxes they remain notoriously difficult to interpret and concerns about the robustness and safety of such denoising methods have been raised. In this work we want to lay the fundamentals for a post-hoc interpretation of existing CT denoising networks by reconstructing their invariances.

Today, the subcutaneous, minimally invasive procedures performed in interventional radiology are usually guided by 2D X-ray fluoroscopy. In 2D X-ray fluoroscopy a series of 2D X-ray images is displayed. For challenging procedures however, 3D X-ray fluoroscopy would be advantageous. In 3D X-ray fluoroscopy, a series of 3D images, which is reconstructed from a series of 2D X-ray images, is displayed. Because the number of images used for guiding an intervention is very high, little dose can be spent per 3D reconstruction of a 3D fluoroscopy. To save dose and to minimize motion artifacts, a reconstruction algorithm that requires very few X-ray projections is desirable. Earlier work showed that guidewires, stents and coils, which are commonly used in interventions, can be reconstructed using only four synthetic X-ray projections. The reconstruction from two or three X-ray projections was only studied briefly. In this work, we improve the method by using a more suitable neural network architecture and by using a multi-channel backprojection instead of a single-channel backprojection. We then apply the improved method to more realistic data measured in an anthropomorphic phantom. The results show that the method produces 3D reconstructions of stents and guidewires with submillimeter accuracy using only three measured X-ray projections.

Purpose During a typical cardiac short scan, the heart can move several millimeters. As a result, the corresponding CT reconstructions may be corrupted by motion artifacts. Especially the assessment of small structures, such as the coronary arteries, is potentially impaired by the presence of these artifacts. In order to estimate and compensate for coronary artery motion, this manuscript proposes the deep partial angle‐based motion compensation (Deep PAMoCo). Methods The basic principle of the Deep PAMoCo relies on the concept of partial angle reconstructions (PARs), that is, it divides the short scan data into several consecutive angular segments and reconstructs them separately. Subsequently, the PARs are deformed according to a motion vector field (MVF) such that they represent the same motion state and summed up to obtain the final motion‐compensated reconstruction. However, in contrast to prior work that is based on the same principle, the Deep PAMoCo estimates and applies the MVF via a deep neural network to increase the computational performance as well as the quality of the motion compensated reconstructions. Results Using simulated data, it could be demonstrated that the Deep PAMoCo is able to remove almost all motion artifacts independent of the contrast, the radius and the motion amplitude of the coronary artery. In any case, the average error of the CT values along the coronary artery is about 25 HU while errors of up to 300 HU can be observed if no correction is applied. Similar results were obtained for clinical cardiac CT scans where the Deep PAMoCo clearly outperforms state‐of‐the‐art coronary artery motion compensation approaches in terms of processing time as well as accuracy. Conclusions The Deep PAMoCo provides an efficient approach to increase the diagnostic value of cardiac CT scans even if they are highly corrupted by motion.

Common deep neural networks (DNNs) for image classification have been shown to rely on shortcut opportunities (SO) in the form of predictive and easy-to-represent visual factors. This is known as shortcut learning and leads to impaired generalization. In this work, we show that common DNNs also suffer from shortcut learning when predicting only basic visual object factors of variation (FoV) such as shape, color, or texture. We argue that besides shortcut opportunities, generalization opportunities (GO) are also an inherent part of real-world vision data and arise from partial independence between predicted classes and FoVs. We also argue that it is necessary for DNNs to exploit GO to overcome shortcut learning. Our core contribution is to introduce the Diagnostic Vision Benchmark suite DiagViB-6, which includes datasets and metrics to study a network's shortcut vulnerability and generalization capability for six independent FoV. In particular, DiagViB-6 allows controlling the type and degree of SO and GO in a dataset. We benchmark a wide range of popular vision architectures and show that they can exploit GO only to a limited extent.

Image guidance for minimally invasive interventions is usually performed by acquiring fluoroscopic images using a C-arm system. However, the projective data provide only limited information about the spatial structure and position of interventional tools such as stents, guide wires or coils. In this work we propose a deep learning-based pipeline for real-time tomographic (four-dimensional) interventional guidance at acceptable dose levels. In the first step, interventional tools are extracted from four cone-beam CT projections using a deep convolutional neural network (CNN). These projections are then reconstructed and fed into a second CNN, which maps this highly undersampled reconstruction to a segmentation of the interventional tools. Our pipeline is capable of reconstructing interventional tools from only four x-ray projections without the need for a patient prior with very high accuracy. Therefore, the proposed approach is capable of overcoming the drawbacks of today's interventional guidance and could enable the development of new minimally invasive radiological interventions by providing full spatiotemporal information about the interventional tools.

Interventional guidance aims at providing the radiologist with detailed information about the location and orientation of interventional tools such as guide wires and stents. Most commonly, this is done by acquiring fluoroscopic images using an interventional C-arm system. Due to its projective nature, fluoroscopy is restricted to provide information from two spatial dimensions, preventing an exact 3D localization of the interventional tools. Analogous to computed tomography for diagnostic imaging, four-dimensional (three spatial dimensions plus the temporal dimension) interventional guidance has the potential to drastically improve both the speed and accuracy of such interventions, but is currently impractical due to the excessively high dose that would be necessary for continuous cone-beam CT (CBCT) scanning at high frame rates. In this work we develop a novel deep learning-based approach to reconstruct interventional tools from only four x-ray projections. We train and test this deep tool reconstruction (DTR) network on simulated data. Only small deviations from the ground truth (GT) reconstruction of the tools were observed, both quantitatively and qualitatively, showing that deep learning-based four-dimensional interventional guidance has the potential to overcome the drawbacks of conventional interventional guidance in the future.

Digital Subtraction Angiography (DSA) aims at selectively displaying vessels by subtracting an unenhanced mask image from a contrast-enhanced fluoroscopic image. This strategy requires the data to be static, i.e. to be acquired without patient or C-arm motion. Thus, conventional DSA cannot be applied to dynamic acquisition protocols such as bolus injection chases, which are particularly useful for the diagnosis of peripheral arterial disease (PAD). Preliminary studies have shown that convolutional neural networks (CNNs) are capable of overcoming this drawback, by predicting DSA-like images directly from their corresponding fluoroscopic x-ray images without the need for the acquisition of a mask image. Here, we demonstrate the potential of this approach for fluoroscopic acquisitions of the lower extremities. We apply the network to twelve different patient exams of which nine are without C-arm motion and the remaining three are bolus chase studies with C-arm motion. For cases where a conventional DSA is feasible we examine very small deviations and observe predictions for the bolus chase studies of similar visual impression as with conventional DSA. The results indicate that Deep DSA has the potential to improve the diagnosis of PAD by generating DSA-equivalent images from bolus chase studies of the lower extremities.

Due to the potential risk of ionizing radiation, the assessment of the administered radiation dose is an important topic in CT. However, dosimetric quantities that are routinely evaluated in CT only refer to the absorbed dose within cylindrical phantoms and do not appropriately represent the actual patient dose. While Monte Carlo (MC), the gold standard for patient-specific dose estimation, is too slow to be applied routinely, faster alternatives are usually far less accurate. To overcome this drawback, we developed the deep dose estimation (DDE) algorithm. DDE uses a deep convolutional neural network to reproduce MC dose estimates given only a CT image and a first-order dose estimate as two-channel input. To learn the corresponding mapping, DDE was trained on artificial data generated from whole-body clinical CT scans of 15 patients. For each patient 60 circular CT scans were simulated for 20 different z-positions (pelvis, abdomen, thorax) and 3 different acquisition settings. The total number of 900 data sets was divided into 720 training data sets (12 patients) and 180 validation data sets (3 patients). Each scan was reconstructed on a 256 × 256 × 48 voxel grid with an isotropic voxel size of 2 mm. In addition a first-order dose estimate as well as a MC dose estimate was calculated for every data set. Using these data, the network's open parameters were determined by minimizing the mean relative error between the output and the MC dose estimate. Evaluating the MRE on the validation data set yields deviations of 3.0 % on average with respect to the ground truth and processing times of about 250 ms per data set. Thus, DDE is able to achieve similar accuracy as MC while performing orders of magnitude faster..