Coherent X-ray microscopy is emerging as a transformative technology for neuronal imaging, with the potential to offer a scalable solution for reconstruction of neural circuits in millimeter sized tissue volumes. Specifically, X-ray holographic nanotomography (XNH) brings together outstanding capabilities in terms of contrast, spatial resolution and data acquisition speed. While recent XNH developments already enabled generating valuable datasets for neurosciences, a major challenge for reconstruction of neural circuits remained overcoming resolving power limits to distinguish smaller neurites and synapses in the reconstructed volumes. Here we present a self-supervised image restoration approach that improves simultaneously spatial resolution, contrast, and data acquisition speed. This enables revealing synapses with XNH, marking a major milestone in the quest for generating connectomes of full mammalian brains. We demonstrate that this method is effective for various types of neuronal tissues and acquisition schemes. We propose a scalable implementation compatible with multi-terabyte image volumes. Altogether, this work brings large scale X-ray nanotomography to a new precision level. ### Competing Interest Statement The authors have declared no competing interest.

BackgroundDeep learning-based unsupervised image registration has recently been proposed, promising fast registration. However, it has yet to be adopted in the online adaptive magnetic resonance imaging-guided radiotherapy (MRgRT) workflow.PurposeIn this paper, we design an unsupervised, joint rigid, and deformable registration framework for contour propagation in MRgRT of prostate cancer.MethodsThree-dimensional pelvic T2-weighted MRIs of 143 prostate cancer patients undergoing radiotherapy were collected and divided into 110, 13, and 20 patients for training, validation, and testing. We designed a framework using convolutional neural networks (CNNs) for rigid and deformable registration. We selected the deformable registration network architecture among U-Net, MS-D Net, and LapIRN and optimized the training strategy (end-to-end vs. sequential). The framework was compared against an iterative baseline registration. We evaluated registration accuracy (the Dice and Hausdorff distance of the prostate and bladder contours), structural similarity index, and folding percentage to compare the methods. We also evaluated the framework's robustness to rigid and elastic deformations and bias field perturbations.ResultsThe end-to-end trained framework comprising LapIRN for the deformable component achieved the best median (interquartile range) prostate and bladder Dice of 0.89 (0.85-0.91) and 0.86 (0.80-0.91), respectively. This accuracy was comparable to the iterative baseline registration: prostate and bladder Dice of 0.91 (0.88-0.93) and 0.86 (0.80-0.92). The best models complete rigid and deformable registration in 0.002 (0.0005) and 0.74 (0.43) s (Nvidia Tesla V100-PCIe 32 GB GPU), respectively. We found that the models are robust to translations up to 52 mm, rotations up to 15 circle$<^>\circ$, elastic deformations up to 40 mm, and bias fields.ConclusionsOur proposed unsupervised, deep learning-based registration framework can perform rigid and deformable registration in less than a second with contour propagation accuracy comparable with iterative registration.

Many of the recent successes of deep learning-based approaches have been enabled by a framework of flexible, composable computational blocks with their parameters adjusted through an automatic differentiation mechanism to implement various data processing tasks. In this work, we explore how the same philosophy can be applied to existing “classical” (i.e., non-learning) algorithms, focusing on computed tomography (CT) as application field. We apply four key design principles of this approach for CT workflow design: end-to-end optimization, explicit quality criteria, declarative algorithm construction by building the forward model, and use of existing classical algorithms as computational blocks. Through four case studies, we demonstrate that auto-differentiation is remarkably effective beyond the boundaries of neural-network training, extending to CT workflows containing varied combinations of classical and machine learning algorithms.

Tomographic algorithms are often compared by evaluating them on certain benchmark datasets. For fair comparison, these datasets should ideally (i) be challenging to reconstruct, (ii) be representative of typical tomographic experiments, (iii) be flexible to allow for different acquisition modes, and (iv) include enough samples to allow for comparison of data-driven algorithms. Current approaches often satisfy only some of these requirements, but not all. For example, real-world datasets are typically challenging and representative of a category of experimental examples, but are restricted to the acquisition mode that was used in the experiment and are often limited in the number of samples. Mathematical phantoms are often flexible and can sometimes produce enough samples for data-driven approaches, but can be relatively easy to reconstruct and are often not representative of typical scanned objects. In this paper, we present a family of foam-like mathematical phantoms that aims to satisfy all four requirements simultaneously. The phantoms consist of foam-like structures with more than 100000 features, making them challenging to reconstruct and representative of common tomography samples. Because the phantoms are computer-generated, varying acquisition modes and experimental conditions can be simulated. An effectively unlimited number of random variations of the phantoms can be generated, making them suitable for data-driven approaches. We give a formal mathematical definition of the foam-like phantoms, and explain how they can be generated and used in virtual tomographic experiments in a computationally efficient way. In addition, several 4D extensions of the 3D phantoms are given, enabling comparisons of algorithms for dynamic tomography. Finally, example phantoms and tomographic datasets are given, showing that the phantoms can be effectively used to make fair and informative comparisons between tomography algorithms.

Neural network pruning techniques can substantially reduce the computational cost of applying convolutional neural networks (CNNs). Common pruning methods determine which convolutional filters to remove by ranking the filters individually, i.e., without taking into account their interdependence. In this paper, we advocate the viewpoint that pruning should consider the interdependence between series of consecutive operators. We propose the LongEst-chAiN (LEAN) method that prunes CNNs by using graph-based algorithms to select relevant chains of convolutions. A CNN is interpreted as a graph, with the operator norm of each operator as distance metric for the edges. LEAN pruning iteratively extracts the highest value path from the graph to keep. In our experiments, we test LEAN pruning on several image-to-image tasks, including the well-known CamVid dataset, and a real-world X-ray CT dataset. Results indicate that LEAN pruning can result in networks with similar accuracy, while using 1.7-12x fewer convolutional filters than existing approaches.

It is often claimed that Bayesian methods, in particular Bayes factor methods for hypothesis testing, can deal with optional stopping. We first give an overview, using elementary probability theory, of three different mathematical meanings that various authors give to this claim: (1) stopping rule independence, (2) posterior calibration and (3) (semi-) frequentist robustness to optional stopping. We then prove theorems to the effect that these claims do indeed hold in a general measure-theoretic setting. For claims of type (2) and (3), such results are new. By allowing for non-integrable measures based on improper priors, we obtain particularly strong results for the practically important case of models with nuisance parameters satisfying a group invariance (such as location or scale). We also discuss the practical relevance of (1)--(3), and conclude that whether Bayes factor methods actually perform well under optional stopping crucially depends on details of models, priors and the goal of the analysis.

At X-ray beamlines of synchrotron light sources, the achievable time-resolution for 3D tomographic imaging of the interior of an object has been reduced to a fraction of a second, enabling rapidly changing structures to be examined. The associated data acquisition rates require sizable computational resources for reconstruction. Therefore, full 3D reconstruction of the object is usually performed after the scan has completed. Quasi-3D reconstruction -- where several interactive 2D slices are computed instead of a 3D volume -- has been shown to be significantly more efficient, and can enable the real-time reconstruction and visualization of the interior. However, quasi-3D reconstruction relies on filtered backprojection type algorithms, which are typically sensitive to measurement noise. To overcome this issue, we propose Noise2Filter, a learned filter method that can be trained using only the measured data, and does not require any additional training data. This method combines quasi-3D reconstruction, learned filters, and self-supervised learning to derive a tomographic reconstruction method that can be trained in under a minute and evaluated in real-time. We show limited loss of accuracy compared to training with additional training data, and improved accuracy compared to standard filter-based methods.

Synchrotron X-ray tomography enables the examination of the internal structure of materials at submicron spatial resolution and subsecond temporal resolution. Unavoidable experimental constraints can impose dose and time limits on the measurements, introducing noise in the reconstructed images. Convolutional neural networks (CNNs) have emerged as a powerful tool to remove noise from reconstructed images. However, their training typically requires collecting a dataset of paired noisy and high-quality measurements, which is a major obstacle to their use in practice. To circumvent this problem, methods for CNN-based denoising have recently been proposed that require no separate training data beyond the already available noisy reconstructions. Among these, the Noise2Inverse method is specifically designed for tomography and related inverse problems. To date, applications of Noise2Inverse have only taken into account 2D spatial information. In this paper, we expand the application of Noise2Inverse in space, time, and spectrum-like domains. This development enhances applications to static and dynamic micro-tomography as well as X-ray diffraction tomography. Results on real-world datasets establish that Noise2Inverse is capable of accurate denoising and enables a substantial reduction in acquisition time while maintaining image quality.

Time-resolved X-ray tomographic microscopy is an invaluable technique to investigate dynamic processes in 3D for extended time periods. Because of the limited signal-to-noise ratio caused by the short exposure times and sparse angular sampling frequency, obtaining quantitative information through post-processing remains challenging and requires intensive manual labor. This severely limits the accessible experimental parameter space and so, prevents fully exploiting the capabilities of the dedicated time-resolved X-ray tomographic stations. Though automatic approaches, often exploiting iterative reconstruction methods, are currently being developed, the required computational costs typically remain high. Here, we propose a highly efficient reconstruction and classification pipeline (SIRT-FBP-MS-D-DIFF) that combines an algebraic filter approximation and machine learning to significantly reduce the computational time. The dynamic features are reconstructed by standard filtered back-projection with an algebraic filter to approximate iterative reconstruction quality in a computationally efficient manner. The raw reconstructions are post-processed with a trained convolutional neural network to extract the dynamic features from the low signal-to-noise ratio reconstructions in a fully automatic manner. The capabilities of the proposed pipeline are demonstrated on three different dynamic fuel cell datasets, one exploited for training and two for testing without network retraining. The proposed approach enables automatic processing of several hundreds of datasets in a single day on a single GPU node readily available at most institutions, so extending the possibilities in future dynamic X-ray tomographic investigations.

Tomography is a powerful tool for reconstructing the interior of an object from a series of projection images. Typically, the source and detector traverse a standard path (e.g., circular, helical). Recently, various techniques have emerged that use more complex acquisition geometries. Current software packages require significant handwork, or lack the flexibility to handle such geometries. Therefore, software is needed that can concisely represent, visualize, and compute reconstructions of complex acquisition geometries. We present tomosipo, a Python package that provides these capabilities in a concise and intuitive way. Case studies demonstrate the power and flexibility of tomosipo.

Recovering a high-quality image from noisy indirect measurements is an important problem with many applications. For such inverse problems, supervised deep convolutional neural network (CNN)-based denoising methods have shown strong results, but the success of these supervised methods critically depends on the availability of a high-quality training dataset of similar measurements. For image denoising, methods are available that enable training without a separate training dataset by assuming that the noise in two different pixels is uncorrelated. However, this assumption does not hold for inverse problems, resulting in artifacts in the denoised images produced by existing methods. Here, we propose Noise2Inverse, a deep CNN-based denoising method for linear image reconstruction algorithms that does not require any additional clean or noisy data. Training a CNN-based denoiser is enabled by exploiting the noise model to compute multiple statistically independent reconstructions. We develop a theoretical framework which shows that such training indeed obtains a denoising CNN, assuming the measured noise is element-wise independent, and zero-mean. On simulated CT datasets, Noise2Inverse demonstrates an improvement in peak signal-to-noise ratio and structural similarity index compared to state-of-the-art image denoising methods, and conventional reconstruction methods, such as Total-Variation Minimization. We also demonstrate that the method is able to significantly reduce noise in challenging real-world experimental datasets.

Computer Tomography (CT) scanners for research applications are often designed to facilitate flexible acquisition geometries. Making full use of such CT scanners requires advanced reconstruction software that can (i) deal with a broad range of geometrical scanning settings, (ii) allows for customization of processing algorithms, and (iii) has the capability to process large amounts of data. FleXbox is a Python-based tomographic reconstruction toolbox focused on these three functionalities. It is built to bridge the gap between low-level tomographic reconstruction packages (e.g. ASTRA toolbox) and high-level distributed systems (e.g. Livermore Tomography Tools). FleXbox allows to model arbitrary source, detector and object trajectories. The modular architecture of FleXbox allows to design an optimal reconstruction approach for a single CT dataset. When multiple datasets of an object are acquired (either different spatial regions or different snapshots in time), they can be combined into a larger high resolution volume or a time series of volumes. The software allows to then create a computational reconstruction pipeline that can run without user interaction and enables efficient computation on large-scale 3D volumes on a single workstation.

Computer Tomography (CT) scanners for research applications are often designed to facilitate flexible acquisition geometries. Making full use of such CT scanners requires advanced reconstruction software that can (i) deal with a broad range of geometrical scanning settings, (ii) allows for customization of processing algorithms, and (iii) has the capability to process large amounts of data. FleXbox is a Python-based tomographic reconstruction toolbox focused on these three functionalities. It is built to bridge the gap between low-level tomographic reconstruction packages (e.g. ASTRA toolbox) and highlevel distributed systems (e.g. Livermore Tomography Tools). FleXbox allows to model arbitrary source, detector and object trajectories. The modular architecture of FleXbox allows to design an optimal reconstruction approach for a single CT dataset. When multiple datasets of an object are acquired (either different spatial regions or different snapshots in time), they can be combined into a larger high resolution volume or a time series of volumes. The software allows to then create a computational reconstruction pipeline that can run without user interaction and enables efficient computation on large-scale 3D volumes on a single workstation. © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).

A major challenge in attaining customized additive manufactured (AM) skull implants is segmentation of 3D scans. Therefore, this study aimed to develop a deep learning algorithm (MSDnet) to automatically segment skull defects in computed tomography (CT) scans. The MSDnet was trained with CT scans and corresponding virtual 3D models of patients who had undergone cranioplasty using AM skull implants. The trained MSDnet was able to segment unseen CT scans accurately and quickly. Deep learning can thus remove the barriers of time and effort during CT image segmentation, thereby making customized AM skull implants more affordable and more accessible to clinicians.

In tomography, the resolution of the reconstructed 3D volume is inherently limited by the pixel resolution of the detector and optical phenomena. Machine learning has demonstrated powerful capabilities for super-resolution in several imaging applications. Such methods typically rely on the availability of high-quality training data for a series of similar objects. In many applications of tomography, existing machine learning methods cannot be used because scanning such a series of similar objects is either impossible or infeasible. In this paper, we propose a novel technique for improving the resolution of tomographic volumes that is based on the assumption that the local structure is similar throughout the object. Therefore, our approach does not require a training set of similar objects. The technique combines a specially designed scanning procedure with a machine learning method for super-resolution imaging. We demonstrate the effectiveness of our approach using both simulated and experimental data. The results show that the proposed method is able to significantly improve resolution of tomographic reconstructions.

Purpose In order to attain anatomical models, surgical guides and implants for computer-assisted surgery, accurate segmentation of bony structures in cone-beam computed tomography (CBCT) scans is required. However, this image segmentation step is often impeded by metal artifacts. Therefore, this study aimed to develop a mixed-scale dense convolutional neural network (MS-D network) for bone segmentation in CBCT scans affected by metal artifacts. Method Training data were acquired from 20 dental CBCT scans affected by metal artifacts. An experienced medical engineer segmented the bony structures in all CBCT scans using global thresholding and manually removed all remaining noise and metal artifacts. The resulting gold standard segmentations were used to train an MS-D network comprising 100 convolutional layers using far fewer trainable parameters than alternative convolutional neural network (CNN) architectures. The bone segmentation performance of the MS-D network was evaluated using a leave-2-out scheme and compared with a clinical snake evolution algorithm and two state-of-the-art CNN architectures (U-Net and ResNet). All segmented CBCT scans were subsequently converted into standard tessellation language (STL) models and geometrically compared with the gold standard. Results CBCT scans segmented using the MS-D network, U-Net, ResNet and the snake evolution algorithm demonstrated mean Dice similarity coefficients of 0.87 +/- 0.06, 0.87 +/- 0.07, 0.86 +/- 0.05, and 0.78 +/- 0.07, respectively. The STL models acquired using the MS-D network, U-Net, ResNet and the snake evolution algorithm demonstrated mean absolute deviations of 0.44 mm +/- 0.13 mm, 0.43 mm +/- 0.16 mm, 0.40 mm +/- 0.12 mm and 0.57 mm +/- 0.22 mm, respectively. In contrast to the MS-D network, the ResNet introduced wave-like artifacts in the STL models, whereas the U-Net incorrectly labeled background voxels as bone around the vertebrae in 4 of the 9 CBCT scans containing vertebrae. Conclusion The MS-D network was able to accurately segment bony structures in CBCT scans affected by metal artifacts.