Recent microscopy imaging techniques allow to precisely analyze cell morphology in 3D image data. To process the vast amount of image data generated by current digitized imaging techniques, automated approaches are demanded more than ever. Segmentation approaches used for morphological analyses, however, are often prone to produce unnaturally shaped predictions, which in conclusion could lead to inaccurate experimental outcomes. In order to minimize further manual interaction, shape priors help to constrain the predictions to the set of natural variations. In this paper, we show how spherical harmonics can be used as an alternative way to inherently constrain the predictions of neural networks for the segmentation of cells in 3D microscopy image data. Benefits and limitations of the spherical harmonic representation are analyzed and final results are compared to other state-of-the-art approaches on two different data sets.

Associations between age, resting-state (RS) peak-alpha-frequency (PAF = frequency showing largest amplitude alpha activity), and thalamic volume (thalamus thought to modulate alpha activity) were examined to understand differences in RS alpha activity between children with autism spectrum disorder (ASD) and typically-developing children (TDC) noted in prior studies. RS MEG and structural-MRI data were obtained from 51 ASD and 70 TDC 6- to 18-year-old males. PAF and thalamic volume maturation were observed in TDC but not ASD. Although PAF was associated with right thalamic volume in TDC (R2 = 0.12, p = 0.01) but not ASD (R2 = 0.01, p = 0.35), this group difference was not large enough to reach significance. Findings thus showed unusual maturation of brain function and structure in ASD as well as an across-group thalamic contribution to alpha rhythms.

The application of deep learning in the field of diffusion imaging is becoming increasingly popular. However, correlations of acquired adjacent gradient directions are often ignored. To make use of this information in a neural network, a spherical convolution is necessary. This work evaluates three different ways to include spherical information: 2D projection, local spherical convolution and Fourier space transform. For comparison, all models are designed to have a similar amount of trainable parameters as well as the same network architecture, and are evaluated by considering the example of signal augmentation. Overall, all models achieved comparable good results, improving the reconstruction performance, compared to a reconstruction without augmentation, by \(\approx \)30% for the fractional anisotropy, \(\approx \)50% for the mean diffusivity, \(\approx \)70% for the mean signal kurtosis and \(\approx \)5% for the diffusion signal itself. Particularly, in comparison to a regular neural network that does not implement a spherical convolution, the average performance for all models that implement a sperical convolution increases slightly for all evaluated measures, where the local spherical convolution shows the most favorable results.

Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations.

Cross-scanner and cross-protocol variability of diffusion magnetic resonance imaging (dMRI) data are known to be major obstacles in multi-site clinical studies since they limit the ability to aggregate dMRI data and derived measures. Computational algorithms that harmonize the data and minimize such variability are critical to reliably combine datasets acquired from different scanners and/or protocols, thus improving the statistical power and sensitivity of multi-site studies. Different computational approaches have been proposed to harmonize diffusion MRI data or remove scanner-specific differences. To date, these methods have mostly been developed for or evaluated on single b-value diffusion MRI data. In this work, we present the evaluation results of 19 algorithms that are developed to harmonize the cross-scanner and cross-protocol variability of multi-shell diffusion MRI using a benchmark database. The proposed algorithms rely on various signal representation approaches and computational tools, such as rotational invariant spherical harmonics, deep neural networks and hybrid biophysical and statistical approaches. The benchmark database consists of data acquired from the same subjects on two scanners with different maximum gradient strength (80 and 300 ​mT/m) and with two protocols. We evaluated the performance of these algorithms for mapping multi-shell diffusion MRI data across scanners and across protocols using several state-of-the-art imaging measures. The results show that data harmonization algorithms can reduce the cross-scanner and cross-protocol variabilities to a similar level as scan-rescan variability using the same scanner and protocol. In particular, the LinearRISH algorithm based on adaptive linear mapping of rotational invariant spherical harmonics features yields the lowest variability for our data in predicting the fractional anisotropy (FA), mean diffusivity (MD), mean kurtosis (MK) and the rotationally invariant spherical harmonic (RISH) features. But other algorithms, such as DIAMOND, SHResNet, DIQT, CMResNet show further improvement in harmonizing the return-to-origin probability (RTOP). The performance of different approaches provides useful guidelines on data harmonization in future multi-site studies.

In clinical settings, diffusion MRI can be used for extracting biomarkers such as fractional anisotropy or for revealing brain connectivity based on fiber tractography. Both are impacted by the free-water partial volume effect that arises at the border of cerebrospinal fluid or in presence of vasogenic edema. Hence, in order to robustly track white matter fibers close to cerebrospinal fluid and in presence of edema, or to extract consistent biomarkers in these cases, the diffusion MRI signal needs to be corrected for partial volume effects. We present a novel method that reproducibly infers plausible free-water volumes across different diffusion MRI acquisition schemes. Based on simulated data closely following the individual characteristics of each measurement, a neural network is trained on synthetic groundtruth data. According to our evaluation, this methodology produces more consistent and more plausible results than previous approaches.

This paper reviews the second challenge on spectral reconstruction from RGB images, i.e., the recovery of whole- scene hyperspectral (HS) information from a 3-channel RGB image. As in the previous challenge, two tracks were provided: (i) a "Clean" track where HS images are estimated from noise-free RGBs, the RGB images are themselves calculated numerically using the ground-truth HS images and supplied spectral sensitivity functions (ii) a "Real World" track, simulating capture by an uncalibrated and unknown camera, where the HS images are recovered from noisy JPEG-compressed RGB images. A new, larger-than-ever, natural hyperspectral image data set is presented, containing a total of 510 HS images. The Clean and Real World tracks had 103 and 78 registered participants respectively, with 14 teams competing in the final testing phase. A description of the proposed methods, alongside their challenge scores and an extensive evaluation of top performing methods is also provided. They gauge the state-of-the-art in spectral reconstruction from an RGB image.

We present a summary of competition results in the multi-shell diffusion MRIDiffusion MRI harmonisationHarmonization and enhancementEnhancement challenge (MUSHAC). MUSHAC is an open competition intended to stimulate the development of computational methods that reduce scanner- and protocol-related variabilities in multi-shell diffusion MRIDiffusion MRI data across multi-site studies. Twelve different methods from seven research groups have been tested in this challenge. The results show that cross-vendor harmonization and enhancementEnhancement can be performed by using suitable computational algorithms such as deep convolutional neural networks. Moreover, parametric models for multi-shell diffusion MRIDiffusion MRI signals also provide reliable performances.

Diffusion MRI is being used increasingly in studies of the brain and other parts of the body for its ability to provide quantitative measures that are sensitive to changes in tissue microstructure. However, inter-scanner and inter-protocol differences are known to induce significant measurement variability, which in turn jeopardises the ability to obtain 'truly quantitative measures' and challenges the reliable combination of different datasets. Combining datasets from different scanners and/or acquired at different time points could dramatically increase the statistical power of clinical studies, and facilitate multi-centre research. Even though careful harmonisation of acquisition parameters can reduce variability, inter-protocol differences become almost inevitable with improvements in hardware and sequence design over time, even within a site. In this work, we present a benchmark diffusion MRI database of the same subjects acquired on three distinct scanners with different maximum gradient strength (40, 80, and 300 mT/m), and with 'standard' and 'state-of-the-art' protocols, where the latter have higher spatial and angular resolution. The dataset serves as a useful testbed for method development in cross-scanner/cross-protocol diffusion MRI harmonisation and quality enhancement. Using the database, we compare the performance of five different methods for estimating mappings between the scanners and protocols. The results show that cross-scanner harmonisation of single-shell diffusion data sets can reduce the variability between scanners, and highlight the promises and shortcomings of today's data harmonisation techniques.

Age-related changes in resting-state (RS) neural rhythms in typically developing children (TDC) but not children with autism spectrum disorder (ASD) suggest that RS measures may be of clinical use in ASD only for certain ages. The study examined this issue via assessing RS peak alpha frequency (PAF), a measure previous studies, have indicated as abnormal in ASD. RS magnetoencephalographic (MEG) data were obtained from 141 TDC (6.13-17.70 years) and 204 ASD (6.07-17.93 years). A source model with 15 regional sources projected the raw MEG surface data into brain source space. PAF was identified in each participant from the source showing the largest amplitude alpha activity (7-13 Hz). Given sex differences in PAF in TDC (females > males) and relatively few females in both groups, group comparisons were conducted examining only male TDC (N = 121) and ASD (N = 183). Regressions showed significant group slope differences, with an age-related increase in PAF in TDC (R2  = 0.32) but not ASD (R2  = 0.01). Analyses examining male children below or above 10-years-old (median split) indicated group effects only in the younger TDC (8.90 Hz) and ASD (9.84 Hz; Cohen's d = 1.05). In the older ASD, a higher nonverbal IQ was associated with a higher PAF. In the younger TDC, a faster speed of processing was associated with a higher PAF. PAF as a marker for ASD depends on age, with a RS alpha marker of more interest in younger versus older children with ASD. Associations between PAF and cognitive ability were also found to be age and group specific.

Brain tumor localization and segmentation is an important step in the treatment of brain tumor patients. It is the base for later clinical steps, e.g., a possible resection of the tumor. Hence, an automatic segmentation algorithm would be preferable, as it does not suffer from inter-rater variability. On top, results could be available immediately after the brain imaging procedure. Using this automatic tumor segmentation, it could also be possible to predict the survival of patients. The BraTS 2018 challenge consists of these two tasks: tumor segmentation in 3D-MRI images of brain tumor patients and survival prediction based on these images. For the tumor segmentation, we utilize a two-step approach: First, the tumor is located using a 3D U-net. Second, another 3D U-net – more complex, but with a smaller output size – detects subtle differences in the tumor volume, i.e., it segments the located tumor into tumor core, enhanced tumor, and peritumoral edema. The survival prediction of the patients is done with a rather simple, yet accurate algorithm which outperformed other tested approaches on the train set when thoroughly cross-validated. This finding is consistent with our performance on the test set - we achieved 3rd place in the survival prediction task of the BraTS Challenge 2018.

Sodium Magnetic Resonance Imaging (sodium MRI) is an imaging modality that has gained momentum over the past decade, because of its potential ability to become a biomarker for several diseases, ranging from cancer to neurodegenerative pathologies, along with monitoring of tissues metabolism. One of the most important limitation to the exploitation of this imaging modality is its characteristic low resolution and signal-to-noise-ratio as compared to the classical proton MRI, which is due to the notably lower concentration of sodium than water in the human body. Therefore, denoising is a central aspect with respect to the clinical use of sodium MRI. In this work, we introduce a Convolutional Denoising Autoencoder that is trained on a training database of thirteen training subjects with three sodium MRI images each. The results illustrate that the denoised images show a strong improvement after application in comparison to the state-of-the-art Non Local Means denoising algorithm. This effect is demonstrated based on different noise metrics and a qualitative evaluation.

Diffusion imaging is an important method in the field of neuroscience, as it is sensitive to changes within the tissue microstructure of the human brain. However, a major challenge when using MRI to derive quantitative measures is that the use of different scanners, as used in multi-site group studies, introduces measurement variability. This can lead to an increased variance in quantitative metrics, even if the same brain is scanned. Contrary to the assumption that these characteristics are comparable and similar, small changes in these values are observed in many clinical studies, hence harmonization of the signals is essential. In this paper, we present a method that does not require additional preprocessing, such as segmentation or registration, and harmonizes the signal based on a deep learningresidual network. For this purpose, a training database is required, which consist of the same subjects, scanned on different scanners. The results show that harmonized signals are significantly more similar to the ground truth signal compared to no harmonization, but also improve in comparison to another deep learning method. The same effect is also demonstrated in commonly used metrics derived from the diffusion MRI signal.

DELIMIT is a framework extension for deep learning in diffusion imaging, which extends the basic framework PyTorch towards spherical signals. Based on several novel layers, deep learning can be applied to spherical diffusion imaging data in a very convenient way. First, two spherical harmonic interpolation layers are added to the extension, which allow to transform the signal from spherical surface space into the spherical harmonic space, and vice versa. In addition, a local spherical convolution layer is introduced that adds the possibility to include gradient neighborhood information within the network. Furthermore, these extensions can also be utilized for the preprocessing of diffusion signals.

Recovering high-dimensional spectral images taken with spectrally low-dimensional camera systems, in the extreme case RGB-images, has been of great interest for a variety of applications. An accurate spectral reconstruction is typically required to either achieve a better color accuracy or to improve object recognition/classification tasks. Almost all published work to date aims at performing a mapping from individual camera signals towards the corresponding spectrum. However, it might be beneficial to consider not only single pixels, but also contextual information. Here, we propose a convolutional neural network architecture that learns a mapping from RGB- to spectral images. We trained the network on the largest hyperspectral data set available to date [3] and analyzed the influence of different error metrics as loss functions. An objective evaluation of the performance in comparison to state of the art spectral reconstruction techniques is given by participating in the NTIRE 2018 challenge on spectral reconstruction [4].

We present a comparison of five different methods that estimate mappings between scanners for diffusion MRI data harmonisation. The methods are evaluated on a dedicated dataset of the same subjects acquired on three distinct scanners with ‘standard’ and ‘state-of-the-art’ protocols, with the latter having higher spatial and angular resolution. Our results show that cross-vendor harmonisation and spatial/angular resolution enhancement of single-shell diffusion data sets can be performed reliably, although some challenges remain. The dataset is available upon request and can serve as a useful testbed for future method development in cross-site/cross-hardware and cross-vendor diffusion MRI harmonisation.

This paper reviews the first challenge on spectral image reconstruction from RGB images, i.e., the recovery of whole-scene hyperspectral (HS) information from a 3channel RGB image. The challenge was divided into 2 tracks: the "Clean" track sought HS recovery from noiseless RGB images obtained from a known response function (representing spectrally-calibrated camera) while the "Real World" track challenged participants to recover HS cubes from JPEG-compressed RGB images generated by an unknown response function. To facilitate the challenge, the BGU Hyperspectral Image Database [4] was extended to provide participants with 256 natural HS training images, and 5+ 10 additional images for validation and testing, respectively. The "Clean" and "Real World" tracks had 73 and 63 registered participants respectively, with 12 teams competing in the final testing phase. Proposed methods and their corresponding results are reported in this review.

Gliomas are the most common primary brain malignancies, with different degrees of aggressiveness, variable prognosis and various heterogeneous histologic sub-regions, i.e., peritumoral edematous/invaded tissue, necrotic core, active and non-enhancing core. This intrinsic heterogeneity is also portrayed in their radio-phenotype, as their sub-regions are depicted by varying intensity profiles disseminated across multi-parametric magnetic resonance imaging (mpMRI) scans, reflecting varying biological properties. Their heterogeneous shape, extent, and location are some of the factors that make these tumors difficult to resect, and in some cases inoperable. The amount of resected tumor is a factor also considered in longitudinal scans, when evaluating the apparent tumor for potential diagnosis of progression. Furthermore, there is mounting evidence that accurate segmentation of the various tumor sub-regions can offer the basis for quantitative image analysis towards prediction of patient overall survival. This study assesses the state-of-the-art machine learning (ML) methods used for brain tumor image analysis in mpMRI scans, during the last seven instances of the International Brain Tumor Segmentation (BraTS) challenge, i.e., 2012-2018. Specifically, we focus on i) evaluating segmentations of the various glioma sub-regions in pre-operative mpMRI scans, ii) assessing potential tumor progression by virtue of longitudinal growth of tumor sub-regions, beyond use of the RECIST/RANO criteria, and iii) predicting the overall survival from pre-operative mpMRI scans of patients that underwent gross total resection. Finally, we investigate the challenge of identifying the best ML algorithms for each of these tasks, considering that apart from being diverse on each instance of the challenge, the multi-institutional mpMRI BraTS dataset has also been a continuously evolving/growing dataset.

Brain tumor localization and segmentation is an important step in the treatment of brain tumor patients. It is the base for later clinical steps, e.g., a possible resection of the tumor. Hence, an automatic segmentation algorithm would be preferable, as it does not suffer from inter-rater variability, and results could be available directly after the brain imaging procedure. Using this automatic tumor segmentation, it could also be possible to predict the survival of patients. The BraTS 2018 challenge consists of these two tasks: tumor segmentation in 3D-MRI images of brain tumor patients and survival prediction based on these images. For the tumor segmentation, we utilize a two-step approach: First, the tumor is located using a 3D U-net. Second, another 3D U-net – more complex, but with a smaller field-of-view – detects subtle differences in the tumor volume, i.e., it segments the located tumor into tumor core, enhanced tumor, and peritumoral edema. The survival prediction of the patients is done with a rather simple, yet accurate algorithm which outperforms other tested approaches.