To better understand early brain development in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; h owever, one of the major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological development in the community, the iSeg-2019 challenge (http://iseg2019.web.unc.edu) provides a set of 6-month infant subjects from multiple sites with different protocols/scanners for the participating methods. T raining/validation subjects are from UNC (MAP) and testing subjects are from UNC/UMN (BCP), Stanford University, and Emory University. By the time of writing, there are 30 automatic segmentation methods participated in the iSeg-2019. In this article, 8 top-ranked methods were reviewed by detailing their pipelines/implementations, presenting experimental results, and evaluating performance across different sites in terms of whole brain, regions of interest, and gyral landmark curves. We further pointed out their limitations and possible directions for addressing the multi-site issue. We find that multi-site consistency is still an open issue. We hope that the multi-site dataset in the iSeg-2019 and this review article will attract more researchers to address the challenging and critical multi-site issue in practice.

This work aims to assist physicians improve their speed and diagnostic accuracy when interpreting portable CXR (p_CXR), which are in especially high demand in the setting of the ongoing COVID-19 pandemic. In this paper, we introduce new deep learning frameworks, named Pair-Flow, to align and enhance the quality of p_CXR to be more consistent, and to more closely match higher quality conventional CXR (c_CXR). The contributions of this work are four folds. Firstly, a new database collection of subject-pair CXR is introduced and available to download. Secondly, a new deep learning-based alignment approach is presented to align subject-pairs dataset to obtain pixel-pairs dataset. Thirdly, a new Pair-Flow approach, an end-to-end invertible transfer deep learning method, to enhance the degraded quality of p_CXR. Finally, the performance of the proposed system is evaluated at both image quality and topological properties.

Medical image segmentation is one of the most challenging tasks in medical image analysis and widely developed for many clinical applications. While deep learning-based approaches have achieved impressive performance in semantic segmentation, they are limited to pixel-wise settings with imbalanced-class data problems and weak boundary object segmentation in medical images. In this paper, we tackle those limitations by developing a new two-branch deep network architecture which takes both higher level features and lower level features into account. The first branch extracts higher level feature as region information by a common encoder-decoder network structure such as Unet and FCN, whereas the second branch focuses on lower level features as support information around the boundary and processes in parallel to the first branch. Our key contribution is the second branch named Narrow Band Active Contour (NB-AC) attention model which treats the object contour as a hyperplane and all data inside a narrow band as support information that influences the position and orientation of the hyperplane. Our proposed NB-AC attention model incorporates the contour length with the region energy involving a fixed-width band around the curve or surface. The proposed network loss contains two fitting terms: (i) a high level feature (i.e., region) fitting term from the first branch; (ii) a lower level feature (i.e., contour) fitting term from the second branch including the (ii1) length of the object contour and (ii2) regional energy functional formed by the homogeneity criterion of both the inner band and outer band neighboring the evolving curve or surface. The proposed NB-AC loss can be incorporated into both 2D and 3D deep network architectures. The proposed network has been evaluated on different challenging medical image datasets, including DRIVE, iSeg17, MRBrainS18 and Brats18. The experimental results have shown that the proposed NB-AC loss outperforms other mainstream loss functions: Cross Entropy, Dice, Focal on two common segmentation frameworks Unet and FCN. Our 3D network which is built upon the proposed NB-AC loss and 3DUnet framework achieved state-of-the-art results on multiple volumetric datasets.

The research in automatic unsupervised visual clustering has received considerable attention over the last couple years. It aims at explaining distributions of unlabeled visual images by clustering them via a parameterized model of appearance. Graph Convolutional Neural Networks (GCN) have recently been one of the most popular clustering methods. However, it has reached some limitations. Firstly, it is quite sensitive to hard or noisy samples. Secondly, it is hard to investigate with various deep network models due to its computational training time. Finally, it is hard to design an end-to-end training model between the deep feature extraction and GCN clustering modeling. This work therefore presents the Clusformer, a simple but new perspective of Transformer based approach, to automatic visual clustering via its unsupervised attention mechanism. The proposed method is able to robustly deal with noisy or hard samples. It is also flexible and effective to collaborate with different deep network models with various model sizes in an end-to-end framework. The proposed method is evaluated on two popular large-scale visual databases, i.e. Google Landmark and MS-Celeb1M face database, and outperforms prior unsupervised clustering methods. Code will be available at https://github.com/VinAIResearch/Clusformer

This work aimed to assist physicians by improving their speed and diagnostic accuracy when interpreting portable CXRs as well as monitoring the treatment process to see whether a patient is improving or deteriorating with treatment. These objectives are in especially high demand in the setting of the ongoing COVID-19 pandemic. With the recent progress in the development of artificial intelligence (AI), we introduce new deep learning frameworks to align and enhance the quality of portable CXRs to be more consistent, and to more closely match higher quality conventional CXRs. These enhanced portable CXRs can then help the doctors provide faster and more accurate diagnosis and treatment planning. The contributions of this work are four-fold. Firstly, a new database collection of subject-pair radiographs is introduced. For each subject, we collected a pair of samples from both portable and conventional machines. Secondly, a new deep learning approach is presented to align the subject-pairs dataset to obtain a pixel-pairs dataset. Thirdly, a new PairFlow approach is presented, an end-to-end invertible transfer deep learning method, to enhance the degraded quality of portable CXRs. Finally, the performance of the proposed system is evaluated by UAMS doctors in terms of both image quality and topological properties. This work was undertaken in collaboration with the Department of Radiology at the University of Arkansas for Medical Sciences (UAMS) to enhance portable/mobile COVID-19 CXRs, to improve the speed and accuracy of portable CXR images and aid in urgent COVID-19 diagnosis, monitoring and treatment.

Accurate segmentation of the brain into gray matter, white matter, and cerebrospinal fluid using magnetic resonance (MR) imaging is critical for visualization and quantification of brain anatomy. Compared to 3T MR images, 7T MR images exhibit higher tissue contrast that is contributive to accurate tissue delineation for training segmentation models. In this paper, we propose a cascaded nested network (CaNes-Net) for segmentation of 3T brain MR images, trained by tissue labels delineated from the corresponding 7T images. We first train a nested network (Nes-Net) for a rough segmentation. The second Nes-Net uses tissue-specific geodesic distance maps as contextual information to refine the segmentation. This process is iterated to build CaNes-Net with a cascade of Nes-Net modules to gradually refine the segmentation. To alleviate the misalignment between 3T and corresponding 7T MR images, we incorporate a correlation coefficient map to allow well-aligned voxels to play a more important role in supervising the training process. We compared CaNes-Net with SPM and FSL tools, as well as four deep learning models on 18 adult subjects and the ADNI dataset. Our results indicate that CaNes-Net reduces segmentation errors caused by the misalignment and improves segmentation accuracy substantially over the competing methods.

Medical image segmentation has played an important role in medical analysis and widely developed for many clinical applications. Deep learning-based approaches have achieved high performance in semantic segmentation but they are limited to pixel-wise setting and imbalanced classes data problem. In this paper, we tackle those limitations by developing a new deep learning-based model which takes into account both higher feature level i.e. region inside contour, intermediate feature level i.e. offset curves around the contour and lower feature level i.e. contour. Our proposed Offset Curves (OsC) loss consists of three main fitting terms. The first fitting term focuses on pixel-wise level segmentation whereas the second fitting term acts as attention model which pays attention to the area around the boundaries (offset curves). The third terms plays a role as regularization term which takes the length of boundaries into account. We evaluate our proposed OsC loss on both 2D network and 3D network. Two common medical datasets, i.e. retina DRIVE and brain tumor BRATS 2018 datasets are used to benchmark our proposed loss performance. The experiments have shown that our proposed OsC loss function outperforms other mainstream loss functions such as Cross-Entropy, Dice, Focal on the most common segmentation networks Unet, FCN.

Accurate segmentation of the brain into major tissue types, e.g., the gray matter, white matter, and cerebrospinal fluid, in magnetic resonance (MR) imaging is critical for quantification of the brain anatomy and function. The availability of 7T MR scanners can provide more accurate and reliable voxel-wise tissue labels, which can be leveraged to supervise the training of the tissue segmentation in the conventional 3T brain images. Specifically, a deep learning based method can be used to build the highly non-linear mapping from the 3T intensity image to the more reliable label maps obtained from the 7T images of the same subject. However, the misalignment between 3T and 7T MR images due to image distortions poses a major obstacle to achieving better segmentation accuracy. To address this issue, we measure the quality of the 3T-7T alignment by using a correlation coefficient map. Then we propose a cascaded nested network (CaNes-Net) for 3T MR image segmentation and a multi-stage solution for training this model with the ground-truth tissue labels from 7T images. This paper has two main contributions. First, by incorporating the correlation loss, the above mentioned obstacle can be well addressed. Second, the geodesic distance maps are constructed based on the intermediate segmentation results to guide the training of the CaNes-Net as an iterative coarse-to-fine process. We evaluated the proposed CaNes-Net with the state-of-the-art methods on 18 in-house acquired subjects. We also qualitatively assessed the performance of the proposed model and U-Net on the ADNI dataset. Our results indicate that the proposed CaNes-Net is able to dramatically reduce mis-segmentation caused by the misalignment and achieves substantially improved accuracy over all the other methods.

Image synthesis from corrupted contrasts increases the diversity of diagnostic information available for many neurological diseases. Recently the image-to-image translation has experienced significant levels of interest within medical research, beginning with the successful use of the Generative Adversarial Network (GAN) to the introduction of cyclic constraint extended to multiple domains. However, in current approaches, there is no guarantee that the mapping between the two image domains would be unique or one-to-one. In this paper, we introduce a novel approach to unpaired image-to-image translation based on the invertible architecture. The invertible property of the flow-based architecture assures a cycle-consistency of image-to-image translation without additional loss functions. We utilize the temporal information between consecutive slices to provide more constraints to the optimization for transforming one domain to another in unpaired volumetric medical images. To capture temporal structures in the medical images, we explore the displacement between the consecutive slices using a deformation field. In our approach, the deformation field is used as a guidance to keep the translated slides realistic and consistent across the translation. The experimental results have shown that the synthesized images using our proposed approach are able to archive a competitive performance in terms of mean squared error, peak signal-to-noise ratio, and structural similarity index when compared with the existing deep learning-based methods on three standard datasets, i.e. HCP, MRBrainS13 and Brats2019.

Accurate segmentation of infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF) is an indispensable foundation for early studying of brain growth patterns and morphological changes in neurodevelopmental disorders. Nevertheless, in the isointense phase (approximately 6-9 months of age), due to inherent myelination and maturation process, WM and GM exhibit similar levels of intensity in both T1-weighted (T1w) and T2-weighted (T2w) MR images, making tissue segmentation very challenging. Despite many efforts were devoted to brain segmentation, only few studies have focused on the segmentation of 6-month infant brain images. With the idea of boosting methodological development in the community, iSeg-2017 challenge (http://iseg2017.web.unc.edu) provides a set of 6-month infant subjects with manual labels for training and testing the participating methods. Among the 21 automatic segmentation methods participating in iSeg-2017, we review the 8 top-ranked teams, in terms of Dice ratio, modified Hausdorff distance and average surface distance, and introduce their pipelines, implementations, as well as source codes. We further discuss limitations and possible future directions. We hope the dataset in iSeg-2017 and this review article could provide insights into methodological development for the community.

The methods using attention module have been studied recently on image processing using Convolution Neural Network (CNN). The main purpose of CNN, where the method is applied, is to emphasize important features and weaken less important ones. From this perspective, we propose Compression and Intensity modules in order to boost the representation of feature map, by focusing on pixel-wise spatial attention. For each pixel, the importance of the spatial information which the feature possesses is identified and enhanced, so that an efficient segmentation task can be performed. The performance of the proposed module with state-of-the-art CNN models outperformed other recent attention modules for the brain MRI segmentation evaluation on MRBrainS18.

Fetal brain extraction is one of the most essential steps for prenatal brain MRI reconstruction and analysis. However, due to the fetal movement within the womb, it is a challenging task to extract fetal brains from sparsely-acquired imaging stacks typically with motion artifacts. To address this problem, we propose an automatic brain extraction method for fetal magnetic resonance imaging (MRI) using multi-stage 2D U-Net with deep supervision (DS U-net). Specifically, we initially employ a coarse segmentation derived from DS U-net to define a 3D bounding box for localizing the position of the brain. The DS U-net is trained with deep supervision loss to acquire more powerful discrimination capability. Then, another DS U-net focuses on the extracted region to produce finer segmentation. The final segmentation results are obtained by performing refined segmentation. We validate the proposed method on 80 stacks of training images and 43 testing stacks. The experimental results demonstrate the precision and robustness of our method with the average Dice coefficient of 91.69%, outperforming the existing methods.

Accurate segmentation of neonatal brain MR images is critical for studying early brain development. Recently, supervised learning-based methods, i.e., using convolutional neural networks (CNNs), have been successfully applied to infant brain segmentation. Although these CNN-based methods have achieved reasonable segmentation results on the testing subjects acquired with similar imaging protocol as the training subjects, they are typically not able to produce reasonable results for the testing subjects acquired with different imaging protocols. To address this practical issue, in this paper, we propose leveraging a cycle-consistent generative adversarial network (CycleGAN) to transfer each testing image (of a new dataset/cross-dataset) into the domain of training data, thus obtaining the transferred testing image with similar intensity appearance as the training images. Then, a densely-connected U-Net based segmentation model, which has been trained on the training data, can be utilized to robustly segment each transferred testing image. Experimental results demonstrate the superior performance of our proposed method, over existing methods, on segmenting cross-dataset of neonatal brain MR images.

In this paper, we propose a novel end-to-end adversarial training on volumetric brain segmentation architecture that allows to enforce long-range spatial label contiguity and label consistency. The proposed network consists of two networks: generator and discriminator. The generator network allows to take volumetric image as input and provides a volumetric probability map for each tissue. Then, the discriminator network learns to differentiate ground-truth maps from the probability maps of generator network. We design a discriminator in a fully convolutional manner to differentiate the predicted probability maps from the ground-truth segmentation distribution with the consideration of the spatial information on voxel level, which makes it difficult to learn the discriminator. In order to overcome it, the proposed discriminator provides a 3D confidence map which indicates corresponding regions of the probability maps close to the ground-truth. Based on the 3D confidence map information, the generator network will refine prediction output close to the ground-truth maps in a high-order structure.

Accurate segmentation of infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid is an indispensable foundation for early studying of brain growth patterns and morphological changes in neurodevelopmental disorders. Nevertheless, in the isointense phase (approximately 6-9months of age), due to inherent myelination and maturation process, WM and GM exhibit similar levels of intensity in both T1-weighted and T2-weighted MR images, making tissue segmentation very challenging. Although many efforts were devoted to brain segmentation, only a few studies have focused on the segmentation of six-month infant brain images. With the idea of boosting methodological development in the community, iSeg-2017 challenge (http://iseg2017.web.unc.edu) provides a set of six-month infant subjects with manual labels for training and testing the participating methods. Among the 21 automatic segmentation methods participating in iSeg-2017, we review the eight top-ranked teams, in terms of Dice ratio, modified Hausdorff distance, and average surface distance, and introduce their pipelines, implementations, as well as source codes. We further discuss the limitations and possible future directions. We hope the dataset in iSeg-2017, and this paper could provide insights into methodological development for the community.

Bone age assessment plays an important role in the endocrinology and genetic investigation of patients. In this paper, we proposed a deep learning-based approach for bone age assessment by integration of the Tanner-Whitehouse (TW3) methods and deep convolution networks based on extracted regions of interest (ROI)-detection and classification using Faster-RCNN and Inception-v4 networks, respectively. The proposed method allows exploration of expert knowledge from TW3 and features engineering from deep convolution networks to enhance the accuracy of bone age assessment. The experimental results showed a mean absolute error of about 0.59 years between expert radiologists and the proposed method, which is the best performance among state-of-the-art methods.

Automatic 6-month infant brain tissue segmentation of magnetic resonance imaging (MRI) is still less accurate owing to the low intensity contrast among tissues. To tackle the problem, we introduce an accurate segmentation method for volumetric infant brain MRI built upon a densely connected network that achieves state-of-the-art accuracy. Specifically, we carefully design a fully convolutional densely connected network with skip connections such that the information from different levels of dense blocks can be directly combined to achieve highly accurate segmentation results. The proposed network, called 3D-SkipDenseSeg, exploits the advantage of the recently DenseNet for classification task and extends this to segment the 6-month infant brain tissue segmentation of magnetic resonance imaging (MRI). Experimental results demonstrate a competitive performance with regard to both segmentation accuracy and parameter efficiency of the proposed method over the existing methods; namely, the proposed 3D-SkipDenseSeg achieved the best dice similarity coefficient (DSC) of 90.37 +/- 1.38% (WM), 92.27 +/- 0.81% (GM), and 95.79 +/- 0.54% (CSF) among the 21 participating teams in the 6-month infant brain dataset (iSeg-2017) and required only 10-30% of the parameters compared to similar deep learning-based methods. (C) 2019 Elsevier Ltd. All rights reserved.

The deep convolutional neural network has achieved outstanding performance on neonatal brain MRI tissue segmentation. However, it may fail to produce reasonable results on unseen datasets that have different imaging appearance distributions with the training data. The main reason is that deep learning models tend to have a good fitting to the training dataset, but do not lead to a good generalization on the unseen datasets. To address this problem, we propose a multi-task learning method, which simultaneously learns both tissue segmentation and geodesic distance regression to regularize a shared encoder network. Furthermore, a dense attention gate is explored to force the network to learn rich contextual information. By using three neonatal brain datasets with different imaging protocols from different scanners, our experimental results demonstrate superior performance of our proposed method over the existing deep learning-based methods on the unseen datasets.

Random linear network coding is a promising solution to improve network capacity by allowing intermediate node that combines incoming packets into network coded packets. Nevertheless, there are two main drawbacks: rank deficiency and processing delay. The first drawback can be solved by performing systematic network coding, in which sender transmits all uncoded packets in the first phase and coded packets in the second phase. The processing delay still remains in practical communication system and multimedia content delivery. In this paper, we propose a practical incremental network coding for application of multimedia content delivery over multi-hop network. We implement the proposed scheme using Raspberry Pi as transmission node and evaluate performance in terms of throughput and the attainable video quality (i.e., peak signal-to-noise ratio). The results demonstrate that our proposed incremental network coding outperforms conventional network coding schemes.