Training deep learning models on cardiac magnetic resonance imaging (CMR) can be a challenge due to the small amount of expert generated labels and inherent complexity of data source. Self-supervised contrastive learning (SSCL) has recently been shown to boost performance in several medical imaging tasks. However, it is unclear how much the pre-trained representation reflects the primary organ of interest compared to spurious surrounding tissue. In this work, we evaluate the optimal method of incorporating prior knowledge of anatomy into a SSCL training paradigm. Specifically, we evaluate using a segmentation network to explicitly local the heart in CMR images, followed by SSCL pretraining in multiple diagnostic tasks. We find that using a priori knowledge of anatomy can greatly improve the downstream diagnostic performance. Furthermore, SSCL pre-training with in-domain data generally improved downstream performance and more human-like saliency compared to end-to-end training and ImageNet pre-trained networks. However, introducing anatomic knowledge to pre-training generally does not have significant impact.

Introduction: Cardiac magnetic resonance (CMR) is frequently utilized to characterize etiology of cardiomyopathy (CM), but there is need for improved disease classification, standardization in the interpretation of findings, and throughput in analysis. Radiomics has been shown to classify disease in a semi-automated manner. Alternatively, deep learning (DL) provides the ability to identify unknown features in image data. Therefore, we sought to compare DL and radiomic approaches to differentiate ischemic vs non-ischemic cardiomyopathy (ICM vs NICM), using cardiac magnetic resonance (CMR) short axis cine images. Methods: We selected 291 patients with cardiomyopathy (CM) who underwent a CMR exam at Cleveland Clinic between 2008 and 2018, of which 249 had NICM (positive label) based on expert review of the CMR exam and electronic medical record documentation. We compared a radiomic and end-to-end DL approach to identify CM etiology from short axis cine images. Automatically generated radiomic features describing myocardial shape, texture, thickness, and motion in the cine images were used to train several machine learning classifiers. In the DL approach, we directly used the cine images to train several DL classifiers, without extracting radiomic features. We evaluated the classifiers through 5-fold cross validation using the area under the curve (AUC), F1-score, and accuracy metrics. Statistical significance was evaluated using paired 2-tailed t-test at 0.05 level. Results: Support vector machine (SVM) and DenseNet121 achieved the best metrics for radiomic and DL approaches respectively. The radiomic and DL approach achieved similar AUCs of 0.852 and 0.858 respectively, but DL approach achieved statistically significant higher F1-score of 0.758 vs 0.585 of the radiomic approach. Conclusions: An end-to-end DL approach more accurately identified NICM vs ICM compared to a radiomics approach, using only cine CMR images.

To better understand early brain development in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; h owever, one of the major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological development in the community, the iSeg-2019 challenge (http://iseg2019.web.unc.edu) provides a set of 6-month infant subjects from multiple sites with different protocols/scanners for the participating methods. T raining/validation subjects are from UNC (MAP) and testing subjects are from UNC/UMN (BCP), Stanford University, and Emory University. By the time of writing, there are 30 automatic segmentation methods participated in the iSeg-2019. In this article, 8 top-ranked methods were reviewed by detailing their pipelines/implementations, presenting experimental results, and evaluating performance across different sites in terms of whole brain, regions of interest, and gyral landmark curves. We further pointed out their limitations and possible directions for addressing the multi-site issue. We find that multi-site consistency is still an open issue. We hope that the multi-site dataset in the iSeg-2019 and this review article will attract more researchers to address the challenging and critical multi-site issue in practice.

Deep convolutional neural networks (DCNN) have proven to be the state-of-the-art methods for brain tissue segmentation; however, their complex architectures, and the large number of parameters make them compu-tationally expensive and difficult to optimize. In this paper, a novel 3D DCNN architecture, which is built upon the U-Net structure, is presented for compact feature representation and efficient parameter reduction in order to segment the brain tissues into white matter, gray matter, and cerebrospinal fluid (Code is available at: htt ps://github.com/basnetr/U-DenseResNet). The basic idea in the proposed method is to use densely connected convolutional layers and residual skip-connections in order to increase the representation capacity, improve the gradient flow, facilitate easier and better learning, and reduce the number of parameters of the network. The loss functions, cross-entropy, dice similarity, and a combination of the two are used for the training of the proposed network. Experimental results show that the proposed approach provides the best performance on the test dataset of the single-modality IBSR18 dataset containing MR scans of diverse age groups and competitive per-formance on the multi-modality brain tissue segmentation challenge, iSeg-2017, containing MR scans of infants while reducing, for both the datasets, the parameters ranging from 40% to 98% compared to that of the other deep-learning based architectures. The proposed method significantly reduces the number of parameters of DCNNs while still providing high degree of accuracy. The proposed method can be used for the study of brain structure and development, in detecting a wide range of abnormal tissues, to aid diagnosis, and for guiding surgical procedures.

Automatic estimation of emotional state has been of great interest as emotion is an important component in user-oriented interactive technologies. This paper investigates the usage of feed-forward convolutional neural network (CNN) and features extracted from such networks for predicting dimensions of continuous-level emotional states. In this context, a two-stream CNN architecture wherein the video and audio data are learned simultaneously, is proposed. End-to-end mapping of audiovisual data to emotional dimensions reveals that the two-stream network performs better than its single-stream counterpart. The representations learned by the CNNs are refined through a minimum redundancy maximum relevance statistical selection method. Then, the support vector regression applied to selected CNN-based features estimates the instantaneous values of emotional dimensions. The proposed method is trained and tested using the audiovisual conversations of well-known RECOLA and SEMAINE databases. Experimentally it is verified that the regression of the CNN-based features outperforms the traditional audiovisual affective features as well as the end-to-end CNN mapping. Through generalization experiments, it is also observed that the learned representations are robust enough to provide an acceptable prediction performance, when the settings of training and testing datasets are widely different. (C) 2019 Elsevier B.V. All rights reserved.

Automatic prediction of continuous-level emotional state requires selection of suitable affective features to develop a regression system based on supervised machine learning. This paper investigates the performance of features statistically learned using convolutional neural networks for instantaneously predicting the continuous dimensions of emotional states. Features with minimum redundancy and maximum relevancy are chosen by using the mutual information-based selection process. The performance of frame-by-frame prediction of emotional state using the moderate length features as proposed in this paper is evaluated on spontaneous and naturalistic human-human conversation of RECOLA database. Experimental results show that the proposed model can be used for instantaneous prediction of emotional state with an accuracy higher than traditional audio or video features that are used for affective computation.