Analyzing neuroimaging data at the population level relies on averaging images that have been acquired on a group of individuals drawn from this population. Traditional group analyses are based on the general linear model, which performs euclidean averaging across individuals, independently at each brain location. It is therefore largely impacted by interindividual differences. In this paper we propose to overcome this variability by using optimal transport to leverage the geometrical properties of multivariate brain patterns. We extend the concept of Wasserstein barycenter, which was initially meant to average probability measures, to make it applicable to arbitrary data that do not necessarily fulfill the properties of a true probability measure. For this, we introduce a new algorithm that estimates a barycenter using the transportation L p distance [8]. We provide an experimental study on how the noise level impacts the quality of the obtained barycenter on artificial data. Our proposed method is compared with the approach introduced in [6] on artificial and real functional MRI.
The use of machine-learning (ML) in neuroimaging offers new perspectives in early diagnosis and prognosis of brain diseases. Indeed, ML algorithms can jointly examine all brain features to capture complex relationships in the data in order to make inferences at a single-subject level. To deal with such high dimensional input and the associated risk of overfitting on the training data, a proper regularization (or feature selection) is required. Standard ℓ 2 -regularized predictors, such as Support Vector Machine, provide dense patterns of predictors. However, in the context of predictive disease signature discovery, it is now essential to understand the brain pattern that underpins the prediction. Despite ℓ 1 -regularized (sparse) has often been advocated as leading to more interpretable models, they generally lead to scattered and unstable patterns. We hypothesize that the integration of prior knowledge regarding the structure of the input images should improve the relevance and the stability of the predictive signature. Such structured sparsity can be obtained by combining together ℓ 1 (possibly ℓ 2 ) and Total variation (TV) penalties. We demonstrated the relevance of using ML with structured sparsity on a large multisite dataset of schizophrenia patients and controls. Using 3D maps of grey matter density, we obtained promising inter-site prediction performances. More importantly, we have uncovered a predictive signature of schizophrenia that is clinically interpretable and stable across resampling. This suggests that structured sparsity provides a major breakthrough over `off-the-shelf' algorithms to perform a robust selection of important brain regions in the context of biomarkers discovery.
In recent years, deep learning has transformed the field of machine learning. In the field of neuroimaging, there are increasing interests in adopting deep learning techniques. However, deep neural networks (DNNs) usually need large quantity of data to perform well, which is often lacking in neuroimaging. In this work, we evaluated three different DNNs (fully-connected neural network, BrainNetCNN [1], and graph convolutional neural network [2]) for functional connectivity (FC)-based prediction of fluid intelligence using the Human Connectome Project. These DNNs were compared with kernel regression, a classical machine learning algorithm. Our results suggested that the DNNs did not outperform kernel regression. However, we do not preclude the possibility that with more participants or different FC features, DNNs might eventually outperform their classical counterpart.
Drowsy driving poses considerable risk not only to drivers themselves but also to other people on the road. It has been demonstrated that information contained in electroencephalography (EEG) signal can be used to identify driving drowsiness. To date, most of work focused on the detection of drowsiness within a session. This hampers the generalization of the trained model to a following session conducted after a few days. As we know, EEG is non-stationary and changes dramatically across sessions, which leads to a great challenge how to establish a model that has a good performance across sessions. In this study, we combined boosting strategy and transfer learning method to establish a model for identifying driving drowsiness states from alertness states based on the features of power spectral density (PSD). The model trained using the data collected a few days ago (session1) was tuned using very small portion of the data collected in the current session can achieve a good performance as tested in the current session (session2). The results demonstrated that the proposed boosting transfer learning method significantly outperformed the support vector machine (SVM) and AdaBoost methods. The proposed method could promote practical use of drowsiness detection system in a real vehicle due to its good cross-session performance.
Brain signal variability has been proposed as an index of the brain’s cognitive capacity. In this work, we examined neural variability by calculating the standard deviation of single trial activation estimates during memory encoding in 30 patients with mild cognitive impairment (MCI) and 31 elderly controls. We deployed a random forest (RF) classifier, using variability maps as features to distinguish MCI patients from controls, and obtained classification accuracies of up to 86%. We then used partial least squares correlation to identify variability patterns associated with task performance and compared them to the weight maps obtained with the RF classifier.
Recently, deep learning (DL) techniques have been gaining interest in the neuroimaging community. In this study, we present 3D convolutional neural network (3D-CNN) as an end-to-end model to label a target task among four sensorimotor tasks for each functional magnetic resonance imaging (fMRI) volume. To the best of our knowledge, this is the first study that employs a single blood-oxygenation-level-dependent (BOLD) fMRI volume as the input of the 3D-CNN for task classification. We hypothesized that 3D-CNN has the capability to extract potentially shift-invariant features in local brain areas while preserving the overall spatial layout of the whole brain fMRI volume. We designed a 3D-CNN model by extending the LeNet-5 CNN for 2D image classification to 3D volume classification. The designed 3D-CNN model was thoroughly evaluated using BOLD fMRI volumes acquired from four sensorimotor tasks in terms of the classification performance and feature representations for each of the four sensorimotor tasks.
Since machine learning models have been applied to neuroimaging data, researchers have drawn conclusions from the derived weight maps. In particular, weight maps of classifiers between two conditions are often described as a proxy for the underlying signal differences between the conditions. Recent studies have however suggested that such weight maps could not reliably recover the source of the neural signals and even led to false positives (FP). In this work, we used semi-simulated data from ElectroCorticoGraphy (ECoG) to investigate how the signal-to-noise ratio and sparsity of the neural signal affect the similarity between signal and weights. We show that not all cases produce FP and that it is unlikely for FP features to have a high weight in most cases.
Low sample size and the absence of labels on certain data limits the performances of predictive algorithms. To overcome this problem, it is sometimes possible to learn a model on a large labeled auxiliary dataset. Yet, this assumes that the two datasets exhibit similar statistical properties which is rarely the case in practice: there is a discrepancy between the large dataset, called the source, and the dataset of interest, called the target. Improving the prediction performance on the target domain by reducing the distribution discrepancy, between the source and the target domains, is known as Domain Adaptation (DA). Presently, Optimal transport DA (OTDA) methods yield state-of-the-art performances on several DA problems. In this paper, we consider the problem of sleep stage classification, and use OTDA to improve the performances of a convolutional neural network. We use features learnt from the electroencephalogram (EEG) and the electrooculogram (EOG) signals. Our results demonstrate that the method significantly improves the network predictions on the target data.
Early identification of individuals at risk of developing Alzheimer’s disease (AD) dementia is important for developing disease-modifying therapies. In this study, given multimodal AD markers and clinical diagnosis of an individual from one or more timepoints, we seek to predict the clinical diagnosis, cognition and ventricular volume of the individual for every month (indefinitely) into the future. We proposed and applied a minimal recurrent neural network (minimalRNN) model to data from The Alzheimer’s Disease Prediction Of Longitudinal Evolution (TADPOLE) challenge, comprising longitudinal data of 1677 participants (Marinescu et al. 2018) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI). We compared the performance of the minimalRNN model and four baseline algorithms up to 6 years into the future. Most previous work on predicting AD progression ignore the issue of missing data, which is a prevalent issue in longitudinal data. Here, we explored three different strategies to handle missing data. Two of the strategies treated the missing data as a “preprocessing” issue, by imputing the missing data using the previous timepoint (“forward filling”) or linear interpolation (“linear filling). The third strategy utilized the minimalRNN model itself to fill in the missing data both during training and testing (“model filling”). Our analyses suggest that the minimalRNN with “model filling” compared favorably with baseline algorithms, including support vector machine/regression, linear state space (LSS) model, and long short-term memory (LSTM) model. Importantly, although the training procedure utilized longitudinal data, we found that the trained minimalRNN model exhibited similar performance, when using only 1 input timepoint or 4 input timepoints, suggesting that our approach might work well with just cross-sectional data. An earlier version of our approach was ranked 5th (out of 53 entries) in the TADPOLE challenge in 2019. The current approach is ranked 2nd out of 63 entries as of June 3rd, 2020.
Neural synchronization plays a crucial role in cognitive functions and in performing tasks as it facilitates the transmission of information among the various brain subregions, and thus their communication. In this paper, we use an approach for analyzing and quantifying the emergence of synchronization patterns used previously in the study of data from toy dynamical models, in neurophysiological signals from a macaque monkey and particularly, from prefrontal-cortex intracranial recordings. Specifically, we study the emergence of synchronization patterns in neural ensembles recorded in the macaque brain while the monkey is performing the same delayed saccade task successfully for a number of times. We quantify the emergence of chimera-like states, metastability and coalition entropy in the recordings coming from intracranial arrays implanted in the macaque’s brain. Our results show the emergence of spatio-temporal co-existing patterns of synchronized and desynchronized behavior, termed chimera-like states with small metastability during the stage where the target and the distractor appears on the screen and when the go cue appears on the screen for the monkey to report, namely the two most crucial stages of the trials to be termed successful. Finally, we perform a statistical hypothesis test on the calculated quantities over the successful trials and demonstrate that our findings are statistically significant in the sense that they cannot be attributed to randomness.
Predictive models applied on brain images can extract imaging biomarkers of pathologies or psychological traits. Yet, a successful prediction may be driven by a confounding effect that is correlated with the effect of interest. For instance fluid intelligence is strongly impacted by age; age is well predicted from brain images; hence successful prediction of fluid intelligence from brain images might have captured nothing more than a biomarker of aging. Here we introduce a non-sparametric approach to control for a confounding effect in a predictive model. It is based on crafting a test set on which the effect of interest is independent from the confounding effect. We name this strategy “anti mutual-information subsampling”. We demonstrate the approach with a large sample resting-state fMRI and psychometric data of healthy aging subjects (n = 608). We show that using a linear model to remove the effect of age on the brain signals (“deconfounding”) leads to pessimistic scores, as previously reported. Anti mutual-information subsampling does not require to remove from the brain signals the shared variance between aging and fluid intelligence, and hence does not display this pessimistic behavior. In addition, it is non-parametric and hence robust to violations of the linear hypothesis.
From a large clinical blood oxygen level dependent (BOLD) functional magnetic resonance imaging (fMRI) study, we report several interrelated findings involving transient supra-network brainwide states (dynamic polarity regimes (dPRs)) of fMRI activation in which large proportions of voxels are simultaneously exhibiting intensities in the upper third or simultaneously exhibiting intensities in the lower third of their own activation profiles. The presence of highly polarized states, those in which many voxels are simultaneously at the high or low end of their own activation profiles, is shown to be significantly anti-correlated with diagnosed schizophrenia and significantly correlated with transient patterns of both strongly modularized network connectivity and diffuse hyperconnectivity between functional brain networks on multiple timescales. Conversely, the weak under-modularized network connectivity and the inhibitory connections between default mode (DMN) and other networks characteristic of schizophrenia patients correlates significantly with non-polarized, more heterogeneous brainwide voxel activation levels. Moreover, the spatial distribution of voxels most likely to contribute to highly polarized states (polarity participation maps (PPMs)) also differ with a high degree of statistical significance between schizophrenia patients and healthy controls. Our findings highlight a particular whole-brain spatiotemporal BOLD activation phenomenon that underpins distinct network connectivity pathologies associated with a serious clinical disorder.
Motion artifacts on brain Magnetic Resonance Images (MRI) constitute an important factor that degrades the image quality, impacting the quantitative analysis based on structural segmentation. Thus, assessing the image quality is essential to determine if the image fulfills the minimal quality level necessary to the research analysis. Nowadays an MR expert, responsible for quality control, performs a visual check on every acquired image. The MRI database is huge, and this quality screening is time-consuming and fatiguing. We propose to automatically detect the images containing motion artifacts using Deep Convolutional Neural Networks (CNN), currently presenting the best performance on image classification contests. Four renowned architectures were chosen to be fine-tuned, and have their results combined to report the motion artifacts presence on the acquisition. Besides, as Deep CNN filters from lower layers map to smaller regions in the original input and our goal is to detect fine-grained image corruption, the CNNs were adapted to use the output from lower intermediate level as features to the binary classifier. The adapted CNNs were trained and tested using an annotated dataset composed of MRI T1-weighted volumetric acquisitions. The training subset contains 48 images, while the testing subset has 20 images. The method consists of two steps. Firstly CNNs were trained using patches from the three MRI planes (sagittal, axial and coronal). Secondly, the results from the patches are combined to provide the result per acquisition. On the second step, an Artificial Neural Network (ANN) classifier was trained combining the patches results from the four modified Deep CNNs and the patch location information. The overall performance on the test set was 88.27% per patch and 100% per acquisition. The proposed technique can be applied to large datasets, providing, to the quality control expert, the motion artifact presence probability, minimizing the time spent on manual quality control.
Cognitive and clinical neuroscience often rely on small datasets, gathered from restricted populations in lab settings only. Dry-sensor EEG is a technology that can be made affordable and easy to use. When combined with gamification of experimental tasks, this could provide a new way to gather data on a large scale. Signal quality is a challenge, but the ease of gathering repeated samples can compensate for this. In this paper we present a new system that uses a novel wearable EEG headset, a tablet-based suite of cognitive games, and a cloud-based data analysis system. A large 12-week at-home feasibility study is described, yielding high adherence and user acceptability, even among older users. Data from that trial replicates well-established EEG and behavioural patterns from the cognitive neuroscience literature. Some initial machine learning analyses show that user age and cognitive performance level can be discriminated from single daily sessions, with much higher classification performance when aggregating over multiple days.
Recent studies have suggested that early acquisitions of 18 F-FBB-PET data (eFBB) provides similar information to 18 F-FDG-PET images. As far as we know, presently this attractive idea has been only tested by experiments focused on the routine clinical practice. In this work, we compare the usefulness of FDG and eFBB images to separate Alzheimer's disease (AD) and non-AD patients using Computer Aided Diagnosis (CAD) systems based on machine learning. Specifically, a Support Vector Machine classifier was used to estimate the potential of both data modalities to separate the groups. Two dimensionality reduction approaches, one based on previous knowledge (predefined regions of interest) and other based on Principal Component Analysis were also investigated. The results suggest that eFBB images could be used as a surrogate of FDG data in CAD systems for AD. However we found slight differences that might indicate that FDG images are more suitable than eFBB data to model the metabolic changes of non-AD patients. In addition, using multimodal systems we evaluated weather FDG and eFBB images contain complementary information and would be worth to use them together.
Magnetoencephalography (MEG) is a functional neuroimaging tool that records the magnetic fields induced by electrical neuronal activity; however, signal from non-neuronal sources can corrupt the data. Eye-Blinks (EB) and Cardiac Activity (CA) are two of the most common types of non-neuronal artifacts. They can be measured by affixing eye proximal electrodes, as in electrooculography (EOG) and chest electrodes, as in electrocardiography (EKG), however this complicates imaging setup, decreases patient comfort, and often induces further artifacts from facial twitching and postural muscle movement. We propose an EOG- and EKG-free approach to identify eye-blink, cardiac, or neuronal signals for automated artifact suppression. Our contributions are two-fold. First, we combine a data driven, multivariate decomposition approach based on Independent Component Analysis (ICA) and a highly accurate classifier constructed as a deep 1-D Convolutional Neural Network. Second, we visualize the features learned to reveal what features the model uses and to bolster user confidence in our model's training and potential for generalization. We train and test three variants of our method on resting state MEG data from 49 subjects. Our cardiac model achieves a 96% sensitivity and 99% specificity on the set-aside test-set. Our eye-blink model achieves a sensitivity of 85% and specificity of 97%. This work facilitates automated MEG processing for both, clinical and research use, and can obviate the need for EOG or EKG electrodes.
Modern functional and diffusion magnetic resonance imaging (fMRI and dMRI) provide data from which macro-scale networks of functional and structural whole brain connectivity can be estimated. Although networks derived from these two modalities describe different properties of the human brain, they emerge from the same underlying brain organization, and functional communication is presumably mediated by structural connections. In this paper, we assess the structure-function relationship by evaluating how well functional connectivity can be predicted from structural graphs. Using high-resolution whole brain networks generated with varying density, we contrast the performance of several non-parametric link predictors that measure structural communication flow. While functional connectivity is not well predicted directly by structural connections, we show that superior predictions can be achieved by taking indirect structural pathways into account. In particular, we find that the length of the shortest structural path between brain regions is a good predictor of functional connectivity in sparse networks (density less than one percent), and that this improvement comes from integrating indirect pathways comprising up to three steps. Our results support the existence of important indirect relationships between structure and function, extending beyond the immediate direct structural connections that are typically investigated.
Functional Magnetic Resonance Imaging (fMRI) relies on multi-step data processing pipelines to accurately determine brain activity; among them, the crucial step of spatial smoothing. These pipelines are commonly suboptimal, given the local optimisation strategy they use, treating each step in isolation. With the advent of new tools for deep learning, recent work has proposed to turn these pipelines into end-to-end learning networks. This change of paradigm offers new avenues to improvement as it allows for a global optimisation. The current work aims at benefitting from this paradigm shift by defining a smoothing step as a layer in these networks able to adaptively modulate the degree of smoothing required by each brain volume to better accomplish a given data analysis task. The viability is evaluated on real fMRI data where subjects did alternate between left and right finger tapping tasks.
Prefrontal cortex (PFC) is thought to support the ability to focus on goal-relevant information by filtering out irrelevant information, a process akin to dimensionality reduction. Here, we find direct evidence of goal-directed data compression within medial PFC during learning, such that the degree of neural compression predicts an individual's ability to selectively attend to concept-specific information. These findings suggest a domain-general mechanism of learning through compression in mPFC.
Over the last decade, machine learning algorithms have proven to be useful tools for exploring neural representations of percepts and concepts in the brain. An important but often neglected next step is it to relate neural representations to human behavior. Here, we introduce a novel approach to definitively linking neural representations to structural properties of stimuli as well as human behavior by analyzing patterns of classification errors using linear mixed-effects (LME) models. An LME model includes a priori predictive models of matching of error patterns between neural decoding and human behavior as fixed effects as well as random effects to account for subject variability. Finally, we demonstrate the viability of this approach using data from a set of fMRI and behavioral experiments testing the influence of visual properties on the neural representation of categories of real-world visual scenes.
