AI helps you reading Science

AI generates interpretation videos

AI extracts and analyses the key points of the paper to generate videos automatically


pub
Go Generating

AI Traceability

AI parses the academic lineage of this thesis


Master Reading Tree
Generate MRT

AI Insight

AI extracts a summary of this paper


Weibo:
Our research aims to create a transfer learning model for multi-site functional magnetic resonance imaging analysis by using site-specific common features, but not by directly transferring the raw neural responses nor by finding a global shared space based on a set of subjects th...

Shared Space Transfer Learning for analyzing multi-site fMRI data

NIPS 2020, (2020)

Cited by: 0|Views31
EI
Full Text
Bibtex
Weibo

Abstract

Multi-voxel pattern analysis (MVPA) learns predictive models from task-based functional magnetic resonance imaging (fMRI) data, for distinguishing when subjects are performing different cognitive tasks -- e.g., watching movies or making decisions. MVPA works best with a well-designed feature set and an adequate sample size. However, mos...More

Code:

Data:

0
Introduction
  • The task-based functional magnetic resonance imaging is one of the prevalent tools in neuroscience to analyze how human brains work [1,2,3,4,5].
  • Multi-voxel pattern analysis (MVPA) learns a classification model based on a set of fMRI responses, which can be used to predict the cognitive tasks performed by a novel subject, who was not part of the training phase [1].
Highlights
  • The task-based functional magnetic resonance imaging is one of the prevalent tools in neuroscience to analyze how human brains work [1,2,3,4,5]
  • Multi-voxel pattern analysis (MVPA) learns a classification model based on a set of functional magnetic resonance imaging (fMRI) responses, which can be used to predict the cognitive tasks performed by a novel subject, who was not part of the training phase [1]
  • As the primary contribution of this paper, we propose Shared Space Transfer Learning (SSTL) as a novel transfer learning (TL) approach that can generate a robust, generalized, accurate classification model from multi-site fMRI datasets, which can be used effectively over each of these sites
  • Our research aims to create a TL model for multi-site fMRI analysis by using site-specific common features, but not by directly transferring the raw neural responses [2] nor by finding a global shared space based on a set of subjects that are appeared in each pair of sites [3]
  • We propose the Shared Space Transfer Learning (SSTL) as a novel transfer learning (TL) technique that can be used for homogeneous multi-site fMRI analysis
  • We develop the Shared Space Transfer Learning (SSTL) as a novel transfer learning (TL) approach that can functionally align homogeneous multi-site fMRI datasets and so improve the prediction performance in every site
Results
  • As the primary contribution of this paper, the authors propose Shared Space Transfer Learning (SSTL) as a novel TL approach that can generate a robust, generalized, accurate classification model from multi-site fMRI datasets, which can be used effectively over each of these sites.
  • This section introduces the proposed Shared Space Transfer Learning (SSTL) as a novel TL approach that can improve the performance of the MVPA on homogeneous multi-site fMRI datasets.
  • The authors' research aims to create a TL model for multi-site fMRI analysis by using site-specific common features, but not by directly transferring the raw neural responses [2] nor by finding a global shared space based on a set of subjects that are appeared in each pair of sites [3].
  • The authors compare SSTL with 6 different existing methods: raw neural responses in MNI space without using TL methods [3], the shared response model (SRM) [3, 5], the maximum independence domain adaptation (MIDA) [17], the Side Information Dependence Regularization (SIDeR) [2], the multidataset dictionary learning (MDDL) [3], and the multi-dataset multi-subject (MDMS) [3].
  • SSTL provides most accurate TL models that lead to better performance, by (1) using a multi-view approach to generate the site-specific common features, (2) using these common features for transferring data to the global shared space.
  • Figure 2 shows the effect of different transfer learning approaches (i.e., MIDA, SIDeR, and SSTL) on the performance of the multi-site fMRI analysis, with 2[a–d] showing
  • These empirical results show that using site-specific common features for transferring multi-site fMRI datasets can boost the performance of the MPV analysis.
  • The runtime of single-view approaches was better than the multi-view methods, perhaps because they process all instances in the training set together rather than analyzing neural responses belonging to each subject separately.
Conclusion
  • The authors propose the Shared Space Transfer Learning (SSTL) as a novel transfer learning (TL) technique that can be used for homogeneous multi-site fMRI analysis.
  • The authors develop the Shared Space Transfer Learning (SSTL) as a novel transfer learning (TL) approach that can functionally align homogeneous multi-site fMRI datasets and so improve the prediction performance in every site.
Summary
  • The task-based functional magnetic resonance imaging is one of the prevalent tools in neuroscience to analyze how human brains work [1,2,3,4,5].
  • Multi-voxel pattern analysis (MVPA) learns a classification model based on a set of fMRI responses, which can be used to predict the cognitive tasks performed by a novel subject, who was not part of the training phase [1].
  • As the primary contribution of this paper, the authors propose Shared Space Transfer Learning (SSTL) as a novel TL approach that can generate a robust, generalized, accurate classification model from multi-site fMRI datasets, which can be used effectively over each of these sites.
  • This section introduces the proposed Shared Space Transfer Learning (SSTL) as a novel TL approach that can improve the performance of the MVPA on homogeneous multi-site fMRI datasets.
  • The authors' research aims to create a TL model for multi-site fMRI analysis by using site-specific common features, but not by directly transferring the raw neural responses [2] nor by finding a global shared space based on a set of subjects that are appeared in each pair of sites [3].
  • The authors compare SSTL with 6 different existing methods: raw neural responses in MNI space without using TL methods [3], the shared response model (SRM) [3, 5], the maximum independence domain adaptation (MIDA) [17], the Side Information Dependence Regularization (SIDeR) [2], the multidataset dictionary learning (MDDL) [3], and the multi-dataset multi-subject (MDMS) [3].
  • SSTL provides most accurate TL models that lead to better performance, by (1) using a multi-view approach to generate the site-specific common features, (2) using these common features for transferring data to the global shared space.
  • Figure 2 shows the effect of different transfer learning approaches (i.e., MIDA, SIDeR, and SSTL) on the performance of the multi-site fMRI analysis, with 2[a–d] showing
  • These empirical results show that using site-specific common features for transferring multi-site fMRI datasets can boost the performance of the MPV analysis.
  • The runtime of single-view approaches was better than the multi-view methods, perhaps because they process all instances in the training set together rather than analyzing neural responses belonging to each subject separately.
  • The authors propose the Shared Space Transfer Learning (SSTL) as a novel transfer learning (TL) technique that can be used for homogeneous multi-site fMRI analysis.
  • The authors develop the Shared Space Transfer Learning (SSTL) as a novel transfer learning (TL) approach that can functionally align homogeneous multi-site fMRI datasets and so improve the prediction performance in every site.
Tables
  • Table1: The datasets
Download tables as Excel
Related work
  • Transfer learning (TL) has a wide range of applications in machine learning — e.g., computer vision, or neural language processing [2, 3, 8,9,10]. However, most of TL techniques cannot be directly used for fMRI analysis [2]. There are several issues [2, 3]. First, fMRI signals (voxel values) have different properties in comparison with other types of data — such as natural images or texts [2]. In particular, the brain signals are highly-correlated with a low rate of the signal to noise ratio (SNR) that relies heavily on derived properties [4]. Moreover, each person has a different neural response for each individual stimulus because different brains have different connectomes [1, 2, 5]. Recent (single site) studies show that the neural responses of all subjects (in that site) can be considered as the noisy rotations of a common template [1, 3,4,5].
Funding
  • This work was supported by the National Natural Science Foundation of China (Nos. 61876082, 61732006, 61861130366), the National Key R&D Program of China (Grant Nos. 2018YFC2001600, 2018YFC2001602, 2018ZX10201002), the Research Fund for International Young Scientists of China (NSFC Grant No 62050410348), the Royal Society-Academy of Medical Sciences Newton Advanced Fellowship (No NAF\R1\180371), the Natural Sciences and Engineering Research Council (NSERC) of Canada, and the Alberta Machine Intelligence Institute (Amii)
Study subjects and analysis
datasets: 8
Our comprehensive experiments validate that SSTL achieves superior performance to other state-of-the-art analysis techniques. Table 1 lists the 8 datasets (A to H) used for our empirical studies. These datasets are provided by Open NEURO repository and are separately preprocessed by easy fMRI4 and FSL 6.0.15 — i.e., normalization, smoothing, anatomical alignment, temporal alignment

reports the empirical studies: 4
Section 3 presents our proposed method. Section 4 reports the empirical studies, and finally, Section 5 presents the conclusion and points out some future works. Transfer learning (TL) has a wide range of applications in machine learning — e.g., computer vision, or neural language processing [2, 3, 8,9,10]

datasets: 8
Sd is the number of subject; #1 is the number of stimulus categories; Td is the number of time points per subjects; #2 lists the other datasets that overlap with this dataset; #3 lists the other datasets whose neural responses can be transferred to this dataset. Table 1 lists the 8 datasets (A to H) used for our empirical studies. These datasets are provided by Open NEURO repository and are separately preprocessed by easy fMRI4 and FSL 6.0.15 — i.e., normalization, smoothing, anatomical alignment, temporal alignment

cases: 42
Note each of the 7 plots is comparing SSTL and χ1, for 6 different χ1 ∈ {MNI, MIDA, SIDeR, SRM, MDDL, MDMS}, for a total of 7 × 6 = 42 comparisons. A 2-sided t-test found p <0.05 in all 42 cases. 4.2 Multi-site classification analysis for sets of datasets that do not overlap

cases: 33
Note each of the 11 plots is comparing SSTL and χ2, for 3 different χ2 ∈ {MNI, MIDA, SIDeR}, for a total of 11 × 3 = 33 comparisons. A 2-sided t-test found p <0.05 in all 33 cases. 4.3 Runtime

Reference
  • Haxby, J.V. & Connolly, A.C. & Guntupalli, J.S. (2014) Decoding neural representational spaces using multivariate pattern analysis. Annual Review of Neuroscience, 37:435–456.
    Google ScholarLocate open access versionFindings
  • Zhou, S. & Li, W. & Cox, C.R. & Lu, H. (2020) Side Information Dependence as a Regularizer for Analyzing Human Brain Conditions across Cognitive Experiments. In: 34th Association for the Advancement of Artificial Intelligence (AAAI). Feb/7–12, New York.
    Google ScholarLocate open access versionFindings
  • Zhang, H. & Chen, P.H. & Ramadge, P.J. (2018) Transfer learning on fMRI datasets. In: 21st International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR 84:595–603, Apr/9–11, Lanzarote, Canary Islands.
    Google ScholarLocate open access versionFindings
  • Yousefnezhad, M. & Selvitella, A. & Han, L. & Zhang, D. (2020) Supervised Hyperalignment for multi-subject fMRI data alignment. IEEE Transactions on Cognitive and Developmental Systems. DOI: 10.1109/TCDS.2020.2965981.
    Findings
  • Chen, P.H. & Chen, J. & Yeshurun, Y. & Hasson, U. & Haxby, J.V. & Ramadge, P.J. (2015) A reduceddimension fMRI shared response model. 28th Advances in Neural Information Processing Systems (NIPS). pp. 460–468, Dec/7–12, Canada.
    Google ScholarLocate open access versionFindings
  • Vega, R. & Greiner, R. (2018) Finding Effective Ways to (Machine) Learn fMRI-Based Classifiers from Multi-site Data. Understanding and Interpreting Machine Learning in Medical Image Computing Applications. pp. 32–39, Sep/16–20, Spain.
    Google ScholarLocate open access versionFindings
  • Gorgolewski, K. & Esteban, O. & Schaefer, G. & Wandell, B. & Poldrack, R. (2017) OpenNeuro - a free online platform for sharing and analysis of neuroimaging data. In OHBM, 1677.
    Google ScholarLocate open access versionFindings
  • Wang, M. & Zhang, D. & Huang, J. & Yap, P.T. & Shen, D. & Liu, M. (2019) Identifying Autism Spectrum Disorder with Multi-Site fMRI via Low-Rank Domain Adaptation. IEEE Transactions on Medical Imaging. Aug/05, 39(3):644–655.
    Google ScholarLocate open access versionFindings
  • Gao, Y. & Zhou, B. & Zhou, Y. & Shi, L. & Tao Y. & Zhang, J. (2018) Transfer Learning-Based Behavioural Task Decoding from Brain Activity. In: The 2nd International Conference on Healthcare Science and Engineering (ICHSE). pp. 77–81, Jun/01–03, China.
    Google ScholarLocate open access versionFindings
  • Thomas, A.W. & Müller, K.R. & Samek, W. (2019) Deep transfer learning for whole-brain FMRI analyses. OR 2.0 Context-Aware Operating Theaters and Machine Learning in Clinical Neuroimaging. pp. 59–67, Springer.
    Google ScholarLocate open access versionFindings
  • Pan, S.J. & Tsang, I.W. & Kwok, J.T. & Yang, Q. (2011) Domain adaptation via transfer component analysis. IEEE Transactions on Neural Networks. 22(2):199–210.
    Google ScholarLocate open access versionFindings
  • Long, M. & Wang, J. & Ding, G. & Pan, S.J. & Philip, S.Y. (2013) Adaptation regularization: A general framework for transfer learning. IEEE Transactions on Knowledge and Data Engineering. 26(5):1076–1089.
    Google ScholarLocate open access versionFindings
  • Long, M. & Wang, J. & Ding, G. & Sun, J. & Yu, P.S. (2013) Transfer feature learning with joint distribution adaptation. In: International Conference on Computer Vision., pp. 2200–2207, Dec/1–8, Sydney.
    Google ScholarLocate open access versionFindings
  • Wang, J. & Feng, W. & Chen, Y. & Yu, H. & Huang, M. & Yu, P.S. (2018) Visual domain adaptation with manifold embedded distribution alignment. In 26th ACM international conference on Multimedia, pp. 402–410, Seoul.
    Google ScholarLocate open access versionFindings
  • Zhang, C. & Qiao, K. & Wang, L. & Tong, L. & Hu, G. & Zhang, R.Y. & Yan, B. (2019). A visual encoding model based on deep neural networks and transfer learning for brain activity measured by functional magnetic resonance imaging. Journal of Neuroscience Methods. Vol. 325, Sep/1, pp. 1–9.
    Google ScholarLocate open access versionFindings
  • Li, H. & Parikh, N.A. & He, L. (2018) A novel transfer learning approach to enhance deep neural network classification of brain functional connectomes. Frontiers in Neuroscience, 12:491.
    Google ScholarLocate open access versionFindings
  • Yan, K. & Kou, L. & Zhang, D. (2018) Learning domain-invariant subspace using domain features and independence maximization. IEEE Transactions on Cybernetics. 48(1):288–299.
    Google ScholarLocate open access versionFindings
  • Gretton, A. & Bousquet, O. & Smola, A. & Schölkopf, B. (2005) Measuring statistical dependence with Hilbert-Schmidt norms. International conference on Algorithmic Learning Theory. pp. 63–77, Springer.
    Google ScholarFindings
  • Mensch, A. & Mairal, J. & Bzdok, D. & Thirion, B. & Varoquaux, G. (2017) Learning neural representations of human cognition across many fMRI studies. Advances in Neural Information Processing Systems (NIPS). pp. 5883–5893, Dec/4-9, Long Beach, USA.
    Google ScholarLocate open access versionFindings
  • Westfall, J. & Nichols, T.E. & Yarkoni, T. (2016) Fixing the stimulus-as-fixed-effect fallacy in task fMRI. Wellcome Open Research. 1(23):1–24, The Wellcome Trust.
    Google ScholarLocate open access versionFindings
  • Rastogi, P. & Van D.B. & Arora, R. (2015) Multiview LSA: Representation Learning via Generalized CCA. 14th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (HLT-NAACL). pp. 556–566, May/31 to Jun/5, Denver, USA.
    Google ScholarFindings
  • Yousefnezhad, M. & Zhang, D. (2017) Deep Hyperalignment. 30th Advances in Neural Information Processing Systems (NIPS). Dec/4-9, Long Beach, USA.
    Google ScholarLocate open access versionFindings
  • Arora, R. & Livescu, K. (2012) Kernel CCA for multi-view learning of acoustic features using articulatory measurements. Machine Learning in Speech and Language Processing (MLSLP). pp. 34–37, Sep/14, Portland, USA.
    Google ScholarLocate open access versionFindings
  • Sapatnekar, S.S. (2011) Overcoming Variations in Nanometer-Scale Technologies. IEEE Journal on Emerging and Selected Topics in Circuits and Systems. 1(1):5–18.
    Google ScholarLocate open access versionFindings
  • Xue, G. & Aron, A.R. & Poldrack, R.A. (2008) Common neural substrates for inhibition of spoken and manual responses. Cerebral Cortex. 18(8):1923–1932.
    Google ScholarLocate open access versionFindings
  • Aron, A.R. & Behrens, T.E. & Smith, S. & Frank, M.J. & Poldrack, R.A. (2007) Triangulating a cognitive control network using diffusion-weighted magnetic resonance imaging (MRI) and functional MRI. Journal of Neuroscience. 27(14):3743–3752.
    Google ScholarLocate open access versionFindings
  • Kelly, A.C. & Uddin, L. Q. & Biswal, B.B. & Castellanos, F.X. & Milham, M.P. (2008) Competition between functional brain networks mediates behavioral variability. NeuroImage 39(1):527–537.
    Google ScholarLocate open access versionFindings
  • Veldhuizen, M.G. & Babbs, R.K. & Patel, B. & Fobbs, W. & Kroemer, N.B. & Garcia,E. & Yeomans, M.R. & Small, D.M. (2017) Integration of sweet taste andmetabolism determines carbohydrate reward. Current Biology. 27, 2476–2485.
    Google ScholarLocate open access versionFindings
  • Smola, A.J. & Schölkopf, B. (2004) A tutorial on support vector regression. Statistics and Computing. 14(3):199–222.
    Google ScholarLocate open access versionFindings
Author
Muhammad Yousefnezhad
Muhammad Yousefnezhad
Alessandro Selvitella
Alessandro Selvitella
Your rating :
0

 

Tags
Comments
小科