Decoding fMRI data with support vector machines and deep neural networks

Background: Multivoxel pattern analysis (MVPA) examines fMRI activation patterns associated with different cognitive conditions. Support vector machines (SVMs) are the predominant method in MVPA. While SVM is intuitive and easy to apply, it is mainly suitable for analyzing data that are linearly separable. Convolutional neural networks (CNNs) are known to have the ability to approximate nonlinear relationships. Applications of CNN to fMRI data are beginning to appear with increasing frequency, but our understanding of the similarities and differences between CNN models and SVM models is limited.New method: We compared the two methods when they are applied to the same datasets. Two datasets were considered: (1) fMRI data collected from participants during a cued visual spatial attention task and (2) fMRI data collected from participants viewing natural images containing varying degrees of affective content.Results: We found that (1) both SVM and CNN are able to achieve above-chance decoding accuracies for attention control and emotion processing in both the primary visual cortex and the whole brain, (2) the CNN decoding accuracies are consistently higher than that of the SVM, (3) the SVM and CNN decoding accuracies are generally not correlated, and (4) the heatmaps derived from SVM and CNN are not significantly overlapping. Comparison with existing methods: By comparing SVM and CNN we pointed out the similarities and differences between the two methods.Conclusions: SVM and CNN rely on different neural features for classification. Applying both to the same data may yield a more comprehensive understanding of neuroimaging data.