Prediction of Infant Cognitive Development with Cortical Surface-Based Multimodal Learning

Exploring the relationship between the cognitive ability and infant cortical structural and functional development is critically important to advance our understanding of early brain development, which, however, is very challenging due to the complex and dynamic brain development in early postnatal stages. Conventional approaches typically use either the structural MRI or resting-state functional MRI and rely on the region-level features or inter-region connectivity features after cortical parcellation for predicting cognitive scores. However, these methods have two major issues: 1) spatial information loss, which discards the critical fine-grained spatial patterns containing rich information related to cognitive development; 2) modality information loss, which ignores the complementary information and the interaction between the structural and functional images. To address these issues, we unprecedentedly invent a novel framework, namely cortical surface-based multimodal learning framework (CSML), to leverage fine-grained multimodal features for cognition development prediction. First, we introduce the fine-grained surface-based data representation to capture spatially detailed structural and functional information. Then, a dual-branch network is proposed to extract the discriminative features for each modality respectively and further captures the modality-shared and complementary information with a disentanglement strategy. Finally, an age-guided cognition prediction module is developed based on the prior that the cognition develops along with age. We validate our method on an infant multimodal MRI dataset with 318 scans. Compared to state-of-the-art methods, our method consistently achieves superior performances, and for the first time suggests crucial regions and features for cognition development hidden in the fine-grained spatial details of cortical structure and function.