As the field expands from early research into the human connectome, there has been a fast expansion in the number of analytical approaches to study resting state functional MRI (rsfMRI) data. With increasing focus on individual differences, topographical brain maps of spatial organization have emerged in addition to traditional functional connectomes. Here, we developed a deep-learning model to embed maps of network topography and faithfully translate to individualized connectomes. Results confirmed the validity of the surface vision transformer based on reconstruction accuracy (0.73±0.09) and accurate topography-to-connectome translation (0.43±0.08). Importantly, translated connectomes retained identifiability and brain-cognition associations. These findings establish a direct mapping from spatial topography to connectomes that can be used to integrate scientific insights across rsfMRI sub-fields. This is an important step towards broadening our conceptualization of the connectome and supporting broader integration of findings to inform a complete understanding of the human connectome.

Cortical folding of the brain is widely regarded as an interplay between genetic programming and biomechanical forces, closely linked to cytoarchitectonic regionalisation. Abnormal folding patterns are frequently observed in neurodevelopmental conditions and psychiatric disorders. However, significant inter-individual variability of secondary and tertiary folds obscures the detection of shape biomarkers and confounds the investigation of folding-functional relationships. Here, we investigate cortical folding heterogeneity at a fine scale, using Multimodal Surface Matching with Hierarchical Templates (MSM-HT), a hierarchical surface registration, to parse cortical folding patterns into a representative family of distinct anatomical templates. By applying this technique both to young adults from the Human Connectome Project (HCP) and neonates in the Developing HCP and Brain Imaging in Babies (BIBS) cohorts, we identify and characterise common lobe-wise folding patterns: observing consistency across both age groups, with neonatal samples showing less variation. Crucially, we highlight significant hemispheric asymmetry within the temporal lobe in adults, with a consistent trend in neonates. This study provides a critical step towards understanding brain asymmetry and complex relationships between folding and function, offering a robust framework to generalise the uncovered cortical folding motifs across datasets and developmental stages. Guo and colleagues parse cortical folding variability into a family of distinct anatomical motifs that generalise across adults and neonates. These motifs shed light on structural asymmetry, heritability, and folding-function links.

Cortical folding of brain is widely regarded as an interplay between genetic programming and biomechanical forces, closely linked to cytoarchitectonic regionalisation. Abnormal folding patterns are frequently observed in neurodevelopmental conditions and psychiatric disorders. However, significant inter-individual variability of secondary and tertiary folds obscures detection of shape biomarkers and confounds investigation of folding-functional relationships. Here we investigate cortical folding heterogeneity at fine scale, using novel hierarchical surface registration (MSM-HT) to parse cortical folding patterns into a representative family of distinct anatomical templates. By applying this technique both to young adults from the Human Connectome Project (HCP) and neonates in the Developing HCP and Brain Imaging in Babies (BIBS) cohorts, we identify and characterize common lobe-wise folding patterns: observing consistency across both age groups, with neonatal samples showing less variation. Crucially, we highlight significant hemispheric asymmetry within the temporal lobe for both adults and neonates, and show that improved correspondence of shape does not translate to improved areal correspondence, affirming previous studies that have pointed to dissociation of folding and functional organisation. This study provides a critical step towards understanding brain asymmetry and complex relationships between folding and function, offering a robust framework to generalise the uncovered cortical folding motifs across datasets and developmental stages. ### Competing Interest Statement The authors have declared no competing interest.

Background To compare the efficiency of three artificial intelligence (AI) frameworks (Standard Machine Learning (ML), Multi-Layer Perceptron (MLP) and Convolution Neural Networks (CNN)) with a reference method (Mean radius of curvature (K)) to predict the posterior radius of curvature of the best-fitted rigid contact lens (RCBFL) in keratoconus eyes. Methods This retrospective study included 197 keratoconus eyes of 135 patients fitted with Rose K2® (Menicon®, Nagoya, Japan) rigid contact lenses with one or more topographies available (MS39®, CSO®, Ferrara, Italy) between January 2020 and September 2022. Two types of topographic data (indices and reconstructed maps from raw data) were used for AI analysis. Three distinct approaches were utilized for leveraging AI: Standard ML methods and MLPs based on topographic indices and CNNs based on topographic maps (i.e., corneal thickness, sagittal, and tangential maps). Seventeen AI framework’s accuracies were compared with the r2 determination coefficient of linear regression between predicted and best-fitted radii. Framework accuracies were compared with the Fisher z-transformation of Pearson correlation coefficients. Results In multiple linear regression, only three topographic indices (i.e., 3- & 5-mm mean K and Kmax) were significantly associated with RCBFL (p ≤ 0.0001). Compared with the reference method (mean-K; r2 = 0.36), a significantly better RCBFL prediction was achieved with Random Forest using the three topographic indices, MLP using all indices, ResNet18 CNN using anterior topographic maps and CNNs using combined parameters (0.69 ≤ r2 ≤ 0.80; p < 0.05). The best accuracy was obtained with the EfficientNetB0 CNN trained with three maps (r2 = 0.80). Conclusions Artificial intelligence methods, particularly CNNs, with corneal topography data of MS39® topographer, have demonstrated superiority over conventional approaches in predicting the posterior curvature radius of Rose K2® rigid contact lenses in patients with keratoconus.

Clinical adoption of deep learning models has been hindered, in part, because the black-box nature of neural networks leads to concerns regarding their trustworthiness and reliability. These concerns are particularly relevant in the field of neuroimaging due to the complex brain phenotypes and inter-subject heterogeneity often encountered. The challenge can be addressed by interpretable deep learning (iDL) methods that enable the visualisation and interpretation of the inner workings of deep learning models. This study systematically reviewed the literature on neuroimaging applications of iDL methods and critically analysed how iDL explanation properties were evaluated. Seventy-five studies were included, and ten categories of iDL methods were identified. We also reviewed five properties of iDL explanations that were analysed in the included studies: biological validity, robustness, continuity, selectivity, and downstream task performance. We found that the most popular iDL approaches used in the literature may be sub-optimal for neuroimaging data, and we discussed possible future directions for the field.

Cortical surface analysis has gained increased prominence, given its potential implications for neurological and developmental disorders. Traditional vision diffusion models, while effective in generating natural images, present limitations in capturing intricate development patterns in neuroimaging due to limited datasets. This is particularly true for generating cortical surfaces where individual variability in cortical morphology is high, leading to an urgent need for better methods to model brain development and diverse variability inherent across different individuals. In this work, we proposed a novel diffusion model for the generation of cortical surface metrics, using modified surface vision transformers as the principal architecture. We validate our method in the developing Human Connectome Project (dHCP), the results suggest our model demonstrates superior performance in capturing the intricate details of evolving cortical surfaces. Furthermore, our model can generate high-quality realistic samples of cortical surfaces conditioned on postmenstrual age(PMA) at scan.

Surface meshes are a favoured domain for representing structural and functional information on the human cortex, but their complex topology and geometry pose significant challenges for deep learning analysis. While Transformers have excelled as domain-agnostic architectures for sequence-to-sequence learning, the quadratic cost of the self-attention operation remains an obstacle for many dense prediction tasks. Inspired by some of the latest advances in hierarchical modelling with vision transformers, we introduce the Multiscale Surface Vision Transformer (MS-SiT) as a backbone architecture for surface deep learning. The self-attention mechanism is applied within local-mesh-windows to allow for high-resolution sampling of the underlying data, while a shifted-window strategy improves the sharing of information between windows. Neighbouring patches are successively merged, allowing the MS-SiT to learn hierarchical representations suitable for any prediction task. Results demonstrate that the MS-SiT outperforms existing surface deep learning methods for neonatal phenotyping prediction tasks using the Developing Human Connectome Project (dHCP) dataset. Furthermore, building the MS-SiT backbone into a U-shaped architecture for surface segmentation demonstrates competitive results on cortical parcellation using the UK Biobank (UKB) and manually-annotated MindBoggle datasets. Code and trained models are publicly available at https://github.com/metrics-lab/surface-vision-transformers.

The development of robust and generalisable models for encoding the spatio-temporal dynamics of human brain activity is crucial for advancing neuroscientific discoveries. However, significant individual variation in the organisation of the human cerebral cortex makes it difficult to identify population-level trends in these signals. Recently, Surface Vision Transformers (SiTs) have emerged as a promising approach for modelling cortical signals, yet they face some limitations in low-data scenarios due to the lack of inductive biases in their architecture. To address these challenges, this paper proposes the surface Masked AutoEncoder (sMAE) and video surface Masked AutoEncoder (vsMAE) - for multivariate and spatio-temporal pre-training of cortical signals over regular icosahedral grids. These models are trained to reconstruct cortical feature maps from masked versions of the input by learning strong latent representations of cortical structure and function. Such representations translate into better modelling of individual phenotypes and enhanced performance in downstream tasks. The proposed approach was evaluated on cortical phenotype regression using data from the young adult Human Connectome Project (HCP) and developing HCP (dHCP). Results show that (v)sMAE pre-trained models improve phenotyping prediction performance on multiple tasks by $\ge 26\%$, and offer faster convergence relative to models trained from scratch. Finally, we show that pre-training vision transformers on large datasets, such as the UK Biobank (UKB), supports transfer learning to low-data regimes. Our code and pre-trained models are publicly available at https://github.com/metrics-lab/surface-masked-autoencoders.

Self-supervision has been widely explored as a means of addressing the lack of inductive biases in vision transformer architectures, which limits generalisation when networks are trained on small datasets. This is crucial in the context of cortical imaging, where phenotypes are complex and heterogeneous, but the available datasets are limited in size. This paper builds upon recent advancements in translating vision transformers to surface meshes and investigates the potential of Masked AutoEncoder (MAE) self-supervision for cortical surface learning. By reconstructing surface data from a masked version of the input, the proposed method effectively models cortical structure to learn strong representations that translate to improved performance in downstream tasks. We evaluate our approach on cortical phenotype regression using the developing Human Connectome Project (dHCP) and demonstrate that pre-training leads to a 26\% improvement in performance, with an 80\% faster convergence, compared to models trained from scratch. Furthermore, we establish that pre-training vision transformer models on large datasets, such as the UK Biobank (UKB), enables the acquisition of robust representations for finetuning in low-data scenarios. Our code and pre-trained models are publicly available at \url{https://github.com/metrics-lab/surface-vision-transformers}.

Cortical neurodevelopment is sensitive to disruption following preterm birth, with lasting impact on cognitive outcomes. The creation of generative models of neurodevelopment could aid clinicians in identifying atrisk subjects but is complicated by the degree of subject variability in cortical folding, and significant heterogeneity in the effect of preterm birth. In this work, we propose a graph convolutional generative adversarial network (GAN) and a training scheme to simulate neonatal cortical surface developmental trajectories. The proposed model is used to smoothly modify two cortical phenotypes: post-menstrual age at scan (PMA) and gestational age at birth (GA) on data from the developing Human Connectome Project (dHCP) [1]. The synthetic images were validated with an independently trained regression network, and compared against follow up scans, indicating that the model can realistically age individuals whilst preserving subject-specific cortical morphology. Deviation between simulated ‘healthy’ scans, and preterm follow up scans generated a metric of individual atypicality, which improved prediction of 18-month cognitive outcome over GA alone.

The Human Connectome Project Multimodal Parcellation (HCP_MMP1.0) provides a robust in vivo map of the cerebral cortex, which demonstrates variability in structure and function that cannot be captured through diffeomorphic image registration alone. The HCP successfully employed a fully-connected neural network architecture to capture this variation, however it is unclear whether this approach generalises to other datasets with less rich imaging protocols. In this paper we propose and validate a novel geometric deep learning framework for generating individualised HCP_MMP1.0 parcellations in UK Biobank (UKB), an extremely rich resource that has led to numerous breakthroughs in neuroscience. To address substantial differences in image acquisition (for example, 6 minutes of resting-state fMRI per subject for UKB vs. 60 minutes per subject for HCP), we introduce a multi-step learning procedure including pretraining with a convolutional autoencoder. Compared to a fully-connected baseline, our proposed framework improved average detection rate across all areas by 10.4% (99.1% vs 88.7%), and detection of the worst performing area by 51.0% (80.9% vs. 29.9%). Importantly, this was not a result of the framework predicting one consistent parcellation across subjects, as visual inspection indicated that our method was sensitive to atypical cortical topographies. Code and trained models will be made available at https://www.github.com .

Structural MRI-derived features of the cortical surface are known to correlate to phenotypes such as age, sex and cognitive outcomes. Deep generative modelling of cortical neurodevelopment can lead to clinically interpretable models of disease or identify atypical cases for clinical intervention, but deep modelling of non-Euclidean domains, such as surfaces, poses additional challenges. In this work, we adapt a graph convolutional network (GCN) to model the neonatal cortical surface, and synthesise realistic, age-conditioned images of myelination and sulcal depth cortical surface maps. We train our models without longitu-dinal data, using randomised aging cycles of varying length, which we validate by ablation and with comparison to a CycleGAN. An independently trained deep regression model evaluates the accuracy of the generated images as the difference between their apparent post-menstrual age (PMA) and their respective target ages, obtaining a mean absolute error (MAE) of 1.02 ± 0.28 weeks (baseline accuracy 0.6 weeks).

Recent state-of-the-art performances of Vision Transformers (ViT) in computer vision tasks demonstrate that a general-purpose architecture, which implements long-range self-attention, could replace the local feature learning operations of convolutional neural networks. In this paper, we extend ViTs to surfaces by reformulating the task of surface learning as a sequence-to-sequence learning problem, by proposing patching mechanisms for general surface meshes. Sequences of patches are then processed by a transformer encoder and used for classification or regression. We validate our method on a range of different biomedical surface domains and tasks: brain age prediction in the developing Human Connectome Project (dHCP), fluid intelligence prediction in the Human Connectome Project (HCP), and coronary artery calcium score classification using surfaces from the Scottish Computed Tomography of the Heart (SCOT-HEART) dataset, and investigate the impact of pretraining and data augmentation on model performance. Results suggest that Surface Vision Transformers (SiT) demonstrate consistent improvement over geometric deep learning methods for brain age and fluid intelligence prediction and achieve comparable performance on calcium score classification to standard metrics used in clinical practice. Furthermore, analysis of transformer attention maps offers clear and individualised predictions of the features driving each task. Code is available on Github: https://github.com/metrics-lab/surface-vision-transformers

The extension of convolutional neural networks (CNNs) to non-Euclidean geometries has led to multiple frameworks for studying manifolds. Many of those methods have shown design limitations resulting in poor modelling of long-range associations, as the generalisation of convolutions to irregular surfaces is non-trivial. Motivated by the success of attention-modelling in computer vision, we translate convolution-free vision transformer approaches to surface data, to introduce a domain-agnostic architecture to study any surface data projected onto a spherical manifold. Here, surface patching is achieved by representing spherical data as a sequence of triangular patches, extracted from a subdivided icosphere. A transformer model encodes the sequence of patches via successive multi-head self-attention layers while preserving the sequence resolution. We validate the performance of the proposed Surface Vision Transformer (SiT) on the task of phenotype regression from cortical surface metrics derived from the Developing Human Connectome Project (dHCP). Experiments show that the SiT generally outperforms surface CNNs, while performing comparably on registered and unregistered data. Analysis of transformer attention maps offers strong potential to characterise subtle cognitive developmental patterns.

The extension of convolutional neural networks (CNNs) to non-Euclidean geometries has led to multiple frameworks for studying manifolds. Many of those methods have shown design limitations resulting in poor modelling of long-range associations, as the generalisation of convolutions to irregular surfaces is non-trivial. Recent state-of-the-art performance of Vision Transformers (ViTs) demonstrates that a general-purpose architecture, which implements self-attention, could replace the local feature learning operations of CNNs. Motivated by the success of attention-modelling in computer vision, we extend ViTs to surfaces by reformulating the task of surface learning as a sequence-to-sequence problem and propose a patching mechanism for surface meshes. We validate the performance of the proposed Surface Vision Transformer (SiT) on two brain age prediction tasks in the developing Human Connectome Project (dHCP) dataset and investigate the impact of pre-training on model performance. Experiments show that the SiT outperforms many surface CNNs, while indicating some evidence of general transformation invariance. Code available at https://github.com/metrics-lab/surface-vision-transformers

The study of functional brain connectivity (FC) is important for understanding the underlying mechanisms of many psychiatric disorders. Many recent analyses adopt graph convolutional networks, to study non-linear interactions between functionally-correlated states. However, although patterns of brain activation are known to be hierarchically organised in both space and time, many methods have failed to extract powerful spatio-temporal features. To overcome those challenges, and improve understanding of long-range functional dynamics, we translate an approach, from the domain of skeleton-based action recognition, designed to model interactions across space and time. We evaluate this approach using the Human Connectome Project (HCP) dataset on sex classification and fluid intelligence prediction. To account for subject topographic variability of functional organisation, we modelled functional connectomes using multi-resolution dual-regressed (subject-specific) ICA nodes. Results show a prediction accuracy of 94.4% for sex classification (an increase of 6.2% compared to other methods), and an improvement of correlation with fluid intelligence of 0.325 vs 0.144, relative to a baseline model that encodes space and time separately. Results suggest that explicit encoding of spatio-temporal dynamics of brain functional activity may improve the precision with which behavioural and cognitive phenotypes may be predicted in the future.

The emerging field of geometric deep learning extends the application of convolutional neural networks to irregular domains such as graphs, meshes and surfaces. Several recent studies have explored the potential for using these techniques to analyse and segment the cortical surface. However, there has been no comprehensive comparison of these approaches to one another, nor to existing Euclidean methods, to date. This paper benchmarks a collection of geometric and traditional deep learning models on phenotype prediction and segmentation of sphericalised neonatal cortical surface data, from the publicly available Developing Human Connectome Project (dHCP). Tasks include prediction of postmenstrual age at scan, gestational age at birth and segmentation of the cortical surface into anatomical regions defined by the M-CRIB-S atlas. Performance was assessed not only in terms of model precision, but also in terms of network dependence on image registration, and model interpretation via occlusion. Networks were trained both on sphericalised and anatomical cortical meshes. Findings suggest that the utility of geometric deep learning over traditional deep learning is highly task-specific, which has implications for the design of future deep learning models on the cortical surface. The code, and instructions for data access, are available from https://github.com/Abdulah-Fawaz/Benchmarking-Surface-DL .

Machine learning is the brain of an artificial intelligence machine. We describe how it is performed and explore its current use in respiratory medicine. Potential future applications and possible issues in integration into clinical practice are discussed. http://bit.ly/31XVruW