Accurate instance segmentation of tooth and pulp from cone-beam computed tomography (CBCT) images is essential but highly challenging due to the pulp's small structures and indistinct boundaries. To address these critical challenges, we propose TIPs designed for Tooth Instance and Pulp segmentation. TIPs initially employs a backbone model to segment a binary mask of the tooth from CBCT images, which is then utilized to derive position prior of the tooth and shape prior of the pulp. Subsequently, we propose the Hierarchical Fusion Mamba models to leverage the strengths of both anatomical priors and CBCT images by extracting and integrating shallow and deep features from Convolution Neural Networks (CNNs) and State Space Sequence Models (SSMs), respectively. This process achieves tooth instance and pulp segmentation, which are then combined to obtain the final pulp instance segmentation. Extensive experiments on CBCT scans from 147 patients demonstrate that TIPs significantly outperforms state-of-the-art methods in terms of segmentation accuracy. Furthermore, we have encapsulated this framework into an openly accessible tool for one-click using. To our knowledge, this is the first toolbox capable of segmentation of tooth and pulp instances, with its performance validated on two external datasets comprising 59 samples from the Toothfairy2 dataset and 48 samples from the STS dataset. These results demonstrate the potential of TIPs as a practical tool to boost clinical workflows in digital dentistry, enhancing the precision and efficiency of dental diagnostics and treatment planning.

Fetal Magnetic Resonance Imaging (MRI) is challenged by fetal movements and maternal breathing. Although fast MRI sequences allow artifact free acquisition of individual 2D slices, motion frequently occurs in the acquisition of spatially adjacent slices. Motion correction for each slice is thus critical for the reconstruction of 3D fetal brain MRI. In this paper, we propose a novel multi-task learning framework that adopts a coarse-to-fine strategy to jointly learn the pose estimation parameters for motion correction and tissue segmentation map of each slice in fetal MRI. Particularly, we design a regression-based segmentation loss as a deep supervision to learn anatomically more meaningful features for pose estimation and segmentation. In the coarse stage, a U-Net-like network learns the features shared for both tasks. In the refinement stage, to fully utilize the anatomical information, signed distance maps constructed from the coarse segmentation are introduced to guide the feature learning for both tasks. Finally, iterative incorporation of the signed distance maps further improves the performance of both regression and segmentation progressively. Experimental results of cross-validation across two different fetal datasets acquired with different scanners and imaging protocols demonstrate the effectiveness of the proposed method in reducing the pose estimation error and obtaining superior tissue segmentation results simultaneously, compared with state-of-the-art methods.

Accurate processing and analysis of non-human primate (NHP) brain magnetic resonance imaging (MRI) serves an indispensable role in understanding brain evolution, development, aging, and diseases. Despite the accumulation of diverse NHP brain MRI datasets at various developmental stages and from various imaging sites/scanners, existing computational tools designed for human MRI typically perform poor on NHP data, due to huge differences in brain sizes, morphologies, and imaging appearances across species, sites, and ages, highlighting the imperative for NHP-specialized MRI processing tools. To address this issue, in this paper, we present a robust, generic, and fully automated computational pipeline, called non-human primates Brain Extraction and Segmentation Toolbox (nBEST), whose main functionality includes brain extraction, non-cerebrum removal, and tissue segmentation. Building on cutting-edge deep learning techniques by employing lifelong learning to flexibly integrate data from diverse NHP populations and innovatively constructing 3D U-NeXt architecture, nBEST can well handle structural NHP brain MR images from multi-species, multi-site, and multi-developmental-stage (from neonates to the elderly). We extensively validated nBEST based on, to our knowledge, the largest assemblage dataset in NHP brain studies, encompassing 1,469 scans with 11 species (e.g., rhesus macaques, cynomolgus macaques, chimpanzees, marmosets, squirrel monkeys, etc.) from 23 independent datasets. Compared to alternative tools, nBEST outperforms in precision, applicability, robustness, comprehensiveness, and generalizability, greatly benefiting downstream longitudinal, cross-sectional, and cross-species quantitative analyses. We have made nBEST an open-source toolbox (https://github.com/TaoZhong11/nBEST) and we are committed to its continual refinement through lifelong learning with incoming data to greatly contribute to the research field.

Brain extraction and image quality assessment are two fundamental steps in fetal brain magnetic resonance imaging (MRI) 3D reconstruction and quantification. However, the randomness of fetal position and orientation, the variability of fetal brain morphology, maternal organs around the fetus, and the scarcity of data samples, all add excessive noise and impose a great challenge to automated brain extraction and quality assessment of fetal MRI slices. Conventionally, brain extraction and quality assessment are typically performed independently. However, both of them focus on the brain image representation, so they can be jointly optimized to ensure the network learns more effective features and avoid overfitting. To this end, we propose a novel two-stage dual-task deep learning framework with a brain localization stage and a dual-task stage for joint brain extraction and quality assessment of fetal MRI slices. Specifically, the dual-task module compactly contains a feature extraction module, a quality assessment head and a segmentation head with feature fusion for simultaneous brain extraction and quality assessment. Besides, a transformer architecture is introduced into the feature extraction module and the segmentation head. We utilize a multi-step training strategy to guarantee a stable and successful training of all modules. Finally, we validate our method by a 5-fold cross-validation and ablation study on a dataset with fetal brain MRI slices in different qualities, and perform a cross-dataset validation in addition. Experiments show that the proposed framework achieves very promising performance.

Magnetic Resonance Imaging (MRI) plays a pivotal role in the accurate measurement of brain subcortical structures in macaques, which is crucial for unraveling the complexities of brain structure and function, thereby enhancing our understanding of neurodegenerative diseases and brain development. However, due to significant differences in brain size, structure, and imaging characteristics between humans and macaques, computational tools developed for human neuroimaging studies often encounter obstacles when applied to macaques. In this context, we propose an Anatomy Attentional Fusion Network (AAF-Net), which integrates multimodal MRI data with anatomical constraints in a multi-scale framework to address the challenges posed by the dynamic development, regional heterogeneity, and age-related size variations of the juvenile macaque brain, thus achieving precise subcortical segmentation. Specifically, we generate a Signed Distance Map (SDM) based on the initial rough segmentation of the subcortical region by a network as an anatomical constraint, providing comprehensive information on positions, structures, and morphology. Then we construct AAF-Net to fully fuse the SDM anatomical constraints and multimodal images for refined segmentation. To thoroughly evaluate the performance of our proposed tool, over 700 macaque MRIs from 19 datasets were used in this study. Specifically, we employed two manually labeled longitudinal macaque datasets to develop the tool and complete four-fold cross-validations. Furthermore, we incorporated various external datasets to demonstrate the proposed tool’s generalization capabilities and promise in brain development research. We have made this tool available as an open-source resource at https://github.com/TaoZhong11/Macaque_subcortical_segmentation for direct application.

BACKGROUND:Accurate segmentation of individual tooth and their substructures including enamel, pulp, and dentin from cone-beam computed tomography (CBCT) images is essential for dental diagnosis and treatment planning in digital dentistry. Existing methods for tooth segmentation based on CBCT images have achieved substantial progress; however, techniques for further segmentation into substructures are yet to be developed. PURPOSE:We aim to propose a novel three-stage progressive deep-learning-based framework for automatically segmenting 3D tooth from CBCT images, focusing on finer substructures, i.e., enamel, pulp, and dentin. METHODS:In this paper, we first detect each tooth using its centroid by a clustering scheme, which efficiently determines each tooth detection by applying learned displacement vectors from the foreground tooth region. Next, guided by the detected centroid, each tooth proposal, combined with the corresponding tooth map, is processed through our tooth segmentation network. We also present an attention-based hybrid feature fusion mechanism, which provides intricate details of the tooth boundary while maintaining the global tooth shape, thereby enhancing the segmentation process. Additionally, we utilize the skeleton of the tooth as a guide for subsequent substructure segmentation. RESULTS:Our algorithm is extensively evaluated on a collected dataset of 314 patients, and the extensive comparison and ablation studies demonstrate superior segmentation results of our approach. CONCLUSIONS:Our proposed method can automatically segment tooth and finer substructures from CBCT images, underlining its potential applicability for clinical diagnosis and surgical treatment.

OBJECTIVE:To propose a Dual-Aware deep learning framework for genotyping of isocitrate dehydrogenase (IDH) in gliomas based on magnetic resonance amide proton transfer (APT) modality data as a means to assist non-invasive diagnosis of gliomas.METHODS:We collected multimodal magnetic resonance imaging (MRI) imaging data of the brain from 118 cases of gliomas, including 68 wild-type and 50 mutant type cases. The delineation of the ROI of brain glioma was completed in all the cases. APT modality imaging does not require contrast agents, and its signal intensity on tumors is positively correlated with tumor malignancy, and the signal intensity on wild-type IDH is higher than that on mutant IDH. For APT modalities, tumor imaging and derived areas are morphologically variable and lack prominent edge contour characteristics compared with other modalities. Based on these characteristics, we propose the Dual-Aware framework, which introduces the Multi-Aware framework to mine multi-scale features, and the Edge Aware module mines the edge features for automatic genotype identification.RESULTS:The introduction of two types of Aware mechanisms effectively improved the identification rate of the model for glioma IDH genotyping. The accuracy and AUC for each modality data were enhanced, and the best performance was achieved on the APT modality with a prediction accuracy of 83.1% and an AUC of 0.822, suggesting its advantages and effectiveness for identifying glioma IDH genotypes.CONCLUSION:The proposed deep learning algorithm model constructed based on the image characteristics of the APT modality is effective for glioma IDH genotyping and identification task and may potentially replace the commonly used T1CE modality to avoid contrast agent injection and achieve non- invasive IDH genotyping.

OBJECTIVE:We propose a novel deep learning algorithm based on attention mechanism and multimodality feature fusion (DDAM) to achieve whole-brain parcellation of macaque brain magnetic resonance (MR) images. METHODS:We collected multimodality brain MR images of 68 macaques (aged 13 to 36 months) with corresponding ground truth labels. To address the complexity and complementary nature of the multimodality data, we employed a multi- encoder structure to adapt to different modalities and performed feature extraction. In the decoder, an attention mechanism was introduced to construct the Attention-based Multimodality Feature Fusion module (AMFF). Leveraging the rich and complementary information between modalities, AMFF effectively integrated multimodality features of varying scales and complexities to enhance the performance of parcellation. We conducted ablation experiments and statistically analyzed the results. RESULTS:The incorporation of the multi- encoder structure and attention mechanism significantly improved the performance of the network for integrating the multimodality features, and achieved an average DSC of 0.904 and an ASD as low as 0.131 for macaque whole-brain parcellation (P < 0.05). The ablation experiments validated the effectiveness of each component of the DDAM method. CONCLUSION:The proposed DDAM method, with its enhanced ability to extract and integrate multimodality features, can significantly improve the accuracy of whole-brain MR image segmentation.

In neuroscience research, automatic segmentation of macaque brain tissues in magnetic resonance imaging (MRI) is crucial for understanding brain structure and function during development and evolution. Acquisition of multimodal information is a key enabler of accurate tissue segmentation, especially in early-developing macaques with extremely low contrast and dynamic myelination. However, many MRI scans of early-developing macaques are acquired only in a single modality. While various generative adversarial networks (GAN) have been developed to impute missing modality data, current solutions treat modality generation and image segmentation as two independent tasks, neglecting their inherent relationship and mutual benefits. To address these issues, this study proposes a novel Collaborative Segmentation-Generation Framework (CSGF) that enables joint missing modality generation and tissue segmentation of macaque brain MR images. Specifically, the CSGF consists of a modality generation module (MGM) and a tissue segmentation module (TSM) that are trained jointly by a cross-module feature sharing (CFS) and transferring generated modality. The training of the MGM under the supervision of the TSM enforces anatomical feature consistency, while the TSM learns multi-modality information related to anatomical structures from both real and synthetic multi-modality MR images. Experiments show that the CSGF outperforms the conventional independent-task mode on an early-developing macaque MRI dataset with 155 scans, achieving superior quality in both missing modality generation and tissue segmentation.

Objective. Accurate segmentation of the pancreas from abdomen CT scans is highly desired for diagnosis and treatment follow-up of pancreatic diseases. However, the task is challenged by large anatomical variations, low soft-tissue contrast, and the difficulty in acquiring a large set of annotated volumetric images for training. To overcome these problems, we propose a new segmentation network and a semi-supervised learning framework to alleviate the lack of annotated images and improve the accuracy of segmentation. Approach. In this paper, we propose a novel graph-enhanced pancreas segmentation network (GEPS-Net), and incorporate it into a semi-supervised learning framework based on iterative uncertainty-guided pseudo-label refinement. Our GEPS-Net plugs a graph enhancement module on top of the CNN-based U-Net to focus on the spatial relationship information. For semi-supervised learning, we introduce an iterative uncertainty-guided refinement process to update pseudo labels by removing low-quality and incorrect regions. Main results. Our method was evaluated by a public dataset with four-fold cross-validation and achieved the DC of 84.22%, improving 5.78% compared to the baseline. Further, the overall performance of our proposed method was the best compared with other semi-supervised methods trained with only 6 or 12 labeled volumes. Significance. The proposed method improved the segmentation performance of the pancreas in CT images under the semi-supervised setting. It will assist doctors in early screening and making accurate diagnoses as well as adaptive radiotherapy.

Spatiotemporal (four-dimensional) infant-dedicated brain atlases are essential for neuroimaging analysis of early dynamic brain development. However, due to the substantial technical challenges in the acquisition and processing of infant brain MR images, 4D atlases densely covering the dynamic brain development during infancy are still scarce. Few existing ones generally have fuzzy tissue contrast and low spatiotemporal resolution, leading to degraded accuracy of atlas-based normalization and subsequent analyses. To address this issue, in this paper, we construct a 4D structural MRI atlas for infant brains based on the UNC/UMN Baby Connectome Project (BCP) dataset, which features a high spatial resolution, extensive age-range coverage, and densely sampled time points. Specifically, 542 longitudinal T1w and T2w scans from 240 typically developing infants up to 26-month of age were utilized for our atlas construction. To improve the co-registration accuracy of the infant brain images, which typically exhibit dynamic appearance with low tissue contrast, we employed the state-of-the-art registration method and leveraged our generated reliable brain tissue probability maps in addition to the intensity images to improve the alignment of individual images. To achieve consistent region labeling on both infant and adult brain images for facilitating region-based analysis across ages, we mapped the widely used Desikan cortical parcellation onto our atlas by following an age-decreasing mapping manner. Meanwhile, the typical subcortical structures were manually delineated to facilitate the studies related to the subcortex. Compared with the existing infant brain atlases, our 4D atlas has much higher spatiotemporal resolution and preserves more structural details, and thus can boost accuracy in neurodevelopmental analysis during infancy.

Brain atlases are of fundamental importance for analyzing the dynamic neurodevelopment in fetal brain studies. Since the brain size, shape, and anatomical structures change rapidly during the prenatal period, it is essential to construct a spatiotemporal (4D) atlas equipped with tissue probability maps, which can preserve sharper early brain folding patterns for accurately characterizing dynamic changes in fetal brains and provide tissue prior informations for related tasks, e.g., segmentation, registration, and parcellation. In this work, we propose a novel unsupervised age-conditional learning framework to build temporally continuous fetal brain atlases by incorporating tissue segmentation maps, which outperforms previous traditional atlas construction methods in three aspects. First, our framework enables learning age-conditional deformable templates by leveraging the entire collection. Second, we leverage reliable brain tissue segmentation maps in addition to the low-contrast noisy intensity images to enhance the alignment of individual images. Third, a novel loss function is designed to enforce the similarity between the learned tissue probability map on the atlas and each subject tissue segmentation map after registration, thereby providing extra anatomical consistency supervision for atlas building. Our 4D temporally-continuous fetal brain atlases are constructed based on 82 healthy fetuses from 22 to 32 gestational weeks. Compared with the atlases built by the state-of-the-art algorithms, our atlases preserve more structural details and sharper folding patterns. Together with the learned tissue probability maps, our 4D fetal atlases provide a valuable reference for spatial normalization and analysis of fetal brain development.

To better understand early brain development in health and disorder, it is critical to accurately segment infant brain magnetic resonance (MR) images into white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). Deep learning-based methods have achieved state-of-the-art performance; h owever, one of the major limitations is that the learning-based methods may suffer from the multi-site issue, that is, the models trained on a dataset from one site may not be applicable to the datasets acquired from other sites with different imaging protocols/scanners. To promote methodological development in the community, the iSeg-2019 challenge (http://iseg2019.web.unc.edu) provides a set of 6-month infant subjects from multiple sites with different protocols/scanners for the participating methods. T raining/validation subjects are from UNC (MAP) and testing subjects are from UNC/UMN (BCP), Stanford University, and Emory University. By the time of writing, there are 30 automatic segmentation methods participated in the iSeg-2019. In this article, 8 top-ranked methods were reviewed by detailing their pipelines/implementations, presenting experimental results, and evaluating performance across different sites in terms of whole brain, regions of interest, and gyral landmark curves. We further pointed out their limitations and possible directions for addressing the multi-site issue. We find that multi-site consistency is still an open issue. We hope that the multi-site dataset in the iSeg-2019 and this review article will attract more researchers to address the challenging and critical multi-site issue in practice.

We propose an algorithm for registration between brain tumor images and normal brain images based on tissue recovery. U-Net is first used in BraTS2018 dataset to segment the brain tumors, and PConv-Net is then used to simulate the generation of missing normal tissues in the tumor region to replace the tumor region. Finally, the normal brain image is registered to the tissue recovery brain image. We evaluated the effectiveness of this method by comparing the registration results of the repaired image and the tumor image corresponding to the surrounding tissues of the tumor area. The experimental results showed that the proposed method could reduce the effect of pathological variation, achieve a high registration accuracy, and effectively simulate and generate normal tissues to replace the tumor regions, thus improving the registration effect between brain tumor images and normal brain images.

Longitudinal brain imaging atlases with densely sampled time-points and ancillary anatomical information are of fundamental importance in studying early developmental characteristics of human and non-human primate brains during infancy, which feature extremely dynamic imaging appearance, brain shape and size. However, for non-human primates, which are highly valuable animal models for understanding human brains, the existing brain atlases are mainly developed based on adults or adolescents, denoting a notable lack of temporally densely-sampled atlases covering the dynamic early brain development. To fill this critical gap, in this paper, we construct a comprehensive set of longitudinal brain atlases and associated tissue probability maps (gray matter, white matter, and cerebrospinal fluid) with totally 12 time-points from birth to 4 years of age (i.e., 1, 2, 3, 4, 5, 6, 9, 12, 18, 24, 36, and 48 months of age) based on 175 longitudinal structural MRI scans from 39 typically-developing cynomolgus macaques, by leveraging state-of-the-art computational techniques tailored for early developing brains. Furthermore, to facilitate region-based analysis using our atlases, we also provide two popular hierarchy parcellations, i.e., cortical hierarchy maps (6 levels) and subcortical hierarchy maps (6 levels), on our longitudinal macaque brain atlases. These early developing atlases, which have the densest time-points during infancy (to the best of our knowledge), will greatly facilitate the studies of macaque brain development.

Quality assessment (QA) and brain extraction (BE) are two fundamental steps in 3D fetal brain MRI reconstruction and quantification. Conventionally, QA and BE are performed independently, ignoring the inherent relation of the two closely-related tasks. However, both of them focus on the brain region representation, so they can be jointly optimized to ensure the network to learn shared features and avoid overfitting. To this end, we propose a novel multi-stage deep learning model for joint QA and BE of fetal MRI. The locations and orientations of fetal brains are randomly variable, and the shapes and appearances of fetal brains change remarkably across gestational ages, thus imposing great challenges to extract shared features of QA and BE. To address these problems, we firstly design a brain detector to locate the brain region. Then we introduce the deformable convolution to adaptively adjust the receptive field for dealing with variable brain shapes. Finally, a task-specific module is used for image QA and BE simultaneously. To obtain a well-trained model, we further propose a multi-step training strategy. We cross validate our method on two independent fetal MRI datasets acquired from different scanners with different imaging protocols, and achieve promising performance.

Fetal Magnetic Resonance Imaging (MRI) is challenged by the fetal movements and maternal breathing. Although fast MRI sequences allow artifact free acquisition of individual 2D slices, motion commonly occurs in between slices acquisitions. Motion correction for each slice is thus very important for reconstruction of 3D fetal brain MRI, but is highly operator-dependent and time-consuming. Approaches based on convolutional neural networks (CNNs) have achieved encouraging performance on prediction of 3D motion parameters of arbitrarily oriented 2D slices, which, however, does not capitalize on important brain structural information. To address this problem, we propose a new multi-task learning framework to jointly learn the transformation parameters and tissue segmentation map of each slice, for providing brain anatomical information to guide the mapping from 2D slices to 3D volumetric space in a coarse to fine manner. In the coarse stage, the first network learns the features shared for both regression and segmentation tasks. In the refinement stage, to fully utilize the anatomical information, distance maps constructed based on the coarse segmentation are introduced to the second network. Finally, incorporation of the signed distance maps to guide the regression and segmentation together improves the performance in both tasks. Experimental results indicate that the proposed method achieves superior performance in reducing the motion prediction error and obtaining satisfactory tissue segmentation results simultaneously, compared with state-of-the-art methods.

Predicting future progression of brain disorders is fundamental for effective intervention of pathological cognitive decline. Structural MRI provides a non-invasive solution to examine brain pathology and has been widely used for longitudinal analysis of brain disorders. Previous studies typically use only complete baseline MRI scans to predict future disease status due to the lack of MRI data at one or more future time points. Since temporal changes of each brain MRI are ignored, these methods would result in sub-optimal performance. To this end, we propose a longitudinal-diagnostic generative adversarial network (LDGAN) to predict multiple clinical scores at future time points using incomplete longitudinal MRI data. Specifically, LDGAN imputes MR images by learning a bi-directional mapping between MRIs of two adjacent time points and performing clinical score prediction jointly, thereby explicitly encouraging task-oriented image synthesis. The proposed LDGAN is further armed with a temporal constraint and an output constraint to model the temporal regularity of MRIs at adjacent time points and encourage the diagnostic consistency, respectively. We also design a weighted loss function to make use of those subjects without ground-truth scores at certain time points. The major advantage of the proposed LDGAN is that it can impute those missing scans in a task-oriented manner and can explicitly capture the temporal characteristics of brain changes for accurate prediction. Experimental results on both ADNI-1 and ADNI-2 datasets demonstrate that, compared with the state-of-the-art methods, LDGAN can generate more reasonable MRI scans and efficiently predict longitudinal clinical measures.

Non-human primates, especially macaque monkeys, with close phylogenetic relationship to humans, are highly valuable and widely used animal models for human neuroscience studies. In neuroimaging analysis of macaques, brain extraction or skull stripping of magnetic resonance imaging (MRI) is a crucial step for following processing. However, the current skull stripping methods largely focus on human brains, and thus often lead to unsatisfactory results when applying to macaque brains, especially for macaque brains during early development. In fact, the macaque brain during infancy undergoes regionally-heterogeneous dynamic development, leading to poor and age-variable contrasts between different anatomical structures, posing great challenges for accurate skull stripping. In this study, we propose a novel framework to effectively combine intensity information and domain-invariant prior knowledge, which are important guidance information for accurate brain extraction of developing macaques from 0 to 36 months of age. Specifically, we introduce signed distance map (SDM) and center of gravity distance map (CGDM) based on the intermediate segmentation results and fuse their information by Dual Self-Attention Module (DSAM) instead of local convolution. To evaluate the performance, we adopt two large-scale and challenging MRI datasets from rhesus macaques and cynomolgus macaques, respectively, with totally 361 scans from two different scanners with different imaging protocols. We perform cross-validation by using one dataset for training and the other one for testing. Experimental results show the robustness of our plug-and-play method on cross-source MRI datasets without any transfer learning.