Hierarchical Vision Transformers (ViTs) have achieved significant success in medical image segmentation due to their large receptive field and ability to leverage long-range contextual information. Convolutional neural networks (CNNs) may also deliver a large receptive field by using large convolutional kernels. However, because they use fixed-sized kernels, CNNs with large kernels remain limited in their ability to adaptively capture multi-scale features from organs that vary greatly in shape and size. They are also unable to utilize global contextual information efficiently. To address these limitations, we propose lightweight Dynamic Large Kernel (DLK) and Dynamic Feature Fusion (DFF) modules. The DLK employs multiple large kernels with varying kernel sizes and dilation rates to capture multi-scale features. Subsequently, DLK utilizes a dynamic selection mechanism to adaptively highlight the most important channel and spatial features based on global information. The DFF is proposed to adaptively fuse multi-scale local feature maps based on their global information. We incorporated DLK and DFF into a hierarchical ViT architecture to leverage their scaling behavior, but they struggle to extract low-level features effectively due to feature embedding constraints in ViT architectures. To tackle this limitation, we propose a Salience layer to extract low-level features from images at their original dimensions without feature embedding. This Salience layer employs a Channel Mixer to capture global representations effectively. We further incorporated the Salience layer into the hierarchical ViT architecture to develop a novel network, termed D-Net. D-Net effectively utilizes a multi-scale large receptive field and adaptively harnesses global contextual information. Extensive experimental results demonstrate its superior segmentation performance compared to state-of-the-art models, with comparably lower computational complexity. The code is made available at https://github.com/sotiraslab/DLK.

Deep learning models for neuroimaging have largely been developed for individual tasks, limiting knowledge transfer across applications. Here we introduce GenFAR, a modular deep learning framework that learns general, clinically informed features from brain MRIs. We trained this modular architecture on 49,246 individuals across 11 cohorts, using 17 diverse classification and regression tasks spanning cognition, clinical, diagnosis, demographics, and biomarkers. This yields aggregated, focused feature sets that capture rich, clinically- and biologically-relevant brain representations. We developed a sequential learning approach where tasks progressively build on previously learned representations. Through an analysis of 5,000 task sequences, we identified an optimal sequence length of six tasks and introduced a Donor Score metric to quantify each task's contribution to downstream performance. This analysis revealed five consistently strong donor tasks (Age, AD/MCI, MMSE, Hypertension, Hyperlipidemia) that formed the base of our sequential model. We demonstrated the utility of our learned representation, in various tasks beyond those included in the training set, to serve as the foundation for specialized secondary predictors. We further showed that using the learned feature representation can substantially increase the sample efficiency of secondary deep learning training tasks and models, as well as improve their accuracy.

Background Radiology reports often contain follow-up recommendations vital for optimal patient care, prevention of complications, and mitigation of legal risk. However, there is a lack of comprehensive comparison methods for identifying these recommendations across a large volume of reports from various modalities, including open-source large language models. Purpose To evaluate the performance of machine learning (ML) models, including Meta's open-source LLAMA3 and OpenAI's Health Insurance Portability and Accountability Act-compliant Generative Pre-trained Transformer, in identifying follow-up recommendations in radiology reports. Materials and Methods In this retrospective study, three sets of radiology reports were analyzed across multiple imaging modalities from a large urban academic medical center: an expert annotated dataset (n = 11 901) from January 1 to January 10, 2015; a dataset (n = 32 959) extracted through regular expressions (ie, sequences of characters that define search patterns in text) from January 11, 2015, to January 1, 2017; and a dataset (n = 4909) annotated during dictation from September 8, 2018, to February 23, 2021. To assess generalization on impressions, two expertly annotated datasets were used: 2000 chest radiography reports from the publicly available MIMIC-CXR database for external testing and 100 institutional CT reports from January 1 to January 15, 2024, for temporal testing. Thirty-two text classification methods were evaluated separately based on the findings and impression sections of these reports. Performance metrics included precision, recall, accuracy, and F1 score; with 95% bootstrapped CIs and areas under the precision-recall curve. Statistical comparisons were performed by using the McNemar test. Results The study included 49 769 reports from 35 509 patients (mean age, 52.2 years ± 22.0 [SD]; 18 477 female patients) for training (n = 37 140), validation (n = 2584), and internal testing (n = 10 045). For the findings section, a generative-discriminative model initialized with Google's Word2vec embeddings (Hybrid-google) achieved the highest F1 score (0.835; 95% CI: 0.825, 0.845). For the impression section, an attention-based bidirectional long short-term memory (LSTM) with random initialization (AttBiLSTM-random) performed best, with an F1 score of 0.979 (95% CI: 0.976, 0.982). Prefixed prompting with GPT-4 demonstrated superior external and temporal generalization performance on the MIMIC-CXR and institutional CT datasets, achieving F1 scores of 0.969 (95% CI: 0.961, 0.977) and 0.973 (95% CI: 0.937, 1.000), respectively. Conclusion ML models showed promise for automating the classification of follow-up recommendations in radiology reports. © RSNA, 2025 Supplemental material is available for this article.

Despite continuous advancements in cancer treatment, brain metastatic disease remains a significant complication of primary cancer and is associated with an unfavorable prognosis. One approach for improving diagnosis, management, and outcomes is to implement algorithms based on artificial intelligence for the automated segmentation of both pre- and post-treatment MRI brain images. Such algorithms rely on volumetric criteria for lesion identification and treatment response assessment, which are still not available in clinical practice. Therefore, it is critical to establish tools for rapid volumetric segmentations methods that can be translated to clinical practice and that are trained on high quality annotated data. The BraTS-METS 2025 Lighthouse Challenge aims to address this critical need by establishing inter-rater and intra-rater variability in dataset annotation by generating high quality annotated datasets from four individual instances of segmentation by neuroradiologists while being recorded on video (two instances doing "from scratch" and two instances after AI pre-segmentation). This high-quality annotated dataset will be used for testing phase in 2025 Lighthouse challenge and will be publicly released at the completion of the challenge. The 2025 Lighthouse challenge will also release the 2023 and 2024 segmented datasets that were annotated using an established pipeline of pre-segmentation, student annotation, two neuroradiologists checking, and one neuroradiologist finalizing the process. It builds upon its previous edition by including post-treatment cases in the dataset. Using these high-quality annotated datasets, the 2025 Lighthouse challenge plans to test benchmark algorithms for automated segmentation of pre-and post-treatment brain metastases (BM), trained on diverse and multi-institutional datasets of MRI images obtained from patients with brain metastases.

Glioblastoma is the most aggressive adult brain tumor, with a median overall survival of approximately 15 months. It is important to build accurate prognostic models for glioblastoma patients to inform clinical management and trials. This study proposes a self-supervised learning-based approach with multimodal data integration for survival prediction and prognostic stratification of glioblastoma patients on the ReSPOND consortium. We curated a multi-parametric MRI dataset (T1, T1CE, T2, FLAIR) of 3,119 glioblastoma patients from 22 institutions across 3 continents. Masked autoencoder (MAE) was adapted to pretrain a Vision Transformer (ViT) encoder by reconstructing the masked image patches. The encoder was utilized for extracting patch embeddings for survival tasks, with cross-attention mechanism to incorporate the molecular and clinical information (age, sex, extent of resection, MGMT) to guide imaging feature aggregation. Imaging and clinical embeddings were fused through a multi-layer perceptron (MLP) for log-risk hazard estimation, optimized using Cox partial likelihood. Model performance and generalizability were assessed via k-fold cross-validation on the ReSPOND consortium and the leave-one-site-out validation was performed on 11 institutions comparing with CoxPH, DeepSurv and DeepHit. Prognostic risk stratification via Kaplan-Meier analysis divided the patients into low-, medium- and high-risk subgroups per site. Multimodal data integration using the proposed framework achieved the highest C-index (0.674 ± 0.017) on the ReSPOND consortium. Integration of clinical information and MGMT consistently boosted the performance of the proposed model across sites (0.615 ± 0.046 vs. 0.662 ± 0.044). The imaging-based approaches, i.e., radiomics and convolutional neural network (CNN) features performed less robustly. The Kaplan-Meier curves and log-rank tests suggested the proposed framework achieved more separable prognostic subgroups. The proposed self-supervised multimodal learning framework shows promise for survival prediction and prognostic risk stratification in glioblastoma. It highlights the challenge for clinical model deployment due to the data heterogeneity in multi-institutional cohort.

Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse multicentric patient data. To this end, the Federated Tumor Segmentation (FeTS) Challenge represents the paradigm for real-world algorithmic performance evaluation. The FeTS challenge is a competition to benchmark (i) federated learning aggregation algorithms and (ii) state-of-the-art segmentation algorithms, across multiple international sites. Weight aggregation and client selection techniques were compared using a multicentric brain tumor dataset in realistic federated learning simulations, yielding benefits for adaptive weight aggregation, and efficiency gains through client sampling. Quantitative performance evaluation of state-of-the-art segmentation algorithms on data distributed internationally across 32 institutions yielded good generalization on average, albeit the worst-case performance revealed data-specific modes of failure. Similar multi-site setups can help validate the real-world utility of healthcare AI algorithms in the future.

Medical Vision Foundation Models (Med-VFMs) have superior capabilities of interpreting medical images due to the knowledge learned from self-supervised pre-training with extensive unannotated images. To improve their performance on adaptive downstream evaluations, especially segmentation, a few samples from target domains are selected randomly for fine-tuning them. However, there lacks works to explore the way of adapting Med-VFMs to achieve the optimal performance on target domains efficiently. Thus, it is highly demanded to design an efficient way of fine-tuning Med-VFMs by selecting informative samples to maximize their adaptation performance on target domains. To achieve this, we propose an Active Source-Free Domain Adaptation (ASFDA) method to efficiently adapt Med-VFMs to target domains for volumetric medical image segmentation. This ASFDA employs a novel Active Learning (AL) method to select the most informative samples from target domains for fine-tuning Med-VFMs without the access to source pre-training samples, thus maximizing their performance with the minimal selection budget. In this AL method, we design an Active Test Time Sample Query strategy to select samples from the target domains via two query metrics, including Diversified Knowledge Divergence (DKD) and Anatomical Segmentation Difficulty (ASD). DKD is designed to measure the source-target knowledge gap and intra-domain diversity. It utilizes the knowledge of pre-training to guide the querying of source-dissimilar and semantic-diverse samples from the target domains. ASD is designed to evaluate the difficulty in segmentation of anatomical structures by measuring predictive entropy from foreground regions adaptively. Additionally, our ASFDA method employs a Selective Semi-supervised Fine-tuning to improve the performance and efficiency of fine-tuning by identifying samples with high reliability from unqueried ones.

Background and Objective: Convolutional neural networks (CNNs) have shown great effectiveness in medical image segmentation. However, they may be limited in modeling large inter-subject variations in organ shapes and sizes and exploiting global long-range contextual information. This is because CNNs typically employ convolutions with fixed-sized local receptive fields and lack the mechanisms to utilize global information. Methods: To address the above limitations, we developed Dynamic Multi-Resolution Convolution (DMRC) and Dynamic Multi-Scale Convolution (DMSC) modules. Both modules enhance the representation capabilities of single convolutions to capture varying scaled features and global contextual information. This is achieved in the DMRC module by employing a convolutional filter on images with different resolutions and subsequently utilizing dynamic mechanisms to model global inter-dependencies between features. In contrast, the DMSC module extracts features at different scales by employing convolutions with different kernel sizes and utilizing dynamic mechanisms to extract global contextual information. The utilization of convolutions with different kernel sizes in the DMSC module may increase computational complexity. To lessen this burden, we propose to use a lightweight design for convolution layers with a large kernel size. Thus, DMSC and DMRC modules are designed as lightweight drop-in replacements for single convolutions, and they can be easily integrated into general CNN architectures for end-to-end training. The segmentation network was proposed by incorporating our DMSC and DMRC modules into a standard U-Net architecture, termed Dynamic Multi-scale and Multi-resolution Convolution network (DMC-Net). Results: To evaluate the effectiveness of DMSC and DMRC modules, we conducted experiments on pancreas segmentation from abdominal computed tomography (CT) images on two commonly used benchmarks, including NIH-Pancreas and MSD-Pancreas datasets, with 2D and 3D versions of the DMC-Net. 2D DMC-Net achieved 85.64 and 79.82 Mean Dice Similarity Coefficient (DSC) scores in the NIH-Pancreas and MSD-Pancreas datasets, respectively. Additionally, 3D DMC-Net achieved 87.97 and 82.92 Mean DSC scores in these two datasets. The DMC-Net outperformed the state-of-the-art methods on pancreas segmentation in CT images. Conclusions: Our proposed DMSC and DMRC can enhance the representation capabilities of single convolutions and improve segmentation accuracy. Furthermore, their lightweight design led to lower computational complexity while maintaining or improving segmentation performance.

Comorbid cardiovascular and metabolic risk factors (CVM) differentially impact brain structure and increase dementia risk, but their specific magnetic resonance imaging signatures (MRI) remain poorly characterized. To address this, we developed and validated machine learning models to quantify the distinct spatial patterns of atrophy and white matter hyperintensities related to hypertension, hyperlipidemia, smoking, obesity, and type-2 diabetes mellitus at the patient level. Using harmonized MRI data from 37,096 participants (45-85 years) in a large multinational dataset of 10 cohort studies, we generated five in silico severity markers that: i) outperformed conventional structural MRI markers with a ten-fold increase in effect sizes, ii) captured subtle patterns at sub-clinical CVM stages, iii) were most sensitive in mid-life (45-64 years), iv) were associated with brain beta-amyloid status, and v) showed stronger associations with cognitive performance than diagnostic CVM status. Integrating personalized measurements of CVM-specific brain signatures into phenotypic frameworks could guide early risk detection and stratification in clinical studies.

Assessment of raw data quality metrics for optical brain mapping data allows for reproducibility and interpretability of results. Docker containers facilitate reproductible, standardized workflows with tunable parameters for broad system compatibility.

Convolutional neural networks (CNNs) and vision transformers (ViTs) are widely employed for medical image segmentation, but they are still challenged by their intrinsic characteristics. CNNs are limited from capturing varying-scaled features and global contextual information due to the employment of fixed-sized kernels. In contrast, ViTs employ self-attention and MLP for global information modeling, but they lack mechanisms to learn spatial-wise local information. Additionally, self-attention leads the network to show high computational complexity. To tackle these limitations, we propose Progressively Entangled Transformer Large Kernel (PTLK) and Collaboratively Entangled Transformer Large Kernel (CTLK) modules to leverage the benefits of self-attention and large kernel convolutions and overcome shortcomings. Specifically, PTLK and CTLK modules employ the Multi-head Large Kernel to capture multi-scale local features and the Efficient Decomposed Self-attention to model global information efficiently. Subsequently, they employ the Attention Entanglement mechanism to enable local and global features to enhance and calibrate each other progressively and collaboratively. Additionally, an Attention-gated Channel MLP (AG-MLP) module is proposed to equip the standard MLP module with the capabilities of modeling spatial information. PTLK and CTLK modules are further incorporated as a Cross Entanglement Decoding (CED) block for efficient feature fusion and decoding. Finally, we propose a novel network for volumetric medical image segmentation that employs an encoder-decoder architecture, termed TransLK-Net. The encoder employs a hierarchical ViT architecture whose block is built by incorporating PTLK and CTLK with AG-MLP into a ViT block, and the decoder employs the CED block.

U-Net has been widely used for segmenting abdominal organs, achieving promising performance. However, when it is used for multi-organ segmentation, first, it may be limited in exploiting global long-range contextual information due to the implementation of standard convolutions. Second, the use of spatial-wise downsampling (e.g., max pooling or strided convolutions) in the encoding path may lead to the loss of deformable or discriminative details. Third, features upsampled from the higher level are concatenated with those that persevered via skip connections. However, repeated downsampling and upsampling operations lead to misalignments between them and their concatenation degrades segmentation performance. To address these limitations, we propose Dynamically Calibrated Convolution (DCC), Dynamically Calibrated Downsampling (DCD), and Dynamically Calibrated Upsampling (DCU) modules, respectively. The DCC module can utilize global inter-dependencies between spatial and channel features to calibrate these features adaptively. The DCD module enables networks to adaptively preserve deformable or discriminative features during downsampling. The DCU module can dynamically align and calibrate upsampled features to eliminate misalignments before concatenations. We integrated the proposed modules into a standard U-Net, resulting in a new architecture, termed Dynamic U-Net. This architectural design enables U-Net to dynamically adjust features for different organs. We evaluated Dynamic U-Net in two abdominal multi-organ segmentation benchmarks. Dynamic U-Net achieved statistically improved segmentation accuracy compared with standard U-Net. Our code is available at https://github.com/sotiraslab/DynamicUNet.

BACKGROUND:Glioblastoma (GBM) is the most aggressive adult primary brain cancer, characterized by significant heterogeneity, posing challenges for patient management, treatment planning, and clinical trial stratification. METHODS:We developed a highly reproducible, personalized prognostication, and clinical subgrouping system using machine learning (ML) on routine clinical data, magnetic resonance imaging (MRI), and molecular measures from 2838 demographically diverse patients across 22 institutions and 3 continents. Patients were stratified into favorable, intermediate, and poor prognostic subgroups (I, II, and III) using Kaplan-Meier analysis (Cox proportional model and hazard ratios [HR]). RESULTS:The ML model stratified patients into distinct prognostic subgroups with HRs between subgroups I-II and I-III of 1.62 (95% CI: 1.43-1.84, P < .001) and 3.48 (95% CI: 2.94-4.11, P < .001), respectively. Analysis of imaging features revealed several tumor properties contributing unique prognostic value, supporting the feasibility of a generalizable prognostic classification system in a diverse cohort. CONCLUSIONS:Our ML model demonstrates extensive reproducibility and online accessibility, utilizing routine imaging data rather than complex imaging protocols. This platform offers a unique approach to personalized patient management and clinical trial stratification in GBM.

Deep learning (DL) methods have achieved great success in medical image segmentation, but they are challenged to demonstrate robust performance across different datasets due to domain and modality gaps. The Source-Free Domain Adaptation techniques adapt DL models to generalize across domains without access to source data, and active learning is implemented to actively query informative target samples to fine-tune models, thus improving their generalization. However, only a few Active Source-Free Domain Adaptation methods have been proposed. Additionally, existing methods focus on same-modality adaptation and lack mechanisms to address modality gaps, thus limiting their applicability. To address these limitations, we propose a novel Active Source-Free Cross-Domain and Cross-Modality Adaptation method for medical image segmentation. This method adapts models across different domains and modalities by employing a novel Active Test Time Sample Query strategy to jointly implement Image Sensitivity Query (ISQ) and Organ Heterogeneity Query (OHQ). ISQ is designed to evaluate samples' imagelevel modality agnostic informativeness, thus querying informative samples from different domains and modalities. OHQ is proposed to query samples with large foreground diversity by measuring the uncertaintyweighted organ boundary discontinuity and uncertainty-weighted organ interior abnormality, thus avoiding the influence of modality-specific background noise. A Dynamic Image-to-Organ Scaling mechanism is proposed to dynamically fuse the results of ISQ and OHQ for sample querying. We evaluated our method on cross-domain and cross-modality volumetric pancreas segmentation tasks. Our method outperformed other state-of-the-art methods on adaptation from a CT domain to another larger CT domain, T1-weighted MR and T2-weighted MR domains.

Alzheimer’s disease (AD) is associated with heterogeneous atrophy patterns. We employed a semi-supervised representation learning technique known as Surreal-GAN, through which we identified two latent dimensional representations of brain atrophy in symptomatic mild cognitive impairment (MCI) and AD patients: the “diffuse-AD” (R1) dimension shows widespread brain atrophy, and the “MTL-AD” (R2) dimension displays focal medial temporal lobe (MTL) atrophy. Critically, only R2 was associated with widely known sporadic AD genetic risk factors (e.g., APOE ε4) in MCI and AD patients at baseline. We then independently detected the presence of the two dimensions in the early stages by deploying the trained model in the general population and two cognitively unimpaired cohorts of asymptomatic participants. In the general population, genome-wide association studies found 77 genes unrelated to APOE differentially associated with R1 and R2. Functional analyses revealed that these genes were overrepresented in differentially expressed gene sets in organs beyond the brain (R1 and R2), including the heart (R1) and the pituitary gland, muscle, and kidney (R2). These genes were enriched in biological pathways implicated in dendritic cells (R2), macrophage functions (R1), and cancer (R1 and R2). Several of them were “druggable genes” for cancer (R1), inflammation (R1), cardiovascular diseases (R1), and diseases of the nervous system (R2). The longitudinal progression showed that APOE ε4, amyloid, and tau were associated with R2 at early asymptomatic stages, but this longitudinal association occurs only at late symptomatic stages in R1. Our findings deepen our understanding of the multifaceted pathogenesis of AD beyond the brain. In early asymptomatic stages, the two dimensions are associated with diverse pathological mechanisms, including cardiovascular diseases, inflammation, and hormonal dysfunction—driven by genes different from APOE—which may collectively contribute to the early pathogenesis of AD. All results are publicly available at https://labs-laboratory.com/medicine/ .

BACKGROUND:Brain ageing is highly heterogeneous, as it is driven by a variety of normal and neuropathological processes. These processes may differentially affect structural and functional brain ageing across individuals, with more pronounced ageing (older brain age) during midlife being indicative of later development of dementia. Here, we examined whether brain-ageing heterogeneity in unimpaired older adults related to neurodegeneration, different cognitive trajectories, genetic and amyloid-beta (Aβ) profiles, and to predicted progression to Alzheimer's disease (AD). METHODS:Functional and structural brain age measures were obtained for resting-state functional MRI and structural MRI, respectively, in 3460 cognitively normal individuals across an age range spanning 42-85 years. Participants were categorised into four groups based on the difference between their chronological and predicted age in each modality: advanced age in both (n = 291), resilient in both (n = 260) or advanced in one/resilient in the other (n = 163/153). With the resilient group as the reference, brain-age groups were compared across neuroimaging features of neuropathology (white matter hyperintensity volume, neuronal loss measured with Neurite Orientation Dispersion and Density Imaging, AD-specific atrophy patterns measured with the Spatial Patterns of Abnormality for Recognition of Early Alzheimer's Disease index, amyloid burden using amyloid positron emission tomography (PET), progression to mild cognitive impairment and baseline and longitudinal cognitive measures (trail making task, mini mental state examination, digit symbol substitution task). FINDINGS:Individuals with advanced structural and functional brain-ages had more features indicative of neurodegeneration and they had poor cognition. Individuals with a resilient brain-age in both modalities had a genetic variant that has been shown to be associated with age of onset of AD. Mixed brain-age was associated with selective cognitive deficits. INTERPRETATION:The advanced group displayed evidence of increased atrophy across all neuroimaging features that was not found in either of the mixed groups. This is in line with biomarkers of preclinical AD and cerebrovascular disease. These findings suggest that the variation in structural and functional brain ageing across individuals reflects the degree of underlying neuropathological processes and may indicate the propensity to develop dementia in later life. FUNDING:The National Institute on Aging, the National Institutes of Health, the Swiss National Science Foundation, the Kaiser Foundation Research Institute and the National Heart, Lung, and Blood Institute.

Background Visual assessment of amyloid PET scans relies on the availability of radiologist expertise, whereas quantification of amyloid burden typically involves MRI for processing and analysis, which can be computationally expensive. Purpose To develop a deep learning model to classify minimally processed brain PET scans as amyloid positive or negative, evaluate its performance on independent data sets and different tracers, and compare it with human visual reads. Materials and Methods This retrospective study used 8476 PET scans (6722 patients) obtained from late 2004 to early 2023 that were analyzed across five different data sets. A deep learning model, AmyloidPETNet, was trained on 1538 scans from 766 patients, validated on 205 scans from 95 patients, and internally tested on 184 scans from 95 patients in the Alzheimer's Disease Neuroimaging Initiative (ADNI) fluorine 18 (18F) florbetapir (FBP) data set. It was tested on ADNI scans using different tracers and scans from independent data sets. Scan amyloid positivity was based on mean cortical standardized uptake value ratio cutoffs. To compare with model performance, each scan from both the Centiloid Project and a subset of the Anti-Amyloid Treatment in Asymptomatic Alzheimer's Disease (A4) study were visually interpreted with a confidence level (low, intermediate, high) of amyloid positivity/negativity. The area under the receiver operating characteristic curve (AUC) and other performance metrics were calculated, and Cohen κ was used to measure physician-model agreement. Results The model achieved an AUC of 0.97 (95% CI: 0.95, 0.99) on test ADNI 18F-FBP scans, which generalized well to 18F-FBP scans from the Open Access Series of Imaging Studies (AUC, 0.95; 95% CI: 0.93, 0.97) and the A4 study (AUC, 0.98; 95% CI: 0.98, 0.98). Model performance was high when applied to data sets with different tracers (AUC ≥ 0.97). Other performance metrics provided converging evidence. Physician-model agreement ranged from fair (Cohen κ = 0.39; 95% CI: 0.16, 0.60) on a sample of mostly equivocal cases from the A4 study to almost perfect (Cohen κ = 0.93; 95% CI: 0.86, 1.0) on the Centiloid Project. Conclusion The developed model was capable of automatically and accurately classifying brain PET scans as amyloid positive or negative without relying on experienced readers or requiring structural MRI. Clinical trial registration no. NCT00106899 © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Bryan and Forghani in this issue.

The translation of AI-generated brain metastases (BM) segmentation into clinical practice relies heavily on diverse, high-quality annotated medical imaging datasets. The BraTS-METS 2023 challenge has gained momentum for testing and benchmarking algorithms using rigorously annotated internationally compiled real-world datasets. This study presents the results of the segmentation challenge and characterizes the challenging cases that impacted the performance of the winning algorithms. Untreated brain metastases on standard anatomic MRI sequences (T1, T2, FLAIR, T1PG) from eight contributed international datasets were annotated in stepwise method: published UNET algorithms, student, neuroradiologist, final approver neuroradiologist. Segmentations were ranked based on lesion-wise Dice and Hausdorff distance (HD95) scores. False positives (FP) and false negatives (FN) were rigorously penalized, receiving a score of 0 for Dice and a fixed penalty of 374 for HD95. The mean scores for the teams were calculated. Eight datasets comprising 1303 studies were annotated, with 402 studies (3076 lesions) released on Synapse as publicly available datasets to challenge competitors. Additionally, 31 studies (139 lesions) were held out for validation, and 59 studies (218 lesions) were used for testing. Segmentation accuracy was measured as rank across subjects, with the winning team achieving a LesionWise mean score of 7.9. The Dice score for the winning team was 0.65 ± 0.25. Common errors among the leading teams included false negatives for small lesions and misregistration of masks in space. The Dice scores and lesion detection rates of all algorithms diminished with decreasing tumor size, particularly for tumors smaller than 100 mm3. In conclusion, algorithms for BM segmentation require further refinement to balance high sensitivity in lesion detection with the minimization of false positives and negatives. The BraTS-METS 2023 challenge successfully curated well- annotated, diverse datasets and identified common errors, facilitating the translation of BM segmentation across varied environments and providing the tools for future development of personalized volumetric reports to patients undergoing BM treatment.

BackgroundMagnetic resonance imaging (MRI) scans are known to suffer from a variety of acquisition artifacts as well as equipment-based variations that impact image appearance and segmentation performance. It is still unclear whether a direct relationship exists between magnetic resonance (MR) image quality metrics (IQMs) (e.g., signal-to-noise, contrast-to-noise) and segmentation accuracy.PurposeDeep learning (DL) approaches have shown significant promise for automated segmentation of brain tumors on MRI but depend on the quality of input training images. We sought to evaluate the relationship between IQMs of input training images and DL-based brain tumor segmentation accuracy toward developing more generalizable models for multi-institutional data.MethodsWe trained a 3D DenseNet model on the BraTS 2020 cohorts for segmentation of tumor subregions enhancing tumor (ET), peritumoral edematous, and necrotic and non-ET on MRI; with performance quantified via a 5-fold cross-validated Dice coefficient. MRI scans were evaluated through the open-source quality control tool MRQy, to yield 13 IQMs per scan. The Pearson correlation coefficient was computed between whole tumor (WT) dice values and IQM measures in the training cohorts to identify quality measures most correlated with segmentation performance. Each selected IQM was used to group MRI scans as "better" quality (BQ) or "worse" quality (WQ), via relative thresholding. Segmentation performance was re-evaluated for the DenseNet model when (i) training on BQ MRI images with validation on WQ images, as well as (ii) training on WQ images, and validation on BQ images. Trends were further validated on independent test sets derived from the BraTS 2021 training cohorts.ResultsFor this study, multimodal MRI scans from the BraTS 2020 training cohorts were used to train the segmentation model and validated on independent test sets derived from the BraTS 2021 cohort. Among the selected IQMs, models trained on BQ images based on inhomogeneity measurements (coefficient of variance, coefficient of joint variation, coefficient of variation of the foreground patch) and the models trained on WQ images based on noise measurement peak signal-to-noise ratio (SNR) yielded significantly improved tumor segmentation accuracy compared to their inverse models.ConclusionsOur results suggest that a significant correlation may exist between specific MR IQMs and DenseNet-based brain tumor segmentation performance. The selection of MRI scans for model training based on IQMs may yield more accurate and generalizable models in unseen validation.

Abstract BACKGROUND Glioblastoma is an infiltrative primary brain tumor with poor prognosis despite multimodal therapy. Recurrence is inevitable secondary to tumor cell infiltration in the peritumoral tissues, beyond contrast enhancing margins, which is the target for surgical resection. We hypothesize that a machine learning model constructed from a diverse, inter-institutional dataset can improve accuracy of generated tumor infiltration maps, thus guiding precision targeted therapies. METHODS 731 MRI scans of treatment-naïve glioblastoma patients from 10 institutions were included. All patients had pre-operative multiparametric-MRI (T1, T1Gd, T2, T2-FLAIR, ADC), and underwent complete resection of the enhancing tumor followed by standard-of-care chemoradiotherapy. 42 patients were used as an independent validation set, and 689 were used for training. Of these 239 patients had histopathologically confirmed recurrence with corresponding MRI scans, which were used as ground-truth for evaluating the location of recurrence using a leave-one-site-out (LSO) method. An AI model combining deep learning and SVM was used to develop a predictive model for infiltration. We validated the generalizability of our results in an unseen, multi-institutional data set. RESULTS Our model predicted locations of recurrence with odds ratio (99% CI) 37.6 (37.1-38.1) on the LSO testing set and 24.3 (23.4-25.2) on the validation set, indicating that areas labeled highly infiltrated were over 37 and 24 times more likely to coincide with future recurrence respectively. CONCLUSIONS We demonstrate that AI-based pattern analysis from multiparametric-MRI can predict tumor infiltration in peritumoral regions with high likelihood of recurrence by decrypting the visually imperceptible heterogeneity of peritumoral tissue. Model performance improved from training on a larger/diverse dataset and combining results of multiple AI methods. Independent validation confirmed the model’s ability to generalize to unseen data. We believe this will serve to advance AI-based biomarkers for predicting future recurrence and facilitate development of multi-modal targeted therapies in this era of precision neuro-oncology.