Advances in multimodal large language models (MLLMs) are extending radiological artificial intelligence (AI) beyond task-specific image analysis toward multimodal understanding and reasoning. Volumetric radiology, however, presents a fundamental representational mismatch: clinical interpretation often requires full-volume spatial context and acquisition-dependent quantitative information, whereas current MLLMs are commonly conditioned on selected two-dimensional (2D) images, compressed visual representations, or report-derived text. Reliable volumetric radiology AI therefore requires representations that preserve task-relevant three-dimensional (3D) information and systems that can access, verify, and integrate this information across clinical workflows. In this Review, we examine more than 200 publications through July 2026. We organize the literature around volumetric representation and multimodal understanding at the model level, agentic orchestration at the system level, and their links to clinical applications and evaluation. We review volumetric foundation models, language alignment and compression strategies, and agentic systems that extend MLLMs through planning, tools, memory, and workflow interaction. We distinguish settings in which selected 2D views or report-mediated reasoning may suffice from those that warrant native volumetric modeling. We also introduce a Claim-Design-Validation framework to assess whether technical, workflow, and clinical claims are matched by appropriate design and validation. Across the literature, native volumetric modeling and agentic capabilities depend on the spatial, quantitative, contextual, and workflow requirements of the intended task. Clinical credibility requires faithful volumetric representation, traceable system behavior, claim-aligned validation, and clearly defined human oversight in realistic workflows.

Existing subject-driven text-to-image generation models suffer from tedious fine-tuning steps and struggle to maintain both text-image alignment and subject fidelity. For generating compositional subjects, it often encounters problems such as object missing and attribute mixing, where some subjects in the input prompt are not generated or their attributes are incorrectly combined. To address these limitations, we propose a subject-driven generation framework and introduce training-free guidance to intervene in the generative process during inference time. This approach strengthens the attention map, allowing for precise attribute binding and feature injection for each subject. Notably, our method exhibits exceptional zero-shot generation ability, especially in the challenging task of compositional generation. Furthermore, we propose a novel metric GroundingScore to evaluate subject alignment thoroughly. The obtained quantitative results serve as compelling evidence showcasing the effectiveness of our proposed method. The code will be released soon.

As a critical statistical technique in oncology, survival prediction is used to estimate the probability of survival or time-to-event outcomes. Identifying survival-related factors from pathology and genomic data is a key approach for analyzing survival outcomes. However, current methods face several challenges, such as the suboptimal adaptation of pre-trained vision foundation models to specific tasks during feature extraction from whole slide images (WSIs), and the fact that many pathology-based models fail to integrate repetitive gene expression information during pre-training. In this study, we propose a plug-and-play multiple instance learning (MIL)-based foundation model tuning strategy to adapt vision foundation models for downstream tasks and incorporate knowledge from genomic data. Specifically, we introduce Task-specific Instance Selection, which utilizes zero-shot learning to efficiently select task-relevant WSI regions, improving tuning efficiency and reducing interference from irrelevant tissue areas. Additionally, we develop a multi-model prompt token for model fine-tuning, which integrates genetic information into the prompt-tuning process and transfers new modality information to pre-trained vision foundation models. To further enhance the model's ability to learn genetic information during fine-tuning, we introduce a Gene Distribution Aware Task as an auxiliary task to the traditional survival task. This auxiliary task helps the model better perceive multimodal information. Extensive experimental results on three public TCGA datasets demonstrate that our model outperforms all previous MIL-based methodologies and fine-tuning approaches in terms of performance.

Objective.Non-contrast-enhanced computed tomography (NCCT) images have limited tissue resolution for gastric cancer diagnosis, while contrast-enhanced computed tomography (CECT) scans involve risks such as allergic reactions and high radiation exposure. This study proposes an auxiliary diagnostic framework that integrates generative adversarial networks (GANs) with radiomics analysis to enhance the utility of NCCT images and facilitate a more informative imaging-based evaluation of gastric cancer.Approach.NCCT images, CECT images, and corresponding clinical data were collected from 1757 gastric cancer patients across four centers. After Symmetric Normalization-based registration, multiple Pix2Pix-based configurations with attention mechanisms and a composite loss function were developed. Using quantitative metrics including mean absolute error (MAE), mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM), ablation experiments identified the optimal model, which outperformed Pix2Pix, CycleGAN, and Brownian bridge diffusion model (BBDM). Finally, synthetic CECT (SCECT) images were applied in radiomics analysis to predict histological grade (low-grade), Lauren classification (diffuse type), and T stage (T3-T4), thereby demonstrating the feasibility of the framework.Main results.The proposed model significantly outperformed Pix2Pix, CycleGAN, and BBDM on internal and external test sets, as evidenced by quantitative metrics such as MAE, MSE, PSNR, and SSIM. Furthermore, SCECT images generated by the proposed model predicted gastric cancer pathology more accurately than original NCCT images. Specifically, the area under the curve improvements on the internal and external test sets were 10.8% and 1.3% for histological grade (low-grade), 12.4% and 13.7% for Lauren classification (diffuse type), and 1.5% and 4.8% for T stage (T3-T4), respectively.Significance.The proposed framework enhances the diagnostic value of NCCT images without requiring contrast agents, offering a safer and more efficient auxiliary tool for gastric cancer diagnosis.

BACKGROUND:Accurate prediction of early recurrence (ER) after radical resection remains a critical challenge in pancreatic ductal adenocarcinoma (PDAC). This study aimed to develop and validate an integrated radiomic-pathology (Rad-Path) model for ER prediction and to elucidate its underlying biological mechanisms. METHODS:A retrospective cohort of 225 PDAC patients who underwent R0 resection was included. Preoperative CT images and whole-slide images (WSI) were collected for the extraction of radiomic features and computational pathology features. Selected features were used to develop 11 distinct machine learning models. The SHapley Additive exPlanations (SHAP) algorithm was employed to evaluate feature importance. Single-cell RNA sequencing (scRNA-seq) and spatial transcriptomics (ST) were performed on prospectively collected specimens. RESULTS:The final Rad-Path model achieved AUCs of 0.851 and 0.814 in the internal and external validation cohorts, respectively. The predicted ER group was specifically linked to the enrichment of fibroblasts and pancreatic stellate cells, as well as dysregulation in extracellular matrix (ECM)-related pathways. This finding was validated histopathologically, as predicted ER patients predominantly displayed a "reactive-dominant" phenotype marked by abundant activated fibroblasts and ECM deposition. CONCLUSION:Our study offers a high-performance predictive model for ER in PDAC and establishes ECM remodeling as a key biological mechanism underlying the predictions.

Vision-language models (VLMs) have emerged as a promising approach for survival analysis in digital pathology, effectively bridging the semantic gap between histological features and clinical outcomes. However, current approaches face significant challenges: (1) generic survival prompts that fail to capture diverse pathological perspectives; (2) limited transferability of pretrained foundational models to survival tasks due to weak supervision and small, highly heterogeneous datasets; and (3) insufficient guidance for capturing complex inter-patient relationships in survival time variations. To address these challenges, MPRSurv is proposed, which leverages multi-perspective prompts to adapt VLMs for survival ranking tasks. First, multi-perspective prompts are designed to construct diverse survival templates from clinical viewpoints and select optimal combinations maximizing inter-class dispersion. Second, a flexible slide-level complementary learning module with a reverse-attention mechanism is introduced, enabling lightweight adaptation of pretrained VLM encoders to capture dataset-specific features not represented in foundation models. Third, a pairwise visual ranking method with uncertainty-aware sample selection is developed to directly guide VLM visual encoders toward learning survival ranking-aware representations under weak supervision. The proposed multi-perspective prompting unifies the framework by providing enriched supervision for both complementary learning and ranking guidance. Extensive evaluation on five TCGA datasets demonstrates that the proposed framework significantly outperforms state-of-the-art methods under both full-sample and low-sample conditions for survival prediction. Code is available: https://github.com/RuofanZhang8/MPRSurv

General Multimodal Large Language Models (MLLMs) often underperform in capturing domain-specific nuances in medical diagnosis, trailing behind fully supervised baselines. Although fine-tuning provides a remedy, the high costs of expert annotation and massive computational overhead limit its scalability. To bridge this gap without updating the weights of the pre-trained backbone of the MLLM, we propose a Clinician Mimetic Workflow. This is a novel In-Context Learning (ICL) framework designed to synergize Discriminative Exemplar Coreset Selection (DECS) and Self-Refined Experience Summarization (SRES). Specifically, DECS simulates a clinician's ability to reference "anchor cases" by selecting discriminative visual coresets from noisy data at the computational level; meanwhile, SRES mimics the cognition and reflection in clinical diagnosis by distilling diverse rollouts into a dynamic textual Experience Bank. Extensive evaluation across all 12 datasets of the MedMNIST 2D benchmark demonstrates that our method outperforms zero-shot general and medical MLLMs. Simultaneously, it achieves performance levels comparable to fully supervised vision models and domain-specific fine-tuned MLLMs, setting a new benchmark for parameter-efficient medical in-context learning. Our code is available at an anonymous repository: https://anonymous.4open.science/r/Synergizing-Discriminative-Exemplars-and-Self-Refined-Experience-ED74.

Recent advances in large models demonstrate significant prospects for transforming the field of medical imaging. These models, including large language models, large visual models, and multimodal large models, offer unprecedented capabilities in processing and interpreting complex medical data across various imaging modalities. By leveraging self-supervised pretraining on vast unlabeled datasets, cross-modal representation learning, and domain-specific medical knowledge adaptation through fine-tuning, large models can achieve higher diagnostic accuracy and more efficient workflows for key clinical tasks. This review summarizes the concepts, methods, and progress of large models in medical imaging, highlighting their potential in precision medicine. The article first outlines the integration of multimodal data under large model technologies, approaches for training large models with medical datasets, and the need for robust evaluation metrics. It then explores how large models can revolutionize applications in critical tasks such as image segmentation, disease diagnosis, personalized treatment strategies, and real-time interactive systems, thus pushing the boundaries of traditional imaging analysis. Despite their potential, the practical implementation of large models in medical imaging faces notable challenges, including the scarcity of high-quality medical data, the need for optimized perception of imaging phenotypes, safety considerations, and seamless integration with existing clinical workflows and equipment. As research progresses, the development of more efficient, interpretable, and generalizable models will be critical to ensuring their reliable deployment across diverse clinical environments. This review aims to provide insights into the current state of the field and provide directions for future research to facilitate the broader adoption of large models in clinical practice.

Neoadjuvant immunochemotherapy (nICT) has demonstrated significant potential in improving pathological response rates and survival outcomes for patients with locally advanced esophageal squamous cell carcinoma (ESCC). However, substantial interindividual variability in therapeutic outcomes highlights the urgent need for more precise predictive tools to guide clinical decision-making. Traditional biomarkers remain limited in both predictive performance and clinical feasibility. In recent years, the application of artificial intelligence (AI) in medical imaging has expanded rapidly. By incorporating voxel-level feature maps, the combination of radiomics and deep learning enables the extraction of rich textural, morphological, and microstructural features, while autonomously learning high-level abstract representations from clinical CT images, thereby revealing biological heterogeneity that is often imperceptible to conventional assessments. Leveraging these high-dimensional representations, AI models can provide more accurate predictions of nICT response. Future advancements in foundation models, multimodal integration, and dynamic temporal modeling are expected to further enhance the generalizability and clinical applicability of AI. AI-powered medical imaging is poised to support all stages of perioperative management in ESCC, playing a pivotal role in high-risk patient identification, dynamic monitoring of therapeutic response, and individualized treatment adjustment, thereby comprehensively advancing precision nICT.

Gastric cancer is a high-incidence malignancy that poses a serious threat to public health in China, ranking among the top three cancers in both incidence and mortality. The majority of patients are diagnosed at an advanced stage, resulting in limited treatment options and poor prognosis. To address key challenges in gastric cancer diagnosis and treatment, a research team led by Professor Jiafu Ji at Peking University Cancer Hospital has focused on the project "Development and Dissemination of Precision Medicine Approaches in Gastric Cancer Management". Through a series of high-quality multicenter clinical studies, the team established a set of new international standards in perioperative treatment, individua-lized drug selection, intelligent noninvasive diagnostics, and novel immunotherapy strategies. These advances have significantly improved treatment efficacy and reduced surgical trauma, achieving key technological breakthroughs in diagnosis, therapy, and mechanistic understanding, and systematically enhancing outcomes for gastric cancer patients. The project ' s findings had a broad international impact, including hosting China ' s first International Gastric Cancer Congress. Through nationwide dissemination, they have promoted the development of precision diagnosis and treatment of gastric cancer as a discipline, and led the formulation of the National Health Commission's guidelines for gastric cancer diagnosis and treatment. In recognition of its achievements, the project was awarded the First Prize of the 2024 Chinese Medical Science and Technology Award.

Large language models (LLMs), trained on vast amounts of textual data, have demonstrated strong capabilities in natural language understanding and generation. In the medical field, LLMs are increasingly applied across various domains such as disease screening, diagnostic assistance, and health management, playing a key role in advancing intelligent healthcare. In recent years, China has actively promoted the integration of artificial intelligence (AI) with healthcare through a series of policies that support enterprises in making breakthroughs in key technologies such as medical LLMs and multimodal data integration. Concurrently, efforts have accelerated the deployment of AI in applications such as health management and precision medicine to gradually establish a full-cycle intelligent healthcare system encompassing prevention, diagnosis, treatment, and rehabilitation. However, the rapid deployment of LLMs in healthcare has highlighted the lack of standardized evaluation criteria and consistent methodologies. To address this, this expert consensus focuses on establishing a retrospective evaluation framework tailored to medical applications. By integrating scientific evaluation metrics, standards, and procedures, the framework provides clear and actionable guidance for model evaluators, developers, and end users. It aims to unify assessment practices, enhance the scientific rigor and comparability of evaluations, and ensure the safe and effective use of LLMs in healthcare, ultimately supporting the high-quality development of AI-powered medical services.

[This corrects the article DOI: 10.1016/j.jncc.2024.01.005.].

The tumor microenvironment (TME) is important to the treatment and prognosis of cancer. Multiplex Immunohistochemical (mIHC) images can display the expression of multiple biomolecular markers while maintaining spatial location, making it an effective tool for TME analysis. However, acquiring mIHC images demands stringent laboratory conditions, extensive time and resources. In this study, we proposed an end-to-end generative framework to transform hematoxylin and eosin (H&E) images into mIHC images. In addition, we introduced a pathological foundation model into the feature encoding of pathological image generation to enhance the performance of our framework and proved the superiority and necessity of the foundation model. Our method successfully generated mIHC images, achieved state-of-the-art (SOTA) performance against popular methods and maintained robustness on cross-cancer. The framework we proposed is expected to provide a more efficient and cheaper tool for the observation and evaluation of TME in pathology.

Personalized medicine for gastric cancer continues to face numerous challenges, primarily due to the complexity of clinical decision making and the difficulty of integrating multimodal data. Artificial intelligence (AI), with its powerful capabilities in feature learning and pattern recognition, is emerging as a key technology to overcome these barriers. It provides critical support in areas such as early screening, histological subtyping, prediction of treatment response, and prognostic risk stratification. This review examines the application of AI in diagnosing and treating gastric cancer, with particular attention to the current mainstream AI methodologies, including feature engineering and deep learning and the rapidly evolving pretrained foundation models and multimodal large models. With the integration of medical images, digital pathology, multiomics data, and structured clinical information, AI systems are increasingly effective at capturing tumor heterogeneity and supporting complex clinical decisions in real time. On the one hand, task-specific models have demonstrated excellent performance in subtyping, staging, and prognosis assessment. On the other hand, the rise of foundation models and general-purpose large models is redefining the limits of AI in cross-task transfer, complex reasoning, and human–machine interaction. These technologies hold promise in addressing key obstacles such as data scarcity, modality heterogeneity, and fragmented clinical workflows, offering a feasible path toward a unified and efficient AI-driven diagnostic and therapeutic system for gastric cancer. As technological maturity progresses alongside the development of robust safety and ethical frameworks, AI is expected to evolve from a static auxiliary interpretation tool into an intelligent decision-making platform capable of semantic understanding, dynamic feedback, and multidisciplinary collaboration—therefore playing a pivotal role across the full spectrum of precision medicine in gastric cancer.

Providing precise and comprehensive diagnostic information to clinicians is crucial for improving the treatment and prognosis of nasopharyngeal carcinoma. Multi-modal foundation models, which can integrate data from various sources, have the potential to significantly enhance clinical assistance. However, several challenges remain: (1) the lack of large-scale visual-language datasets for nasopharyngeal carcinoma; (2) the inability of existing pre-training and fine-tuning methods to capture the hierarchical features required for complex clinical tasks; (3) current foundation models having limited visual perception due to inadequate integration of multi-modal information. While curriculum learning can improve a model’s ability to handle multiple tasks through systematic knowledge accumulation, it still lacks consideration for hierarchical features and their dependencies, affecting knowledge gains. To address these issues, we propose the Hierarchical Feature Fusion Curriculum Learning method, which consists of three stages: visual knowledge learning, coarse-grained alignment, and fine-grained fusion. First, we introduce the Hybrid Contrastive Masked Autoencoder to pre-train visual encoders on 755K multi-modal images of nasopharyngeal carcinoma CT, MRI, and endoscopy to fully extract deep visual information. Then, we construct a 65K visual instruction fine-tuning dataset based on open-source data and clinician diagnostic reports, achieving coarse-grained alignment with visual information in a large language model. Finally, we design a Mixture of Experts Cross Attention structure for deep fine-grained fusion of global multi-modal information. Our model outperforms previously developed specialized models in all key clinical tasks for nasopharyngeal carcinoma, including diagnosis, report generation, tumor segmentation, and prognosis.

Whole slide image (WSI) plays an important role in cancer research. Cell recognition is the foundation and key steps of WSI analysis at the cellular level, including cell segmentation, subtypes detection and molecular expression prediction at the cellular level. Current end-to-end supervised learning models rely heavily on a large amount of manually labeled data and self-supervised learning models are limited to cell binary segmentation. All of these methods lack the ability to predict the expression level of molecules in single cells. In this study, we proposed a two-phase generative adversarial learning framework, named GCESS, which can achieve end-to-end cell binary segmentation, subtypes detection and molecular expression prediction simultaneously. The framework uses generative adversarial learning to obtain better cell binary segmentation results in the first phase by integrating the cell binary segmentation results of some segmentation models and generates multiplex immunohistochemistry (mIHC) images through generative adversarial networks to predict the expression of cell molecules in the second phase. The cell semantic segmentation results can be obtained by spatially mapping the binary segmentation and molecular expression results in pixel level. The method we proposed achieves a Dice of 0.865 on cell binary segmentation, an accuracy of 0.917 on cell semantic segmentation and a Peak Signal to Noise Ratio (PSNR) of 20.929 dB on mIHC images generating, outperforming other competing methods (P-value <0.05). The method we proposed will provide an effective tool for cellular level analysis of digital pathology images and cancer research.

Automated diagnostic systems (ADS) have shown significant potential in the early detection of polyps during endoscopic examinations, thereby reducing the incidence of colorectal cancer. However, due to high annotation costs and strict privacy concerns, acquiring high-quality endoscopic images poses a considerable challenge in the development of ADS. Despite recent advancements in generating synthetic images for dataset expansion, existing endoscopic image generation algorithms failed to accurately generate the details of polyp boundary regions and typically required medical priors to specify plausible locations and shapes of polyps, which limited the realism and diversity of the generated images. To address these limitations, we present Polyp-Gen, the first full-automatic diffusion-based endoscopic image generation framework. Specifically, we devise a spatial-aware diffusion training scheme with a lesion-guided loss to enhance the structural context of polyp boundary regions. Moreover, to capture medical priors for the localization of potential polyp areas, we introduce a hierarchical retrieval-based sampling strategy to match similar fine-grained spatial features. In this way, our Polyp-Gen can generate realistic and diverse endoscopic images for building reliable ADS. Extensive experiments demonstrate the state-of-the-art generation quality, and the synthetic images can improve the downstream polyp detection task. Additionally, our Polyp-Gen has shown remarkable zero-shot generalizability on other datasets. The source code is available at https://github.com/CUHK-AIM-Group/Polyp-Gen.