Knowledge-enhanced Visual-Language Pretraining for Computational Pathology
arxiv(2024)
摘要
In this paper, we consider the problem of visual representation learning for
computational pathology, by exploiting large-scale image-text pairs gathered
from public resources, along with the domain specific knowledge in pathology.
Specifically, we make the following contributions: (i) We curate a pathology
knowledge tree that consists of 50,470 informative attributes for 4,718
diseases requiring pathology diagnosis from 32 human tissues. To our knowledge,
this is the first comprehensive structured pathology knowledge base; (ii) We
develop a knowledge-enhanced visual-language pretraining approach, where we
first project pathology-specific knowledge into latent embedding space via
language model, and use it to guide the visual representation learning; (iii)
We conduct thorough experiments to validate the effectiveness of our proposed
components, demonstrating significant performance improvement on various
downstream tasks, including cross-modal retrieval, zero-shot classification on
pathology patches, and zero-shot tumor subtyping on whole slide images (WSIs).
All codes, models and the pathology knowledge tree will be released to the
research community
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要