WsiCaption: Multiple Instance Generation of Pathology Reports for Gigapixel Whole-Slide Images
arxiv(2023)
摘要
Whole slide images are the foundation of digital pathology for the diagnosis
and treatment of carcinomas. Writing pathology reports is laborious and
error-prone for inexperienced pathologists. To reduce the workload and improve
clinical automation, we investigate how to generate pathology reports given
whole slide images. On the data end, we curated the largest WSI-text dataset
(TCGA-PathoText). In specific, we collected nearly 10000 high-quality WSI-text
pairs for visual-language models by recognizing and cleaning pathology reports
which narrate diagnostic slides in TCGA. On the model end, we propose the
multiple instance generative model (MI-Gen) which can produce pathology reports
for gigapixel WSIs. We benchmark our model on the largest subset of
TCGA-PathoText. Experimental results show our model can generate pathology
reports which contain multiple clinical clues. Furthermore, WSI-text prediction
can be seen as an approach of visual-language pre-training, which enables our
model to be transferred to downstream diagnostic tasks like carcinoma grading
and phenotyping. We observe that simple semantic extraction from the pathology
reports can achieve the best performance (0.838 of F1 score) on BRCA subtyping
without adding extra parameters or tricky fine-tuning. Our collected dataset
and related code are available.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要