MRScore: Evaluating Radiology Report Generation with LLM-based Reward System
arxiv(2024)
摘要
In recent years, automated radiology report generation has experienced
significant growth. This paper introduces MRScore, an automatic evaluation
metric tailored for radiology report generation by leveraging Large Language
Models (LLMs). Conventional NLG (natural language generation) metrics like BLEU
are inadequate for accurately assessing the generated radiology reports, as
systematically demonstrated by our observations within this paper. To address
this challenge, we collaborated with radiologists to develop a framework that
guides LLMs for radiology report evaluation, ensuring alignment with human
analysis. Our framework includes two key components: i) utilizing GPT to
generate large amounts of training data, i.e., reports with different
qualities, and ii) pairing GPT-generated reports as accepted and rejected
samples and training LLMs to produce MRScore as the model reward. Our
experiments demonstrate MRScore's higher correlation with human judgments and
superior performance in model selection compared to traditional metrics. Our
code and datasets will be available on GitHub.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要