COGNET-MD, an evaluation framework and dataset for Large Language Model benchmarks in the medical domain
CoRR(2024)
摘要
Large Language Models (LLMs) constitute a breakthrough state-of-the-art
Artificial Intelligence (AI) technology which is rapidly evolving and promises
to aid in medical diagnosis either by assisting doctors or by simulating a
doctor's workflow in more advanced and complex implementations. In this
technical paper, we outline Cognitive Network Evaluation Toolkit for Medical
Domains (COGNET-MD), which constitutes a novel benchmark for LLM evaluation in
the medical domain. Specifically, we propose a scoring-framework with increased
difficulty to assess the ability of LLMs in interpreting medical text. The
proposed framework is accompanied with a database of Multiple Choice Quizzes
(MCQs). To ensure alignment with current medical trends and enhance safety,
usefulness, and applicability, these MCQs have been constructed in
collaboration with several associated medical experts in various medical
domains and are characterized by varying degrees of difficulty. The current
(first) version of the database includes the medical domains of Psychiatry,
Dentistry, Pulmonology, Dermatology and Endocrinology, but it will be
continuously extended and expanded to include additional medical domains.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要