Generating Diverse Criteria On-the-Fly to Improve Point-wise LLM Rankers
arxiv(2024)
摘要
The most recent pointwise Large Language Model (LLM) rankers have achieved
remarkable ranking results. However, these rankers are hindered by two major
drawbacks: (1) they fail to follow a standardized comparison guidance during
the ranking process, and (2) they struggle with comprehensive considerations
when dealing with complicated passages. To address these shortcomings, we
propose to build a ranker that generates ranking scores based on a set of
criteria from various perspectives. These criteria are intended to direct each
perspective in providing a distinct yet synergistic evaluation. Our research,
which examines eight datasets from the BEIR benchmark demonstrates that
incorporating this multi-perspective criteria ensemble approach markedly
enhanced the performance of pointwise LLM rankers.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要