Rethinking Response Evaluation from Interlocutor's Eye for Open-Domain Dialogue Systems
CoRR(2024)
摘要
Open-domain dialogue systems have started to engage in continuous
conversations with humans. Those dialogue systems are required to be adjusted
to the human interlocutor and evaluated in terms of their perspective. However,
it is questionable whether the current automatic evaluation methods can
approximate the interlocutor's judgments. In this study, we analyzed and
examined what features are needed in an automatic response evaluator from the
interlocutor's perspective. The first experiment on the Hazumi dataset revealed
that interlocutor awareness plays a critical role in making automatic response
evaluation correlate with the interlocutor's judgments. The second experiment
using massive conversations on X (formerly Twitter) confirmed that dialogue
continuity prediction can train an interlocutor-aware response evaluator
without human feedback while revealing the difficulty in evaluating generated
responses compared to human responses.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要