Measuring Bargaining Abilities of LLMs: A Benchmark and A Buyer-Enhancement Method
CoRR(2024)
摘要
Bargaining is an important and unique part of negotiation between humans. As
LLM-driven agents learn to negotiate and act like real humans, how to evaluate
agents' bargaining abilities remains an open problem. For the first time, we
formally described the Bargaining task as an asymmetric incomplete information
game, defining the gains of the Buyer and Seller in multiple bargaining
processes. It allows us to quantitatively assess an agent's performance in the
Bargain task. We collected a real product price dataset, AmazonHistoryPrice,
and conducted evaluations of various LLM agents' bargaining abilities. We find
that playing a Buyer is much harder than a Seller, and increasing model size
can not effectively improve the Buyer's performance. To address the challenge,
we propose a novel approach called OG-Narrator that integrates a deterministic
Offer Generator to control the price range of Buyer's offers, and an LLM
Narrator to create natural language sentences for generated offers.
Experimental results show that OG-Narrator improves the buyer's deal rates from
26.67
baselines, even a model that has not been aligned.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要