Scope Ambiguities in Large Language Models
arxiv(2024)
摘要
Sentences containing multiple semantic operators with overlapping scope often
create ambiguities in interpretation, known as scope ambiguities. These
ambiguities offer rich insights into the interaction between semantic structure
and world knowledge in language processing. Despite this, there has been little
research into how modern large language models treat them. In this paper, we
investigate how different versions of certain autoregressive language models –
GPT-2, GPT-3/3.5, Llama 2 and GPT-4 – treat scope ambiguous sentences, and
compare this with human judgments. We introduce novel datasets that contain a
joint total of almost 1,000 unique scope-ambiguous sentences, containing
interactions between a range of semantic operators, and annotated for human
judgments. Using these datasets, we find evidence that several models (i) are
sensitive to the meaning ambiguity in these sentences, in a way that patterns
well with human judgments, and (ii) can successfully identify human-preferred
readings at a high level of accuracy (over 90
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要