PHAnToM: Personality Has An Effect on Theory-of-Mind Reasoning in Large Language Models
arxiv(2024)
摘要
Recent advances in large language models (LLMs) demonstrate that their
capabilities are comparable, or even superior, to humans in many tasks in
natural language processing. Despite this progress, LLMs are still inadequate
at social-cognitive reasoning, which humans are naturally good at. Drawing
inspiration from psychological research on the links between certain
personality traits and Theory-of-Mind (ToM) reasoning, and from prompt
engineering research on the hyper-sensitivity of prompts in affecting LLMs
capabilities, this study investigates how inducing personalities in LLMs using
prompts affects their ToM reasoning capabilities. Our findings show that
certain induced personalities can significantly affect the LLMs' reasoning
capabilities in three different ToM tasks. In particular, traits from the Dark
Triad have a larger variable effect on LLMs like GPT-3.5, Llama 2, and Mistral
across the different ToM tasks. We find that LLMs that exhibit a higher
variance across personality prompts in ToM also tends to be more controllable
in personality tests: personality traits in LLMs like GPT-3.5, Llama 2 and
Mistral can be controllably adjusted through our personality prompts. In
today's landscape where role-play is a common strategy when using LLMs, our
research highlights the need for caution, as models that adopt specific
personas with personalities potentially also alter their reasoning abilities in
an unexpected manner.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要