Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation
arxiv(2023)
摘要
Large Language Models (LLMs) drive current AI breakthroughs despite very
little being known about their internal representations. In this work, we
propose to shed the light on LLMs inner mechanisms through the lens of
geometry. In particular, we develop in closed form (i) the intrinsic
dimension in which the Multi-Head Attention embeddings are constrained to exist
and (ii) the partition and per-region affine mappings of the feedforward
(MLP) network of LLMs' layers. Our theoretical findings further enable the
design of novel principled solutions applicable to state-of-the-art LLMs.
First, we show that, through our geometric understanding, we can bypass LLMs'
RLHF protection by controlling the embedding's intrinsic dimension through
informed prompt manipulation. Second, we derive interpretable geometrical
features that can be extracted from any (pre-trained) LLM, providing a rich
abstract representation of their inputs. We observe that these features are
sufficient to help solve toxicity detection, and even allow the identification
of various types of toxicity. Our results demonstrate how, even in large-scale
regimes, exact theoretical results can answer practical questions in LLMs.
Code: https://github.com/RandallBalestriero/SplineLLM
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要