Shortened LLaMA: A Simple Depth Pruning for Large Language Models
CoRR(2024)
摘要
Structured pruning of modern large language models (LLMs) has emerged as a
way of decreasing their high computational needs. Width pruning reduces the
size of projection weight matrices (e.g., by removing attention heads) while
maintaining the number of layers. Depth pruning, in contrast, removes entire
layers or blocks, while keeping the size of the remaining weights unchanged.
Most current research focuses on either width-only or a blend of width and
depth pruning, with little comparative analysis between the two units (width
vs. depth) concerning their impact on LLM inference efficiency. In this work,
we show that a simple depth pruning approach can compete with recent width
pruning methods in terms of zero-shot task performance. Our pruning method
boosts inference speeds, especially under memory-constrained conditions that
require limited batch sizes for running LLMs, where width pruning is
ineffective. We hope this work can help deploy LLMs on local and edge devices.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要