OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization
arxiv(2024)
摘要
Structured pruning is a promising approach for reducing the inference costs
of large vision and language models. By removing carefully chosen structures,
e.g., neurons or attention heads, the improvements from this approach can be
realized on standard deep learning hardware. In this work, we focus on
structured pruning in the one-shot (post-training) setting, which does not
require model retraining after pruning. We propose a novel combinatorial
optimization framework for this problem, based on a layer-wise reconstruction
objective and a careful reformulation that allows for scalable optimization.
Moreover, we design a new local combinatorial optimization algorithm, which
exploits low-rank updates for efficient local search. Our framework is time and
memory-efficient and considerably improves upon state-of-the-art one-shot
methods on vision models (e.g., ResNet50, MobileNet) and language models (e.g.,
OPT-1.3B – OPT-30B). For language models, e.g., OPT-2.7B, OSSCAR can lead to
125× lower test perplexity on WikiText with 2× inference time
speedup in comparison to the state-of-the-art ZipLM approach. Our framework is
also 6× – 8× faster. Notably, our work considers models with
tens of billions of parameters, which is up to 100× larger than what has
been previously considered in the structured pruning literature.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要