AWEQ: Post-Training Quantization with Activation-Weight Equalization for Large Language Models.
CoRR(2023)
摘要
Large language models(LLMs) exhibit excellent performance across a variety of
tasks, but they come with significant computational and storage costs.
Quantizing these models is an effective way to alleviate this issue. However,
existing methods struggle to strike a balance between model accuracy and
hardware efficiency. This is where we introduce AWEQ, a post-training method
that requires no additional training overhead. AWEQ excels in both
ultra-low-bit quantization and 8-bit weight and activation (W8A8) quantization.
There is an observation that weight quantization is less challenging than
activation quantization. AWEQ transfers the difficulty of activation
quantization to weights using channel equalization, achieving a balance between
the quantization difficulties of both, and thereby maximizing performance. We
have further refined the equalization method to mitigate quantization bias
error, ensuring the robustness of the model. Extensive experiments on popular
models such as LLaMA and OPT demonstrate that AWEQ outperforms all existing
post-training quantization methods for large models.
更多查看译文
关键词
large language models,language models,post-training,activation-weight
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要