RANQ: Region-Adaptive Non-uniform Quantization for Efficient LLM Compression

Large language models (LLMs) have demonstrated remarkable performance across a wide range of domains. However, their deployment in real-world environments remains challenging due to the substantial model sizes and limited hardware resources. Quantization has been widely adopted to alleviate these issues, but the presence of outliers in the weight distribution of LLMs significantly degrades quantization performance, particularly in low-bit settings. To address this challenge, we propose a region-adaptive non-uniform quantization (RANQ) method that takes into account the intrinsic characteristics of LLM weight distributions. RANQ divides the entire weight space into central and outlier regions, applying distinct scale factors to each region to minimize performance degradation caused by quantization. Furthermore, we introduce a layer-wise reconstruction-based threshold optimization algorithm, which adaptively determines the optimal region boundaries according to the distributional properties of each layer, enabling more precise low-bit quantization. Experimental results on the LLaMA family using the WikiText2 and C4 datasets demonstrate that the proposed method consistently outperforms existing post training quantization (PTQ) techniques, such as round-to-nearest (RTN) and GPTQ.