A Fast Zero-Quantized Percentage Model for Video Coding with RDO Quantization.

In video coding, the percentage of zero-quantized coefficients, denoted by rho, is directly determined by the quantization algorithm adopted. rho-domain rate distortion (RD) modeling is widely employed to optimize the implementation algorithm for customizable modules such as rate control and mode decision etc. How to calculate or estimate rho according to quantization algorithm is the first step task for rho-domain RD modeling. There are two typical quantization algorithm, soft-decision quantization such as dead-zone, and soft-decision quantization such as rate distortion optimized quantization (RDOQ). RDOQ is more frequently employed in the latest video encoders compared with deadzone quantization due to its inspiring coding performance. In HDQ based video encoder, rho can be easily obtained by simply rounding. However, it is computation-intensive to calculate rho in video encoder with RDOQ, in which complicated trellis search is employed. This paper focus on developing estimation model for quickly estimating rho for RDOQ based video coding. The contribution of this article is as follows: First, this paper develops the rho model adaptively according to an adaptive deadzone offset model, which is modeled by imitating the behavior of RDOQ. Second, an accurate rho model is adaptively built offline as function of weighted SATD (sum of absolute transformed distortion) denoted by WSATD, quantization step size q, and average WSATD/q estimated from ensemble. The weight in WSATD is adaptively determined according to the adaptive offset to simulate the behavior pattern of RDOQ as much as possible. Experimental results verify that the proposed model can quickly and accurately predict the rho results of RDOQ with moderate implementation complexity. The proposed rho model can be employed to estimate the percentage of zero quantized coefficients which can be used for fast all-zero detection and rho domain rate distortion modeling in RDOQ based HEVC video encoder.