Rate control (RC) schemes allow audio and video encoders to produce bitstreams according to specific overall bitrate constraints. However, when no rate capping is enforced, the instantaneous bitrate may vary strongly and may exceed the target rate by an order of magnitude, potentially causing playback stutter especially in video streaming scenarios. This paper introduces a rate capping extension for the two RC modes in VVenC, an open Versatile Video Coding (VVC) compliant encoder implementation. After a revisit of VVenC’s two-pass RC approach, the algorithmic details of the rate capping model are described. The paper concludes with an objective evaluation of the performance of the RC extension in a random-access configuration.

Contemporary motion picture content, consisting of scenes with different amounts of visual complexity or camera noise, represents demanding input for video encoders operating in rate control (RC) modes. This paper presents improvements to the 2-pass RC method integrated into VVenC, an open VVC encoder implementation, outlined in previous publications. We specifically introduce three extensions to our RC solution: first, frame type adaptation operating near scene cuts, along with an associated simple detector; second, rate stabilization means to allow for more reliable lookahead based 2-pass RC operation in on-the-fly encoding applications; and third, a low-complexity approach for estimating the instantaneous intensity of camera noise or film grain to avoid large variations in bit consumption when encoding individual frames in the final RC pass. Experimental evaluation confirms that these extensions significantly improve both the objective (BD rate) and subjective (visual) RC performance of VVenC especially on challenging video content.

The user experience in adaptive HTTP streaming relies on offering bitrate ladders with suitable operation points for all users and typically involves multiple resolutions. While open GOP coding structures are generally known to provide substantial coding efficiency benefit, their use in HTTP streaming has been precluded through lacking support of reference picture resampling (RPR) in AVC and HEVC. The newly emerging Versatile Video Coding (VVC) standard supports RPR, but only conversational scenarios were primarily investigated during the design of VVC. This paper aims at enabling usage of RPR in HTTP streaming scenarios through analysing the drift potential of VVC coding tools and presenting a constrained encoding method that avoids severe drift artefacts in resolution switching with open GOP coding in VVC. In typical live streaming configurations, the presented method achieves -8.7% BD-rate reduction compared to closed GOP coding while in a typical Video on Demand configuration, -1.89% BD-rate reduction is reported. The constraints penalty compared to regular open GOP coding is 0.65% BD-rate in the worst case. The presented method was integrated into the publicly available open source VVC encoder VVenC v0.3.

To provide an encoder and a decoder that improve coding efficiency by dividing blocks into non-rectangular parts.SOLUTION: An encoder separates a block 200 into two regions 202a and 202b called wedgelet partitions by one line segment 201. A line segment separating the regions is determined by a start point S and an end point E, both of which are set on the boundary of the block. Partition information is stored in the form of a partition pattern to use wedgelet partitions (areas 202a and 202b).SELECTED DRAWING: Figure 4

The recently finalized Versatile Video Coding (VVC) standard promises to reduce the video bitrate by 50% compared to its predecessor, High Efficiency Video Coding (HEVC). The increased efficiency comes at a cost of increased computational burden. The Fraunhofer Versatile Video Encoder VVenC is the first openly available optimized implementation providing access to VVC’s efficiency at only 46% of the runtime of the VVC test model VTM, when not using multi-threading. An alternative operating point allows $30 \times$ faster encoding for the price of around 12% bitrate increase, while still providing around 38% bitrate reduction compared to HEVC test model HM. In the fastest configuration, VVenC runs over $140 \times$ faster than VTM while still providing over 10% bitrate reduction compared to HM. Even faster encoding is possible with multi-threading. This paper provides an overview of VVenC’s main features and some evaluation results.

Versatile Video Coding (VVC) is the emerging video coding standard to be finalized by the Joint Video Experts Team in July 2020. Compared to its predecessor, the High Efficiency Video Coding (HEVC) standard, VVC provides 50% bit-rate reduction at comparable visual quality for natural video content in high-definition (HD) and ultra high-definition (UHD) resolution. To achieve this, the standard incorporates more advanced and generalized algorithms, leading to an increase in computational complexity. This includes for example additional in-loop filters, decoder-side motion refinement and search as well as an increased number of transforms, which creates a grand challenge for implementers to achieve live decoding on general-purpose CPUs. In this paper, the work on an efficient software decoder implementation for the upcoming VVC standard is described, including optimization of sample operations using single instruction multiple data (SIMD) instructions and parallelization approaches with multithreading. As a result, the presented decoder can perform live decoding of 10bit HD video at 60 frames per second (fps) and 10bit UHD video at 30fps on modern mobile consumer hardware, showcasing that VVC live decoding is possible already right before finalization of the standard.

In this paper we present a "State-Based Multi-Parameter Probability Estimation" (SBMP) for Context-Based Adaptive Binary Arithmetic Coding (CABAC) which employs a two hypotheses probability estimator based on exponentially weighted moving averages. It uses a logarithmic state representation and a single subsampled transition table with only 32 elements for the probability update. This reduces the memory requirements virtually without affecting the compression efficiency, compared to corresponding approaches that use a linear state representation and a computation-based probability update. The proposed scheme is based on simple operations like table look-ups and additions. Compared to the state-of-the-art probability estimator of the video compression standard H.265/HEVC, the compression efficiency is increased by up to 1 % Bjøntegaard-Delta bit rate (BD rate) when applied to draft 2 of the Versatile Video Coding (VVC) standard. Furthermore, SBMP was recently adopted to working draft 2 of the MPEG-7 part 17 standard for compression of neural networks for multimedia content description and analysis.

A key component of state-of-the art video coding is motion-compensated prediction, also called inter prediction. Current standards allow uni- and bi-prediction, i.e. linear superposition of up to two motion-compensated prediction signals. It is well-known that by a superposition of more than two prediction signals (or hypotheses), the energy of the prediction error can be further reduced. In this paper, it is shown that allowing the encoder to choose among different weights for the individual hypotheses is beneficial from a rate-distortion perspective. A practical multi-hypothesis inter prediction scheme based on the Versatile Video Coding Test Model (VTM) is presented. For VTM-1, in the Random Access configuration according to the JVET Common Test Conditions, the average luma BD bit rate is in the range of -1.6 % to -1.9 % for different settings using up to four prediction hypotheses. For VTM-2, the corresponding BD bit rate is -0.95 %. For higher bit rates (i.e., QP values 12, 17, 22, 27) the BD bit rates are -2.2 % for VTM-1 and -1.4 % for VTM-2.

Les informations provenant du codage/decodage de la couche de base, c'est-a-dire les indications d'optimisation de couche de base, sont exploitees pour rendre la prediction a compensation de mouvement de la couche d'amelioration plus efficace par le codage plus efficace des parametres de mouvement de la couche d'amelioration.