In recent years, video-related applications, including online conferences, remote education, and screen sharing—have expanded from social media to industrial products, driving growing demand for ultrahigh-efficiency screen content coding (SCC). Screen content typically contains diverse repeated patterns with variations not only in scale, size and sharpness, but also under spatial transformations such as mirroring and rotation. To exploit the redundancy of these transformed repeating patterns, this article proposes a multiscan-direction string prediction (MSDSP) framework for ultrahigh-efficiency SCC. The MSDSP implemented in the latest exploration video model of the audio video coding standard (AVS) integrates versatile string types and multiscan directions, a multiunit hash table search strategy, and an optimized coding scheme. Experimental results demonstrate that the proposed MSDSP achieves an average Y BD-rate reduction up to 24.64% over the baseline EVM for four typical screen snapshots with diverse repeated patterns. Under the AVS3 SCC common test conditions using text and graphics with motion and MC YUV sequences, the technique delivers average Y BD-rate reductions of 2.47% (all intra), 1.49% (low delay), and 1.7% (random access), with negligible computational overhead in both encoding and decoding.

Screen Content Coding (SCC) is an indispensable tool for enabling distributed collaboration, such as video conferencing. Encoders in the latest video coding standards, particularly for SCC scenarios, employ a wider variety of partitioning tree splitting types, recursively traversing all branches, as well as a larger number of coding modes and submodes, to achieve higher coding efficiency compared to encoders in previous standards. This process leads to very high coding complexity, as each tree leaf node, called a coding unit (CU), for every partitioning size and location in the picture is repeatedly visited and evaluated multiple times during the optimal partitioning search. Additionally, each CU visit involves evaluating a vast number of coding options and their combinations to identify the best one. The complexity is further exacerbated in SCC due to the addition of many new CU coding modes and options. To significantly reduce SCC complexity without coding efficiency loss, this paper proposes a new technique, Accelerated Revisit CU-coding (ARC), along with an SCC search space analysis for in-depth operation-level and run/platform-independent assessment of SCC complexity. ARC exploits the correlation between the first visit and subsequent revisits of a CU with the same location and size. By fully leveraging the correlation and information from the first visit, ARC significantly accelerates revisit CU-coding while maintaining the same high coding efficiency. ARC is implemented in HPM, the AVS3 reference software. Experiments demonstrate that ARC reduces encoding runtime by 29.74%, 47.78%, and 54.25% for 1920x1080 FHD, 4 K UHD, and 8 K UHD test sequences, respectively, in All Intra configuration, without coding efficiency loss. These runtime reductions align with corresponding search space reductions of 30.91%, 49.67%, and 54.41%, as obtained from the search space analysis.

String prediction (SP) is a highly efficient screen content coding technique adopted into international and China video coding standards. However, SP requires a high number of SRAM fetches to decode and output a block for display, leading to low throughput (T). Low T results in a high decoder and SRAM clock frequency to output the required number of display pixels, which is determined by the specific display resolution and frame rate. To achieve hardware-decoder-friendly high throughput SP (HTSP), this paper exploits specific SRAM fetch rate constraints for five SRAM-cell sizes commonly used in hardware decoder designs. Additionally, the optimal reference string selection process is formulated as a multi-constraint rate-distortion optimization (MCRDO) problem and a novel reference string searching method is presented. HTSP boosts throughput by up to 4 times compared to the state-of-the- art SP, with only a negligible impact on coding efficiency.

String prediction (SP) is a highly efficient screen content coding (SCC) tool that has been adopted in international and Chinese video coding standards. SP exhibits a highly flexible and efficient ability to predict repetitive matching patterns. However, SP also suffers from low throughput of decoded display output pixels per memory access, which is synchronized with the decoder clock, due to the high number of memory accesses required to decode an SP coding unit for display. Even in state-of-the-art (SOTA) SP, the worst-case scenario involves two memory accesses for decoding each 4-pixel basic string unit across two memory access units, resulting in a throughput as low as two pixels per memory access (PPMA). To solve this problem, we are the first to propose a technique called memory access number constraint-based string prediction (MANC-SP) to achieve high throughput in SCC. First, a novel MANC-SP framework is proposed, a well-designed memory access number constraint rule is established on the basis of statistical data, and a constrained RDO-based string searching method is presented. Compared with the existing SOTA SP, the experimental results demonstrate that MANC-SP can improve the throughput from 2 to 2.67 PPMA, achieving a throughput improvement of 33.33% while maintaining a negligible impact on coding efficiency and complexity.

The third generation of Audio Video Coding Standard (AVS3) adopts sophisticated block partitioning structure with a multi-type tree, which includes Quad-Tree (QT), Binary-Tree (BT), and Extend Quad-Tree (EQT). During multi-type tree recursive block partitioning and pre-coding process of encoding, each Coding Unit (CU) of each Size at each Location (CUeSeL) of the picture is repeatedly pre-coded many times when visiting each branch of each tree. Although there are early termination mechanism, the number of visits for each CUeSeL is still very high and up to more than fifty. This process brings very high coding complexity. In order to reduce the coding complexity of AVS3, this paper proposes a method named Direct Non-First Pre-coding Skip (DNFPS) for a CUeSeL to directly and completely skip all non-first pre-coding under specific conditions. The conditions include, but are not limited to, those related to the first pre-coding. The contributions of the paper are as follows. In I pictures, for a CUeSeL with top-left location (x, y), if both x and y are multiples of 64 or if a set of conditions related to first pre-coding is satisfied, then each non-first pre-coding is skipped, and the coding result of the first pre-coding is reused for each non-first pre-coding skipped. In non-I pictures, each non-first pre-coding is skipped, and the coding result of the first pre-coding is reused for each non-first pre-coding skipped, provided a set of conditions related to first pre-coding is satisfied. The experimental results demonstrate that DNFPS has a negligible impact on coding efficiency under All Intra (AI) configurations, and coding runtime is reduced by 16 %.

During and following the global COVID-19 pandemic, the use of screen content coding applications such as large-scale cloud office, online teaching, and teleconferencing has surged. The vast amount of online data generated by these applications, especially online teaching, has become a vital source of Internet video traffic. Consequently, there is an urgent need for low-complexity online teaching screen content (OTSC) coding techniques. Energy-efficient low-complexity green coding techniques for OTSC, named GCOTSC, are proposed based on the unique characteristics of OTSC. In the inter-frame prediction mode, the input frames are first divided into visually constant frames (VCFs) and non-VCFs using a VCF identifier. A new VCF mode has been proposed to code VCFs efficiently. In the intra-frame prediction mode, a heuristic multi-type least probable option skip mode based on static and dynamic historical information is proposed. Compared with the AVS3 screen content coding algorithm, using the typical online teaching screen content and AVS3 SCC common test condition, the experimental results show that the GOTSC achieves an average 59.06% reduction of encoding complexity in low delay configuration, with almost no impact on coding efficiency.

String Prediction (SP) is a very efficient screen content coding (SCC) tool. In SP, the self-referencing string plays an important role to improve coding efficiency. But general self-referencing string has the problem of very low pixel copying throughput and is prohibited in the non-self-referencing based SP which has been adopted in the third-generation Audio Video Standard (AVS3). To overcome the problem and bring back the coding gain of self-referencing string, a line-based self-referencing string (LSRS) enabled SP technique is proposed. Moreover, to keep the pixel copying throughput and coding complexity of LSRS enabled SP the same as non-self-referencing based SP, an unbroken-line decomposition algorithm is presented to decompose an LSRS into multiple non-self-referencing strings. In this way, LSRS can be treated in the same way as a non-self-referencing string with the best trade-off between coding efficiency and complexity. Compared with non-self-referencing based SP, using AVS3 reference software HPM, for twelve SCC common test condition YUV test sequences in text and graphics with motion category and mixed content category, the proposed LSRS technique achieves the average Y BD-rate reduction of 0.81% and 0.59% as well as the maximum Y BD-rate reduction of 2.04% and 1.31% for All Intra and Low Delay configurations, respectively, with almost no additional encoding and decoding complexity. The proposed LSRS enabled SP technique has been adopted in AVS3.

The third generation of Audio Video coding Standard (AVS3) adopts multi-type tree, including Quad-Tree (QT), Binary-Tree (BT) plus Extend Quad-Tree (EQT), partition structure. During multi-type tree recursive pre-coding process of encoding, each Coding Units (CU) of each Size at each Location (CUeSeL) of the picture is repeatedly pre-coded many times when visiting each branch of each tree. Although there is early termination mechanism, the number of visit for each CUeSeL is still very high and up to more than twenty. This process brings very high coding complexity. To reduce the coding complexity of AVS3, this paper proposes a method named non-first pre-coding skip (NFPS) for CUeSeL. The contributions of the paper are as follows. In I picture, for a CU with top-left location (x, y), if both x and y are multiples of 64, then each non-first pre-coding is skipped and the coding result of the first pre-coding is reused for each non-first pre-coding. In non-I picture, for a CU with top-left location (x, y), if both x and y are multiples of 64 and a set of conditions is satisfied, then each non-first pre-coding is skipped and the coding result of the first pre-coding is reused for each non-first pre-coding. The experimental results demonstrate that NFPS has a negligible impact on coding efficiency under All intra (AI), Low Delay B (LDB), Random Access (RA) configurations, and coding runtime is reduced by 6% for AI and almost not changed for LDB and RA.

String prediction (SP) is a very efficient screen content coding (SCC) tool which has been adopted in the third generation of Audio Video Standard (AVS3). It is observed that two special types of strings occur frequently. To further improve the coding efficiency for SCC on top of the original SP, a new variation of SP named Equal-value-string and Copy-above-string based SP (ECSP) is proposed. An ECSP coding unit uses only three types of strings: Equal-value-string, Copy-above-string, and Unpredictable-pixel-string. Compared with the AVS3 reference software HPM9.0 with ECSP disabled, using AVS3 SCC Common Test Condition and YUV 4:2:0 test sequences, the proposed technique achieves an average Y BD-rate reduction of 5.54 and 3.01% for All Intra and Low Delay configurations, respectively, with low additional encoding and decoding complexity. The proposed ECSP has been adopted in the AVS3 standard.

Driven by growing applications that use computer screens as interfaces for daily remote interactions, almost all current video coding standards have included screen content coding (SCC) tools. Recently, an efficient SCC tool called intra string copy (ISC) has been adopted in the third-generation of audio video coding standard in China (AVS3). ISC has two coding unit (CU) level sub-modes: fully-matching-string and partially-matching-string based string prediction (FPSP) sub-mode and equal-value-string, unit-basis-vector-string, and unmatched-pixel-string based string prediction (EUSP) sub-mode. To further improve the coding efficiency of SCC, this paper proposes four enhancement techniques of ISC (EISC), including CU partition improvements, point vector (PV) relocation and reactivation, line-based overlapping string prediction, and an optimized coding method for string length in the EUSP sub-mode. Compared with the latest AVS3 reference software HPM with EISC disabled, using AVS3 SCC common test condition and YUV test sequences in text and graphics with motion and mixed content categories, the proposed technique achieves an average Y BD-rate reduction of 2.39% and 1.49% for all intra (AI) and low-delay B (LDB) configurations, respectively, with low additional encoding complexity and almost no additional decoding complexity. All proposed ISC enhancement techniques have been adopted in AVS3.

An efficient SCC tool named Intra String Copy (ISC) has been proposed and adopted in AVS3 recently. ISC has two CU-level sub-modes: FPSP (fully-matching-string and partially-matching-string based string prediction) sub-mode and EUSP (equal-value-string, unit-basis-vector-string and unmatched-pixel-string based string prediction) sub-mode. Compared with the latest AVS3 reference software HPM with SCC tools disabled, using AVS3 SCC Common Test Condition and YUV test sequences in text and graphics with motion (TGM) and mixed content (MC) categories, the proposed tool achieves an average Y BD-rate reduction of 57.7%/39.5% and 77.2%/57.9% for TGM and MC in All Intra (AI)/Low Delay B(LDB) configurations, respectively, with low additional encoding complexity and almost the same decoding complexity.

In the past, string prediction (also known as string matching) was applied only to RGB and YUV 4:4:4 format screen content coding. This paper proposes a string prediction approach to 4:2:0 format screen content coding implemented in the third generation of Audio Video Standard (AVS3) in China. String prediction is applied to both YUV CU and Y CU. To further improve the coding performance, several improved technicals of string prediction are presented, including a mixed string searching strategy for finding the optimal reference string, a joint picture-level, CU-level, and pixel-level early termination strategy to reduce coding complexity, and two effective coding methods for string prediction parameters. For low-complexity hardware implementation of string prediction decoder, the memory access bandwidth is reduced by introducing string constraints. Meanwhile, string prediction reuses the reference pixel buffer of intra block copy (IBC). Compared with the newest AVS3 reference software HPM7.0 with string prediction disabled, the proposed string prediction approach achieves up to 18.48% Y BD-rate reduction. Using AVS3 Screen Content Coding (SCC) Common Test Condition and YUV test sequences in Text and Graphics with Motion category, the proposed technique achieves an average Y BD-rate reduction of 10.33%, 8.47%, 6.91% for All Intra (AI), Random Access (RA) and Low Delay (LD) configurations, respectively, with low additional encoding and decoding complexity. The proposed string prediction approach has been adopted in the newest AVS3 reference software HPM7.0.

Screen contents have become a popular image type driven by the growing market for transferring display screen between devices, especially mobile devices. Due to the ultra-high quality display featured in most of nowadays mobile devices, lossless screen content coding (SCC) is usually required or preferred. Mobile devices also require ultra-low power consumption in all tasks including SCC. To address these issues, this paper proposes an ultra-low coding complexity technique based on string matching for high efficiency lossless SCC. The technique covers three major coding phases of fast searching, prediction, and entropy coding. Condensed hash table (CHT) based fast searching is proposed to speed-up reference string searching process. Coplanar prediction (CP) and predictor-dependent residual (PDR) are presented to first efficiently predict an unmatchable pixel using multiple neighboring pixels and then further reduce the entropy of prediction residuals. To achieve a good trade-off between coding complexity and efficiency, 4-bit-aligned variable length code (4bVLC) and byte-aligned multi-variable-length-code (BMVLC) are proposed to code the prediction residuals and three string matching parameters, respectively. For 184 screen content images commonly used, compared with X265 and PNG in the default configuration and lossless mode, the proposed technique achieves 35.67% less total compressed bytes with only 0.96% encoding and 1.54% decoding runtime, and 10.04% less total compressed bytes with only 6.83% encoding and 24.32% decoding runtime, respectively. The proposed technique also outperforms X265 and PNG in all other configurations. For twelve HEVC-SCC CTC images, compared with PNG in fast, default and slow configurations and X265 in ultrafast and default configurations, the proposed technique shows significant advantage with both high coding efficiency and ultra-low coding complexity.

目的 随着云计算和移动互联网技术的飞速发展,屏幕图像编码已成为视频压缩领域新的研究热点.帧内块复制(intra block copy,IBC)算法是屏幕内容编码(screen content coding,SCC)中的核心算法,已经成为高效视频编码(high efficiency video coding,HEVC)等标准中屏幕内容编码的重要组成部分.为了进一步消除IBC算法中的位移矢量(displacement vector,DV)参数编码的冗余,根据DV参数具有的固有帧内块匹配特性和相关性,提出了一种DV参数编码算法.方法 首先,对待编码DV采用邻近块与最近块相结合的DV预测编码方案,进一步提高DV的预测编码效率;然后,对预测编码效率不高的DV,提出了一种基于区域划分与调整的高效DV直接编码方案.对SCC标准测试数据集合中的17个测试数据集合的3种编码配置从编码效率和复杂度两方面进行了评测.结果 实验结果表明,对于SCC标准测试数据集合中移动的文字和图形序列类别,在全帧内、随机接入和低延迟这3种有损编码配置下,提出的DV算法与HEVC SCC中的IBC算法相比,在编解码复杂度没有任何影响的前提下,Y分量的BD(Bj?ntegaard Delta)-rate平均降低率分别为1.04％、0.87％和0.93％,全帧内配置下Y分量的BD-rate降低率可达2.99％.结论 本文方法优于HEVC-SCC中的IBC算法中的DV编码算法,能有效提升编码效率.

In the past, string prediction (also known as string matching) was applied only to RGB and YUV 4:4:4 format screen content coding. This paper proposes a string prediction approach to 4:2:0 format screen content coding implemented in the third generation of Audio Video Standard (AVS3) in China. String prediction is applied to both YUV CU and Y CU. To further improve the coding performance, several improved technicals of string prediction are presented, including a mixed string searching strategy for finding the optimal reference string, a joint picture-level, CU-level, and pixel-level early termination strategy to reduce coding complexity, and two effective coding methods for string prediction parameters. For low-complexity hardware implementation of string prediction decoder, the memory access bandwidth is reduced by introducing string constraints. Meanwhile, string prediction reuses the reference pixel buffer of intra block copy (IBC). Compared with the newest AVS3 reference software HPM7.0 with string prediction disabled, the proposed string prediction approach achieves up to 18.48% Y BD-rate reduction. Using AVS3 Screen Content Coding (SCC) Common Test Condition and YUV test sequences in Text and Graphics with Motion category, the proposed technique achieves an average Y BD-rate reduction of 10.33%, 8.47%, 6.91% for All Intra (AI), Random Access (RA) and Low Delay (LD) configurations, respectively, with low additional encoding and decoding complexity. The proposed string prediction approach has been adopted in the newest AVS3 reference software HPM7.0.

In the past, string prediction (also known as string matching) was applied only to RGB and YUV 4:4:4 format screen content coding. This paper proposes a string prediction approach to 4:2:0 format screen content coding implemented in the third generation of Audio Video Standard (AVS3) in China. String prediction is applied to both YUV CU and Y CU. To further improve the coding performance, several improved technicals of string prediction are presented, including a mixed string searching strategy for finding the optimal reference string, a joint picture-level, CU-level, and pixel-level early termination strategy to reduce coding complexity, and two effective coding methods for string prediction parameters. For low-complexity hardware implementation of string prediction decoder, the memory access bandwidth is reduced by introducing string constraints. Meanwhile, string prediction reuses the reference pixel buffer of intra block copy (IBC). Compared with the newest AVS3 reference software HPM7.0 with string prediction disabled, the proposed string prediction approach achieves up to 18.48% Y BD-rate reduction. Using AVS3 Screen Content Coding (SCC) Common Test Condition and YUV test sequences in Text and Graphics with Motion category, the proposed technique achieves an average Y BD-rate reduction of 10.33%, 8.47%, 6.91% for All Intra (AI), Random Access (RA) and Low Delay (LD) configurations, respectively, with low additional encoding and decoding complexity. The proposed string prediction approach has been adopted in the newest AVS3 reference software HPM7.0.

Screen content coding (SCC) is increasingly used in mobile devices, and ultralow coding complexity is required for low power consumption. This article proposes an ultralow complexity string matching approach to SCC. The proposed approach has two essential features: 1) allowing only two types of most effective reference strings that have two most frequently occurring values of offset (i.e., displacement vector) named dual unity offset and 2) fully optimized coding of string matching parameters for maximum coding efficiency. Since at most only two offset values and corresponding reference string positions are allowed for any current string being coded, the optimal reference string searching process is extremely simple and only needs to select the best string from at most two candidates. Moreover, in the proposed approach, the value ranges of string matching parameters, i.e., the string offset vector and string length are very limited, resulting in few bits to code the two parameters and high coding efficiency. Compared with the AVS3 reference software HPM7.0 with IBC disabled, using AVS3 SCC common test condition and YUV test sequences in text and graphics with motion category, the proposed technique achieves Y average BD-rate reduction of 10.8% and 5.8% for all intra (AI) and low-delay B (LDB) configurations, respectively, at ultralow encoding and decoding complexity.

Alpha channel is being applied in an increasing number of mobile web applications on mobile devices that require ultra-low power consumption in all cases including compute-intensive video encoding and decoding. Thus, we propose an ultra-low coding complexity and high efficiency alpha channel lossless coding approach. A novel coding framework and four new coding schemes are proposed for alpha channel coding. The framework fuses a string matching technique and a proposed prediction coding scheme named bit-depth preserving prediction (BDPP) together to reduce the correlations within and between repeated identical patterns and neighboring pixels. To achieve a good tradeoff between complexity and efficiency, either the unmatchable bytes are coded directly or the BDPP residuals of unmatchable bytes are coded by a proposed bytewise entropy coding scheme named 0.5-1-2byte-size-code. The other string matching parameters are coded by another proposed bytewise entropy coding scheme named byte-size multi-variable-length-code. To speed up the string-matching search, we apply a fast string search scheme that combines special position search and hash-based search. For the selected typical 236 alpha test images, compared with x265 in the fastest configuration and lossless mode, the proposed lossless approach achieves 14.33% less total compressed bytes with only 2.75% encoding and 1.83% decoding runtime. The proposed approach also outperforms the conventional lossless coding techniques such as LZ4HC, ZLIB, and PNG.

利用串匹配算法分析计算机屏幕图像和互联网图像的图像编码的编码参数offset的统计特性,以LZ4 HC为具体实现基础,提出了一种基于串匹配的offset循环映射屏幕图像编码(ORMSM)算法.该算法主要包括对偏移量参数offset采用1到图像宽度w之间的循环映射和对偏移量参数offset进行w的特殊位置搜索2个部分.将ORMSM算法与SMH-PLC,PNG,HEVC(x265)算法相比较,实验结果表明,ORMSM算法具有明显的高性能和超低复杂度.