In DNA data storage, data is retrieved from an unordered pool of DNA strands, posing challenges for sequential access and necessitating strong forward error correction (FEC). We propose a progressive video playback framework that overcomes these challenges by (a) hierarchically chunking a fragmented MP4 (fMP4) video file at frame granularity, prioritizing key frames in the encoding, and (b) using a decay-weighted Raptor fountain code biased toward early frames. By decaying the coding priority over frame index, the decoder can recover initial frames with fewer reads, enabling near real-time playback of the video from DNA. We integrate an inner ReedSolomon (RS) code and sequence checksum to correct sequencing errors, so that the fountain decoder operates on an effective erasure channel. Simulation results demonstrate that our biased coding dramatically reduces recovery time for first frames, allowing the video frame rate to ramp up to 30 fps as sequencing continues. This work is the first to achieve progressive video decoding from DNA storage using Raptor codes, extending recent advances in progressive image decoding in this domain.
In this paper, we propose an intra prediction mode derivation method for beyond VVC. Our proposed method utilizes template matching-based block vectors to identify reference blocks with similar characteristics to the current block, aiming to derive more accurate prediction modes. Unlike conventional mode derivation method such as in enhanced compression model (ECM) that rely solely on neighboring samples, our approach leverages block-level similarities while avoiding additional signaling overhead. The key feature of our approach is two-fold: (1) utilizing multiple template matching candidates to identify similar reference blocks within the current frame, and (2) evaluating prediction modes using the neighboring samples of the current block for multiple reference blocks. By combining these two aspects, our method leverages block-level similarities while maintaining computational efficiency through the reuse of existing reference samples. Experimental results under JVET common test conditions show that the proposed method achieves 0.10 % bitrate reduction compared with ECM-14.
This paper reports on the process of composing a set of test sequences for evaluation of responses to the Call for Evidence on video compression with capability beyond VVC currently prepared in the Joint Video Experts Team of ISO/IEC JTC 1/SC 29 and ITU-T SG21. The goal is to compose a test set covering the intended application spaces with associated challenging target bitrate values. These values are selected such that candidate coding schemes can be evaluated without saturation effects in the low or high visual quality ranges. After pre-screening candidate sequences, a novel succession test method is applied to efficiently determine viable rate points with distinct quality levels. The outcome is evaluated using the Degradation Category Rating (DCR) method. The applied succession test method was found to be useful and efficient for assessing the draft rate point allocation for the test sequences. Some refinement of the method might be explored in future experiments.
Efficient inter prediction is fundamental to video coding, as it reduces temporal redundancy by leveraging information from neighboring frames. The quality of these reference frames critically impacts prediction accuracy. While numerous neural network-based methods have been proposed to enhance reference frames, most rely on convolutional architectures that excel at modeling local spatial features but struggle with complex motion due to their limited receptive fields. Improving their performance often demands higher model complexity, resulting in a suboptimal trade-off between performance and computational complexity. To address these limitations, we present the first exploration of a Mamba-based network that precisely captures spatiotemporal information with linear computational cost for inter prediction enhancement. Specifically, a Mamba-based video frame interpolation model is employed to synthesize a virtual reference frame using a pair of previously reconstructed frames selected from the decoded picture buffer as input. The synthesized frame is then inserted into the reference picture list and utilized during motion estimation and compensation. Moreover, we propose the QP-Progress training strategy to better align the interpolation network with the encoder’s operating conditions and improve robustness to compression artifacts, without sacrificing motion modeling accuracy. This strategy progressively incorporates higher-QP compressed frames during training while retaining low-QP samples to maintain the motion modeling capacity acquired in the early training stages. Experimental results show that our method achieves coding efficiency gains of 4.46%/4.12%/4.20% for Y/U/V components under random access configuration against NNVC-12.0 anchor. Compared with other existing approaches, our method achieves superior performance gain with significantly lower complexity.
This paper introduces a novel intra prediction method aimed at improving the coding efficiency of the Enhanced Compression Model (ECM). The proposed Occurrence-based Intra Coding (OBIC) method determines the optimal intra modes for block coding by analyzing the sample-wise occurrences of intra modes in the neighboring blocks. These occurrences are aggregated into a histogram of occurrences (HoC), from which the most frequent intra modes in the causal neighborhood are identified. The five most frequently-occurring modes’ predictions are then blended with planar prediction to predict the block. The OBIC method was introduced at the 33rd JVET meeting and adopted into the ECM software at the 34th meeting. The proposed method achieves average BD-Rate savings of 0.10% for the All-Intra (AI) configuration and 0.05% for the Random Access (RA) configuration compared to version 12.0 of the ECM software.
This paper presents a comprehensive performance evaluation of objective quality assessment methods for evaluating the visual quality of compressed light field content. Leveraging a carefully curated dataset and a subjective experiment, we examine the performance of 21 widely-used image and video quality metrics covering both full-reference and no-reference techniques. The performance of metrics is assessed using correlation measures and validated through statistical significance testing. Our findings identify the most reliable metrics for light field quality assessment and highlight persistent challenges in capturing subtle perceptual degradations. The study contributes empirical evidence and methodological guidance to support ongoing standardization efforts within the JPEG Pleno framework.
High Dynamic Range (HDR) video standards offer improvements in brightness, contrast, and color reproduction for enhanced visual realism. However, diverse displays and inconsistent implementations lead to perceptual variations despite having a theoretically standardized luminance encoding with the perceptual quantizer (PQ). This paper examines how global luminance-related display parameters - peak luminance, black level, and transfer function - influence human perception and preference of HDR video. A signal-based display simulation generated 24 different display configurations applied to five HDR video sequences, which were displayed on two Sony BVM-HX310 reference screens. Thirty-two subjects assessed these sequences using two double-stimulus sensory evaluation tests: a hedonic test for individual preference, and an analytical test assessing perceived contrast, brightness, detail, saturation, and realness via a comparison rating scale. Multidimensional analyses revealed that global luminance-related parameters affect perception, with contrast emerging as the most influential factor for preference and brightness preferences varying among subjects, while perceived detail was enhanced by both contrast and brightness. Using the results from the subjective test, ColorVideoVDP was evaluated for our use case and the results showed that this model can capture the trends in subjective preference despite not being trained for such use cases. These findings offer empirical insights into how display properties shape the subjective experience of HDR video. - Supplementary data: https://github.com/JuliusPre/Sens-Eval-HDR-Disp
Learning-based methods have demonstrated significant efficacy in video codecs in recent years. However, practical implementation necessitates fulfilling several constraints. The neural networks used must be exceptionally low in computational complexity and require minimal area cost in hardware. They must be designed to employ fixed-point operations rather than floating-point operations to ensure decoder-side compliance. Practical decoder hardware implementations also require block-by-block processing to facilitate pipelining. Crucially, even after these simplifications, the compression improvement resulting from such a tool must be big enough to justify the implementation cost. In this study, we present a neural network based in-loop filtering tool for AVM that adheres to all these constraints while still achieving about −1.12% PSNR-Y improvement and −1.89% VMAF improvement in a random-access setting, with as much as −4.71% VMAF improvement for 4K content. We also present a lighter version of the neural network which reduces complexity to less than half, but retains more than half of the compression improvement.
With the increasing demand for immersive video experiences, efficient compression of multiview content has become crucial for reducing storage and transmission costs. The introduction of stereoscopic video support on head-mounted displays, along with the emergence of smartphones capable of easily capturing stereoscopic videos, further highlights the need for optimized encoding solutions. Although the efficient but computationally intensive Multi-View High Efficiency Video Coding (MV-HEVC) standard has been available since 2014, only recently has x265—a real-time open-source HEVC en-coder—introduced support for multiview encoding. This work (i) evaluates the encoding efficiency of multiview x265 across all presets, and compares it with MV-HEVC, (ii) proposes a perceptual quality-aware preset selection method, and (iii) conducts a comparative study on single-view and stereoscopic videos. The results show that the slowest x265 preset (placebo) leads to 8.5% and 35.3% higher bitrates than MV-HEVC for the same objective quality (PSNR) on natural and synthetic videos, respectively. However, MV-HEVC is significantly slower, with encoding times 46% and 41% longer for the two content types. The proposed preset selection strategy, based on various $\Delta$ AVQT (%) targets, shows the trade-off between perceptual quality and encoding time. Finally, stereoscopic video requires 71.5% and 65% more bitrate than single-view video to achieve the same objective and perceptual quality, respectively, while offering nearly identical perceptual quality at a constant bitrate.
Versatile Video Coding (VVC) is the latest video compression standard designed to meet the demands of highresolution content and diverse applications. One important feature that distinguishes VVC from its predecessor (HEVC) is the addition of Reference Picture Resampling (RPR). RPR allows for adaptive change of the coding resolution and, at the same time, access reference pictures by performing a resampling of reference pictures. This functionality is helpful in real-time transmission scenarios. In this paper, we demonstrate that applying already implemented In-Loop filters, specifically the Sample Adaptive Offset (SAO) and the Adaptive Loop Filter (ALF), to enhance upscaled images within the coding loop can significantly improve image quality. Coding at low resolution while applying in-loop filters at output resolution is shown to be an effective method for exploiting reconstruction errors in the high-resolution domain, while maintaining a reduced complexity in the prediction process. An advantage of the proposed method is that existing tools can be reused with minimal adjustments. Additionally, the proposed method retains a similar latency reduction over full-resolution coding as RPR. This can be attributed to the fact that inloop filters can be applied CTU-parallel. A general weakness of low-resolution coding is that content with very sharp edges, such as screen content, may appear slightly blurred. While we observe a significant visual improvement over RPR with our method, we still notice a visual blur in some sequences. As a solution to this problem, a specified edge enhancement filter is applied. By applying this, we observe further improvements in the performance for content with sharp edges. The proposed method demonstrates an average YUV BD-rate of −5.48%/−5.62%/−5.64% in random access over the VTM anchor in RPR functionality test conditions, as defined in VTM; the reference implementation of VVC by the Joint Video Experts Team (JVET).
We propose a novel Ultra-Low Bitrate Multimodal Generative Face Video Coding Framework for talking face videos. It fully leverages both audio and facial semantic information to maintain relatively high reconstruction quality at ultra-low bitrate. Talking face videos are segmented into “talking” and “silent” portion using an improved Voice Activity Detection (VAD) algorithm. Audio data is transmitted during the “talking” portion using AAC encoding, and video information is transmitted for the rest by extracting facial semantic features. At the decoder side, two generative models namely Interactive Face Video Coding(IFVC) and AniPortrait are adopted to reconstruct video from audio data and facial semantic feature. To alleviate the temporal artifacts introduced by segment-wise generation, a URP-NET frame interpolation is applied. Meanwhile, to improve the subjective quality of the generated video, a frame selection mechanism is proposed to guide the two generative models. Extensive experiments demonstrate that our proposed method achieves superior perceptual quality and temporal consistency at ultra-low bitrate, significantly outperforming existing Generative Face Video Coding(GFVC) approaches. Moreover, we introduce a frame selection mechanism at the decoding stage, which further enhances the overall generation quality.
Autoencoder-based Learned Image Codecs (LICs) typically exhibit similar computational complexities for both encoding and decoding processes. In contrast, the emerging paradigm of Implicit Neural Codecs (INCs) focuses on overfitting compact neural networks to source images, resulting in highly efficient decoders at the expense of prolonged encoding times. These approaches often leverage multiresolution latent representations and employ entropy models that utilize neighborhood context. In this work, we propose a method that harnesses multiresolution neighborhoods to reduce redundancy and improve coding efficiency. Importantly, our approach maintains the computational complexity of both encoding and decoding while achieving up to a 2.7% saving in BD-Rate on the KODAK dataset. Code is available at github.com/caribankai/MRC-Cool-Chic.
Context Adaptive Binary Arithmetic Coding (CABAC) employs simple binarization algorithms such as unary and truncated unary codes, which are highly efficient for encoding small values under geometric distribution. However, these binarization methods become inefficient for low quantization parameters (QP) where larger coefficients are dominant thus violating the geometric distribution assumption. To address this limitation, we propose an adaptive unary and truncated unary binarization method in which output bins vary depending on the predictor. Given a predictor value k, the proposed method encodes unary and truncated unary binaries from the ${k}$-th bin instead of the first bin (i.e. ${k}={0}$). The value of k-th bin determines whether the order of encoding is forward or backward. This approach effectively skips redundant lower flags signaling while performing necessary corrections. The proposed method was integrated on residual coding of the Enhanced Compression Model (ECM) using CABAC probability as a predictor. Experimental results show −0.02%/−0.01%/−0.03% Y-BDBR reduction for all intra (AI) / random access (RA) / low delay B (LDB) configurations under common test conditions over ECM 16.1. The Y-BDBR improvements over VTM-23.8 are −0.03%, −0.01%, −07% for AI, RA, and LDB configuration, respectively. Notably, the proposed method reduces Y-BDBR by −0.13% for all intra under low QP settings for all resolutions, and −0.18% for HD and UHD resolutions.
The proliferation of automated visual analysis calls for compression methods tailored to the unique requirements of Video Coding for Machines (VCM). In this paper, we propose a computationally lightweight post-processing method that is based on a learned component referred to as a task enhancement tile (TET). A TET is spatially tiled over the reconstructed visual data and added to it element-wise. It only requires one addition per pixel in each color channel before the machine task can be applied. Our results with the VVC test model (VTM) demonstrate coding gains of up to $39.0 \%$ for object detection and $29.2 \%$ for instance segmentation on image datasets, while evaluation on a video dataset shows gains of up to $35.2 \%$ for object detection, relative to the VTM anchor. The proposed solution also offers extremely low computational cost, preservation of human-viewable content, full compliance with video coding standards, no requirement for side information transmission from encoder to decoder, and generalization across tasks, models, and encoding parameters.
DNA molecules have been proposed as a support for long-term data storage. This enables a quaternary representation through the four nucleotides prevalent in the genetic material of living beings. Recently, the JPEG Committee launched a Call for Proposals for image coding algorithms in DNA support, retaining the top performer as its starting point for the development of a new standard. However, the process of storage and sequencing of molecules usually results in errors, but this additional challenge was not fully addressed in the initial JPEG DNA solution. This paper proposes a generalized architecture, for which the original JPEG DNA solution is a specific case, that allows for the inclusion of error-correcting capabilities. Moreover, a nucleotide decoding mechanism capable of translating insertion and deletion errors into substitutions is proposed and combined with ReedSolomon codes in the architecture to achieve correct decoding of images even in the presence of substitution and indel errors. The proposed mechanism is observed to achieve superior results in terms of information density when compared to state-of-the-art methods.
3DGS has received increasing attention due to its 3D representation efficiency and rendering fidelity for novel views. However, the lack of efficient data compression significantly limits the practical deployment of 3DGS. While existing compression methods for 3D Gaussian representations have shown promising results, they still suffer from two major limitations: (1) less efficient inter-redundancy modeling for gaussian primitives, (2) compression and adequate compaction are carried out independently. To address the first issue, we propose a wavelet-based triplane representation to further eliminate inter-redundancy among Gaussian primitives. Regarding the latter challenge, For the second challenge, we unify compaction and compression within a cohesive framework, enabling more efficient representation and storage of 3D scenes. Through our proposed solutions, we achieve a better rate-distortion performance for 3DGS data compression. Experimental results show that our method achieves a BDRate reduction of $\mathbf{2 0 \% - 3 0 \%}$ compared to state-of-the-art 3DGS coding approaches, while enabling over 5 times faster in both encoding/decoding time complexity.
In this paper, we propose a novel image compression method for content-based image retrieval in the context of DNA data storage. Storing data on DNA is an extremely promising solution due to its compactness, long-term durability, and energy efficiency. However, its compactness introduces two challenges: the need for efficient data access and the ability to flexibly handle new (and not predefined) types of queries. To address the efficiency challenge, our approach enables direct image retrieval within the DNA domain. To ensure flexibility, we design a compact data identifier that is a semantic representation of the image and serves as a header at the beginning of the DNA strand. Our approach shows high visual and quantitative performance, outperforming state-of-the-art method for various types of query. This highlights that hybridization can be effectively modeled using cosine similarity, without the need for training.
In this paper, we propose a new benchmark dataset for still picture coding performance, named SPCP. SPCP is a large-scale benchmark dataset covering 10 codecs, 310 configurations, up to 101 quality parameters, 28 IQA metrics, and 3 computational times. The codecs are JPEG, MozJPEG, Jpegli, JPEG2000, JPEG XR, JPEG XL, WebP, HEIC, and AVIF-AOM/SVT. Also, we provide a public web interface for comparative analysis of still picture coding performance. Covering classical and newer codecs, SPCP provides a standardized testbed to support reproducible and fair comparison of image encoders. The full dataset is publicly available to accelerate codec development and evaluation in both academic and industrial communities. Our SPCP dataset is available at https://fukushimalab.github.io/spcp/.
While Implicit Neural Representations (INRs) show promise for video compression, existing methods can suffer from a fundamental limitation: their content-agnostic processing fails to adapt to videos’ inherent semantic structure, resulting in suboptimal performance on long-form, multi-scene content. We present the first data-driven adaptive framework specifically for INR-based compression that addresses this architectural flaw. Through systematic empirical analysis, we establish quantitative relationships between scene characteristics and compression requirements, leading to three evidencebased design components: scene-aware segmentation that respects natural content boundaries, complexity-guided dynamic parameter allocation proportional to motion characteristics, and conditional intra-scene splitting for high-motion segments. When implemented with a HiNeRV backbone, our framework reduces encoding time by up to 43% while improving PSNR by up to 1.5 dB, outperforming both monolithic INR approaches and Nirvana, a leading method for long-video INR compression. This work establishes content-awareness as an important principle for practical neural video compression.
Holographic imaging enables true three-dimensional visualization for applications such as augmented and virtual reality; however, the immense data size of high-quality holograms poses a major barrier to their practical adoption. This paper investigates a preprocessing strategy to improve the compression of holographic data using the High-Efficiency Video Coding (HEVC) standard. By downsampling the hologram before encoding and subsequently upsampling it after decoding, we demonstrate that it is possible to achieve better reconstruction quality at low bitrates compared to encoding the full-resolution data. This counterintuitive result basically comes from the reduction in spatial complexity, which allows the HEVC encoder to allocate more bits to preserving critical high-frequency information that would otherwise be lost. Although the hologram phase is highly sensitive to scaling, the overall perceptual quality improves at bitrates below 1 Bpp, with gains of approximately 0.1 in SSIM and 0.015 in VIF. Our work highlights a critical principle in holographic codec design: optimizing the trade-off between spatial complexity and quantization error is crucial for maximizing reconstruction quality, particularly in bandwidth-constrained environments.
