The problem of proper identification of the color space associated with digital luma and chroma data has been widely reported by video processing professionals.The problem arises from confusing, and sometimes conflicting, statements regarding color space usage and description.Although standards allow for the carriage of descriptive metadata regarding color space, some applications do not require the presence of such metadata.Those standards typically recite assumptions on color space that should be followed in the absence of embedded color space descriptors.The unfortunate result of this approach is a state of confusion in the industry, and consequently the possibility of errors in rendering the output of decoded video and images.Our work represents the first known attempt to determine color space directly from luma/chroma pixel data, and provides an alternative to sole reliance on potentially missing or incorrect metadata, or weakly followed defaults.Although a color space is defined by many parameters, such as primary chromaticity, transfer characteristics and matrix coefficients, we chose to focus on determining which standard for matrix coefficients had been used to create a given luma/chroma image.We addressed the problem via deep convolutional neural networks (DCNNs), trained on millions of images.Our results are encouraging, and suggest that DCNNs can be used to solve this ill-posed problem.

Deep convolutional neural networks have been successfully deployed by large, well-funded teams, but their wider adoption is often limited by the cost and schedule ramifications of their requirement for massive amounts of labeled data. We address this problem through the use of a parameterized synthetic image generator. Our approach is particularly novel in that we have been able to fine tune the generator’s parameters through the use of a generative adversarial network. We describe our approach, and present results that demonstrate its potential benefits. We demonstrate the PSIG-GAN by creating images for training a DCNN to detect the existence and location of weeds in lawn grass.

We present a novel approach to video coding that can dramatically reduce decoder DRAM bandwidth requirements while incurring a minimal reduction in compression efficiency, and which may yield an increase in compression efficiency under certain circumstances. Our approach is based on the principle that areas of video pictures where there is high motion are typically captured with significant blur along the direction of motion. This blur permits the judicious use of reduced resolution reference pictures for prediction without significantly reducing prediction quality. Our approach makes it feasible to limit worst-case DRAM bandwidth through the use of reasonably sized on-chip caches for pixel data, which can lead to provably compliant real-time behavior. The reductions in DRAM bandwidth can be expected to yield commensurate reductions in DRAM power dissipation, and consequently improvements in battery life for mobile devices. Although we concentrate our analysis on decoders, our approach can yield even greater advantages for encoders, which require additional bandwidth for motion estimation. We show that the compression efficiency of our encoder can approach that of a standard reference encoder on natural video sequences, but that it may fall short by a modest amount on pathological synthetic sequences.

Prior to the advent of ITU-R Recommendation BT.709 the overwhelming majority of compressed digital video and imagery used the colorspace conversion matrix specified in ITU-R Recommendation BT.601. The introduction of high definition video formats led to the adoption of Rec. BT.709 for use in colorspace conversion by new systems, and this resulted in confusion in the industry. Specifically, video decoders may not be able to determine the correct matrix to use for converting from the luma/chroma representation used for coding, to the Red-Green-Blue representation needed for display. This confusion has led to a situation where some viewers of decompressed video streams experience subtle, but noticeable, errors in coloration. We have successfully developed and trained a deep convolutional neural network to address this heretofore unsolved problem. We obtained outstanding accuracy on ImageNet data, and on YouTube video frames, and our work can be expected to lead to more accurate color rendering delivered to users of digital imaging and video systems.

We present a novel approach to video coding that can dramatically reduce decoder DRAM bandwidth requirements while incurring a minimal reduction in compression efficiency, and which may yield an increase in compression efficiency under certain circumstances. Our approach is based on the principle that areas of video pictures where there is high motion are typically captured with significant blur along the direction of motion. This blur permits the judicious use of reduced resolution reference pictures for prediction without significantly reducing prediction quality. Our approach makes it feasible to limit worstcase DRAM bandwidth through the use of reasonably sized onchip caches for pixel data, which can lead to provably compliant real-time behavior. The reductions in DRAM bandwidth can be expected to yield commensurate reductions in DRAM power dissipation, and consequently improvements in battery life for mobile devices. Although we concentrate our analysis on decoders, our approach can yield even greater advantages for encoders, which require additional bandwidth for motion estimation. We show that the compression efficiency of our encoder can approach that of a standard reference encoder on natural video sequences, but that it may fall short by a modest amount on pathological synthetic sequences.