An UHD 4K@120fps Hardware for the VVC Prediction Refinement with Optical Flow

This work presents a hardware design for the Prediction Refinement with Optical Flow (PROF). The PROF is incorporated in the Affine Prediction from the recent Versatile Video Coding (VVC) standard. While the Affine Prediction allows the representation of complex motions in the scene as rotation, zoom, and skew, by performing motion compensation over each subblock of the current block, the PROF enhances video encoding efficiency by generating the virtual rotation of each subblock. However, the PROF supports a wide range of block sizes, demanding intensive computational processing and therefore requiring dedicated hardware design. So, the PROF Hardware Design was developed to process one 4×4 subblock at each clock cycle. The synthesis results for TSMC 40nm technology show that the PROF Hardware Design can process 120 frames per second of Ultra-High Definition UHD 4K resolution videos while dissipating 0.98mW when operating at 62MHz. This work also presents integration between the PROF Hardware Design and an Affine Motion Compensation hardware. To the best of author's knowledge, this is the first hardware design for PROF processing in the literature.