Learning To Extract Flawless Slow Motion From Blurry Videos

In this paper we introduce the task of generatinga sharp slow-motion video given a low frame rate blurry video. We propose a data-driven approach, where the training data is captured with a high frame rate camera and blurry images are simulated through an averagingprocess. While it is possible to train a neural network to recover the sharp framesfrom their average,there is no guaranteeof the temporal smoothnessfor the formed video, as the frames are estimated independently. To address the temporal smoothness requirement we propose a system with two networks: One, DeblurNet, to predict sharp keyframes and the second, InterpNet, to predict intermediateframes between the generatedkeyframes. A smooth transitionis ensured by interpolatingbetween consecutive keyframes using InterpNet. Moreover, the proposed scheme enablesfurther increase in frame rate without retrainingthe network, by applying InterpNet recursively between pairsof sharpframes. We evaluate the proposed method on several datasets, including a novel dataset captured with a Sony RX V camera. We also demonstrateits performance of increasingtheframe rate up to 20 times on real blurry videos.