Outpainting Natural Scenery Images by Fusing Forecasting Information

Abstract Image outpainting receives less attention due to the challenges of predicting the spatial content consistency and maintaining the high quality in generated images. All-sides image prediction is a generalized task in outpainting, aiming to extrapolate the outside content for all sides of an image. It is a challenging task to maintain spatial and semantic consistency between the original input and the generated multi-step regions. In this paper, we embed a novel Multi-view Recurrent Content Transfer module into an Encoder-Decoder architecture for long-range all-side image outpainting. A multi-head attention mechanism is leveraged to fuse information from different representation sub-spaces at different positions to enhance the consistency of generated images and the original input. Our model could obtain sufficient temporal information in predicting the extended feature maps, which improves the quality of long-range images extrapolation. We experimentally demonstrate that our proposed method could produce visually appealing results for outside image outpainting against the state-of-the-art image inpainting and outpainting approaches. Modelling the temporal relationship could help generate the outside regions and reconstruct the input regions smoothly and realistically. In addition, an attempt to possibly allow for arbitrary output resolutions is supported as well.