Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach
CoRR(2024)
摘要
Image outpainting aims to generate the content of an input sub-image beyond
its original boundaries. It is an important task in content generation yet
remains an open problem for generative models. This paper pushes the technical
frontier of image outpainting in two directions that have not been resolved in
literature: 1) outpainting with arbitrary and continuous multiples (without
restriction), and 2) outpainting in a single step (even for large expansion
multiples). Moreover, we develop a method that does not depend on a pre-trained
backbone network, which is in contrast commonly required by the previous SOTA
outpainting methods. The arbitrary multiple outpainting is achieved by
utilizing randomly cropped views from the same image during training to capture
arbitrary relative positional information. Specifically, by feeding one view
and positional embeddings as queries, we can reconstruct another view. At
inference, we generate images with arbitrary expansion multiples by inputting
an anchor image and its corresponding positional embeddings. The one-step
outpainting ability here is particularly noteworthy in contrast to previous
methods that need to be performed for N times to obtain a final multiple
which is N times of its basic and fixed multiple. We evaluate the proposed
approach (called PQDiff as we adopt a diffusion-based generator as our
embodiment, under our proposed Positional Query scheme) on
public benchmarks, demonstrating its superior performance over state-of-the-art
approaches. Specifically, PQDiff achieves state-of-the-art FID scores on the
Scenery (21.512), Building Facades (25.310), and WikiArts
(36.212) datasets. Furthermore, under the 2.25x, 5x and 11.7x
outpainting settings, PQDiff only takes 40.6%, 20.3% and
10.2% of the time of the benchmark state-of-the-art (SOTA) method.
更多查看译文
关键词
Diffusion models,image outpainting
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要