FilterPrompt: Guiding Image Transfer in Diffusion Models
arxiv(2024)
摘要
In controllable generation tasks, flexibly manipulating the generated images
to attain a desired appearance or structure based on a single input image cue
remains a critical and longstanding challenge. Achieving this requires the
effective decoupling of key attributes within the input image data, aiming to
get representations accurately. Previous research has predominantly
concentrated on disentangling image attributes within feature space. However,
the complex distribution present in real-world data often makes the application
of such decoupling algorithms to other datasets challenging. Moreover, the
granularity of control over feature encoding frequently fails to meet specific
task requirements. Upon scrutinizing the characteristics of various generative
models, we have observed that the input sensitivity and dynamic evolution
properties of the diffusion model can be effectively fused with the explicit
decomposition operation in pixel space. This integration enables the image
processing operations performed in pixel space for a specific feature
distribution of the input image, and can achieve the desired control effect in
the generated results. Therefore, we propose FilterPrompt, an approach to
enhance the model control effect. It can be universally applied to any
diffusion model, allowing users to adjust the representation of specific image
features in accordance with task requirements, thereby facilitating more
precise and controllable generation outcomes. In particular, our designed
experiments demonstrate that the FilterPrompt optimizes feature correlation,
mitigates content conflicts during the generation process, and enhances the
model's control capability.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要