Mitigate Replication and Copying in Diffusion Models with Generalized Caption and Dual Fusion Enhancement
ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)(2023)
摘要
While diffusion models demonstrate a remarkable capability for generating
high-quality images, their tendency to `replicate' training data raises privacy
concerns. Although recent research suggests that this replication may stem from
the insufficient generalization of training data captions and duplication of
training images, effective mitigation strategies remain elusive. To address
this gap, our paper first introduces a generality score that measures the
caption generality and employ large language model (LLM) to generalize training
captions. Subsequently, we leverage generalized captions and propose a novel
dual fusion enhancement approach to mitigate the replication of diffusion
models. Our empirical results demonstrate that our proposed methods can
significantly reduce replication by 43.5
model while maintaining the diversity and quality of generations. Code is
available at https://github.com/HowardLi0816/dual-fusion-diffusion.
更多查看译文
关键词
Generative model,diffusion model,training data privacy,replication mitigation
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要