Dual-Stream Diffusion Net for Text-to-Video Generation
CoRR(2023)
摘要
With the emerging diffusion models, recently, text-to-video generation has
aroused increasing attention. But an important bottleneck therein is that
generative videos often tend to carry some flickers and artifacts. In this
work, we propose a dual-stream diffusion net (DSDN) to improve the consistency
of content variations in generating videos. In particular, the designed two
diffusion streams, video content and motion branches, could not only run
separately in their private spaces for producing personalized video variations
as well as content, but also be well-aligned between the content and motion
domains through leveraging our designed cross-transformer interaction module,
which would benefit the smoothness of generated videos. Besides, we also
introduce motion decomposer and combiner to faciliate the operation on video
motion. Qualitative and quantitative experiments demonstrate that our method
could produce amazing continuous videos with fewer flickers.
更多查看译文
关键词
generation,dual-stream,text-to-video
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要