Chrome Extension
WeChat Mini Program
Use on ChatGLM

Video-P2P: Video Editing with Cross-attention Control

2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2024)

Cited 0|Views141
No score
Abstract
This paper presents Video-P2P, a novel framework for real-world video editing with cross-attention control. While attention control has proven effective for image editing with pre-trained image generation models, there are currently no large-scale video generation models publicly available. Video-P2P addresses this limitation by adapting an image generation diffusion model to complete various video editing tasks. Specifically, we propose to first tune a Text-to-Set (T2S) model to complete an approximate inversion and then optimize a shared unconditional embedding to achieve accurate video inversion with a small memory cost. For attention control, we introduce a novel decoupled-guidance strategy, which uses different guidance strategies for the source and target prompts. The optimized unconditional embedding for the source prompt improves reconstruction ability, while an initialized unconditional embedding for the target prompt enhances editability. Incorporating the attention maps of these two branches enables detailed editing. These technical designs enable various text-driven editing applications, including word swap, prompt refinement, and attention re-weighting. Video-P2P works well on real-world videos for generating new characters while optimally preserving their original poses and scenes. It significantly outperforms previous approaches.
More
Translated text
Key words
Video Editing,Diffusion Model,Attention Map,Image Editing,Inverse Estimation,Reconstruction Ability,Denoising,Convolution Kernel,Inverse Method,Video Content,Latent Features,Temporal Consistency,Original Video,Temporal Coherence,3rd Row,Semantic Consistency
AI Read Science
Must-Reading Tree
Example
Generate MRT to find the research sequence of this paper
Chat Paper
Summary is being generated by the instructions you defined