Efficient Diffusion-Driven Corruption Editor for Test-Time Adaptation
arxiv(2024)
摘要
Test-time adaptation (TTA) addresses the unforeseen distribution shifts
occurring during test time. In TTA, both performance and, memory and time
consumption serve as crucial considerations. A recent diffusion-based TTA
approach for restoring corrupted images involves image-level updates. However,
using pixel space diffusion significantly increases resource requirements
compared to conventional model updating TTA approaches, revealing limitations
as a TTA method. To address this, we propose a novel TTA method by leveraging a
latent diffusion model (LDM) based image editing model and fine-tuning it with
our newly introduced corruption modeling scheme. This scheme enhances the
robustness of the diffusion model against distribution shifts by creating
(clean, corrupted) image pairs and fine-tuning the model to edit corrupted
images into clean ones. Moreover, we introduce a distilled variant to
accelerate the model for corruption editing using only 4 network function
evaluations (NFEs). We extensively validated our method across various
architectures and datasets including image and video domains. Our model
achieves the best performance with a 100 times faster runtime than that of a
diffusion-based baseline. Furthermore, it outpaces the speed of the model
updating TTA method based on data augmentation threefold, rendering an
image-level updating approach more practical.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要