Adapt and Diffuse: Sample-adaptive Reconstruction via Latent Diffusion Models
CoRR(2023)
摘要
Inverse problems arise in a multitude of applications, where the goal is to
recover a clean signal from noisy and possibly (non)linear observations. The
difficulty of a reconstruction problem depends on multiple factors, such as the
structure of the ground truth signal, the severity of the degradation and the
complex interactions between the above. This results in natural
sample-by-sample variation in the difficulty of a reconstruction task, which is
often overlooked by contemporary techniques. Our key observation is that most
existing inverse problem solvers lack the ability to adapt their compute power
to the difficulty of the reconstruction task, resulting in subpar performance
and wasteful resource allocation. We propose a novel method that we call
severity encoding, to estimate the degradation severity of noisy, degraded
signals in the latent space of an autoencoder. We show that the estimated
severity has strong correlation with the true corruption level and can give
useful hints at the difficulty of reconstruction problems on a sample-by-sample
basis. Furthermore, we propose a reconstruction method based on latent
diffusion models that leverages the predicted degradation severities to
fine-tune the reverse diffusion sampling trajectory and thus achieve
sample-adaptive inference times. Our framework acts as a wrapper that can be
combined with any latent diffusion-based baseline solver, imbuing it with
sample-adaptivity and acceleration. We perform numerical experiments on both
linear and nonlinear inverse problems and demonstrate that our technique
greatly improves the performance of the baseline solver and achieves up to
10× acceleration in mean sampling speed.
更多查看译文
关键词
latent diffusion
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要