LIPT: Latency-aware Image Processing Transformer
arxiv(2024)
摘要
Transformer is leading a trend in the field of image processing. Despite the
great success that existing lightweight image processing transformers have
achieved, they are tailored to FLOPs or parameters reduction, rather than
practical inference acceleration. In this paper, we present a latency-aware
image processing transformer, termed LIPT. We devise the low-latency proportion
LIPT block that substitutes memory-intensive operators with the combination of
self-attention and convolutions to achieve practical speedup. Specifically, we
propose a novel non-volatile sparse masking self-attention (NVSM-SA) that
utilizes a pre-computing sparse mask to capture contextual information from a
larger window with no extra computation overload. Besides, a high-frequency
reparameterization module (HRM) is proposed to make LIPT block
reparameterization friendly, which improves the model's detail reconstruction
capability. Extensive experiments on multiple image processing tasks (e.g.,
image super-resolution (SR), JPEG artifact reduction, and image denoising)
demonstrate the superiority of LIPT on both latency and PSNR. LIPT achieves
real-time GPU inference with state-of-the-art performance on multiple image SR
benchmarks.
更多查看译文
AI 理解论文
溯源树
样例
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要