Window-Diffusion: Accelerating Diffusion Language Model Inference with Windowed Token Pruning and Caching

Diffusion language models (DLMs) generate text through iterative denoising, but at each denoising step the model still performs attention over the full sequence, including many tokens that remain masked, resulting in substantial redundant computation. Existing acceleration methods typically require retraining the DLM and imposing constrained update patterns to reduce inference cost, which limits their direct applicability to pretrained DLMs. Our token-level analysis reveals pronounced structural locality in DLM inference. In practice, only a small set of tokens near the undecoded prefix tends to drive decoding, while distant undecoded tokens have limited influence and the retained local context can be effectively reused. Motivated by these observations, we propose Window-Diffusion (Code repository: https://github.com/vhicrgit/Window-Diffusion ), a window-based token pruning and caching method for inference. We maintain a local computation window that slides rightward as denoising progresses, and partition undecoded tokens into: (i) active tokens that are computed online, (ii) buffer tokens whose KV states are cached and periodically refreshed, and (iii) far-field tokens that are pruned outside the window. Computation is restricted to active and buffer tokens within the window, while far-field tokens are omitted at each stage. Extensive experiments across diverse benchmarks and model scales demonstrate the effectiveness of our method. Our approach achieves up to 6.6 × speedup with negligible quality degradation.