DaTMe
A training-free diffusion model acceleration method that constructs a dynamics-aware importance map via Tweedie denoising estimation. It adaptively identifies anchor tokens and safe merge candidates, reducing redundant computation while maintaining generation quality.
Timeline (last 90 days)
The researchers propose Dynamics-aware Token Merging (DaTMe), a training-free acceleration strategy that bases token merging decisions on the dynamics of the diffusion process.
DaTMe constructs a dynamics-aware importance map from the Tweedie denoising estimate, fusing two types of signals: a temporal signal (the degree of change in the predicted clean image between adjacent steps) and a structural signal (local high-frequency content such as boundaries and fine details).
At a given compression ratio, DaTMe adaptively partitions tokens according to the importance map into retained anchors and merged safe candidates.
DaTMe was experimentally validated on two diffusion models, PixArt-a and FLUX.
Experimental results show that DaTMe can maintain visual fidelity across the entire image, rather than only protecting foreground regions.
The core argument of DaTMe is that diffusion dynamics can serve as an effective basis for token-level merging decisions.
DaTMe provides an acceleration path that requires no retraining and can be directly inserted into existing diffusion pipelines.
All Facts (7)
The researchers propose Dynamics-aware Token Merging (DaTMe), a training-free acceleration strategy that bases token merging decisions on the dynamics of the diffusion process.
50%UnverifiedDaTMe constructs a dynamics-aware importance map from the Tweedie denoising estimate, fusing two types of signals: a temporal signal (the degree of change in the predicted clean image between adjacent steps) and a structural signal (local high-frequency content such as boundaries and fine details).
50%UnverifiedAt a given compression ratio, DaTMe adaptively partitions tokens according to the importance map into retained anchors and merged safe candidates.
50%UnverifiedDaTMe was experimentally validated on two diffusion models, PixArt-a and FLUX.
50%UnverifiedExperimental results show that DaTMe can maintain visual fidelity across the entire image, rather than only protecting foreground regions.
50%UnverifiedThe core argument of DaTMe is that diffusion dynamics can serve as an effective basis for token-level merging decisions.
50%UnverifiedDaTMe provides an acceleration path that requires no retraining and can be directly inserted into existing diffusion pipelines.
50%