SoftCache
A training-free inference acceleration method for diffusion Transformers that uses staleness signals to adaptively modulate sampling step sizes, achieving soft caching rather than binary hard caching, significantly improving speed while preserving generation quality.
Timeline (last 90 days)
The researchers propose SoftCache, which performs soft caching: the margin of the staleness signal determines how far each caching step advances, so that when the margin is large the sampler skips multiple scheduled nodes, and as a refresh approaches the step size gradually shrinks back to the original step size.
The core innovation of SoftCache is turning the caching decision from a discrete binary choice into continuous step-size modulation.
SoftCache is completely training-free and adds no parameters specific to any backbone network, so it can be directly applied to different diffusion Transformer architectures without hyperparameter tuning for each model.
SoftCache relies only on the empirical correlation between the staleness signal and the error, not on the weights or structure of a specific network, so the same set of thresholds and step-size mapping rules can be directly transferred to DiTs of different scales.
Experimental results show that at speedups of 3.5x to 6.9x, SoftCache leads all baseline methods on all four fidelity metrics.
The specific speedup values vary by model and task: the speedup on FLUX differs from that on the video models, but overall it falls within the range of 3.5x to 6.9x.
SoftCache treats the staleness signal as a continuous error predictor rather than merely a trigger condition; the magnitude of the signal itself contains information about the current approximation quality.
All Facts (7)
The researchers propose SoftCache, which performs soft caching: the margin of the staleness signal determines how far each caching step advances, so that when the margin is large the sampler skips multiple scheduled nodes, and as a refresh approaches the step size gradually shrinks back to the original step size.
50%UnverifiedThe core innovation of SoftCache is turning the caching decision from a discrete binary choice into continuous step-size modulation.
50%UnverifiedSoftCache is completely training-free and adds no parameters specific to any backbone network, so it can be directly applied to different diffusion Transformer architectures without hyperparameter tuning for each model.
50%UnverifiedSoftCache relies only on the empirical correlation between the staleness signal and the error, not on the weights or structure of a specific network, so the same set of thresholds and step-size mapping rules can be directly transferred to DiTs of different scales.
50%UnverifiedExperimental results show that at speedups of 3.5x to 6.9x, SoftCache leads all baseline methods on all four fidelity metrics.
50%UnverifiedThe specific speedup values vary by model and task: the speedup on FLUX differs from that on the video models, but overall it falls within the range of 3.5x to 6.9x.
50%UnverifiedSoftCache treats the staleness signal as a continuous error predictor rather than merely a trigger condition; the magnitude of the signal itself contains information about the current approximation quality.
50%