SoftCache: Adaptive Step-Size Caching Acceleration for Diffusion Transformers

Diffusion models and flow models generate samples through iterative steps, each requiring a full forward pass of a large network, which dominates the generation cost. Training-free caching techniques accelerate generation by reusing activations from recent computations to avoid redundant computation. Dynamic caching methods accumulate cheap probes of the network input into a staleness signal, and when the signal crosses a threshold, they perform a fresh evaluation to decide when to refresh. These methods perform hard caching: the signal is a binary switch between refresh and reuse, and every step—whether fresh or cached—advances by a preset step size.

From Hard Caching to Soft Caching

The researchers observed a key phenomenon: the staleness signal carries far more information than just a switch. The degree to which the signal falls below the threshold—its "headroom"—can predict the error introduced by reuse. Based on this insight, they proposed SoftCache, which performs soft caching: instead of using a binary switch, the headroom determines how far each cached step advances. When the headroom is large, the sampler skips multiple predetermined nodes; as a refresh approaches, the step size gradually shrinks back to the original step size.

The core innovation of this method lies in transforming the caching decision from a discrete binary choice into continuous step-size modulation. Traditional hard caching methods follow a fixed schedule at every step, whereas SoftCache allows adaptive jumps. These adaptive jumps may land between predetermined nodes, causing the sampler to deviate from the fixed grid. This flexibility is precisely the source of acceleration—taking large strides in regions where error is controllable, and returning to fine steps in regions that require precision.

The Specific Meaning of Staleness Signals and Headroom

In dynamic caching methods, the cheap probes of the network input typically refer to intermediate features or statistics with extremely low computational cost, such as the activation mean of certain layers, the cosine similarity between the input and the input at the last refresh, or the difference norm of the timestep embedding. These probes are accumulated into a scalar signal that measures the "staleness" of the current input relative to the input at the last full forward pass. When this signal exceeds a preset threshold, it indicates that the error introduced by reusing old activations may be too large, so a new full evaluation is performed and the signal is reset.

"Headroom" is defined as the difference between the threshold and the current signal value. A larger headroom means that the current input differs less from the input at the last refresh, and the approximation error introduced by reusing old activations is also smaller; a smaller headroom means it is approaching the critical point for the next refresh. SoftCache precisely leverages this continuous quantity to predict error, rather than merely judging whether it has crossed the boundary. This shift from binary triggering to continuous prediction allows the sampler to safely skip multiple predetermined nodes when headroom is ample, thereby achieving a larger speedup ratio.

Training-Free and Tuning-Free Design

One notable advantage of SoftCache is that it is completely training-free and does not add any parameter settings specific to a particular backbone network. This means it can be directly applied to different diffusion Transformer architectures without hyperparameter tuning for each model. The paper validated the generality of the method on several mainstream models: FLUX in the image generation domain, and HunyuanVideo and Wan2.1 in the video generation domain.

This plug-and-play characteristic is particularly important for practical deployment. The inference cost of current diffusion models remains one of the core bottlenecks for application deployment, especially in video generation scenarios, where there are many iterative steps and the network is large, making the compute cost of generating a video considerable. Training-free caching methods do not require additional fine-tuning or distillation processes and can be directly layered onto existing inference pipelines.

Generality Across Diffusion Transformer Architectures

Diffusion Transformers (DiTs) replace the U-Net backbone in traditional diffusion models with a pure Transformer structure to process latent representations of images or videos. Due to the flexibility of Transformers for sequence modeling, DiTs have become the mainstream architecture for current high-quality generative models; for example, FLUX, HunyuanVideo, and Wan2.1 all adopt such designs. However, differences among DiTs in layer count, hidden dimension, attention mechanism, and training data cause their activation distributions and error sensitivities to vary.

SoftCache's training-free nature means it does not need to retrain an error predictor or fine-tune a caching strategy for each DiT. Its core mechanism—using the headroom of the staleness signal to modulate step size—depends only on the empirical correlation between the signal itself and the error, not on the weights or structure of a specific network. Therefore, the same set of thresholds and step-size mapping rules can be directly transferred to DiTs of different scales, requiring only a minimal set of general hyperparameters (such as the maximum jump step count) to work. This significantly reduces the engineering cost of adapting to new models in practical deployment.

Balancing Speed and Fidelity

Experimental results show that at speedups of 3.5x to 6.9x, SoftCache leads all baseline methods on all four fidelity metrics. This result is noteworthy because caching acceleration methods typically face a trade-off between speed and quality—the higher the speedup factor, the more pronounced the degradation in generation quality. SoftCache maintains or even improves fidelity while significantly accelerating, indicating that adaptive step-size strategies better match the actual error distribution of the diffusion process than fixed-step cache refreshing.

The specific speedup values vary by model and task: the speedup on FLUX differs from that on video models, but overall falls within the range of 3.5x to 6.9x. For video generation, this speedup means that a generation process that originally took several minutes can be shortened to the tens-of-seconds level, which has practical significance for interactive applications and batch generation scenarios.

Rethinking Caching Strategies

The significance of SoftCache lies not only in its specific acceleration effects, but also in how it reexamines the use of caching signals. Previous work treated the staleness signal as a trigger condition—either refresh or reuse. SoftCache instead treats it as a continuous error predictor, where the magnitude of the signal itself contains information about the current approximation quality.

This shift in thinking opens up new optimization space. If headroom can predict error, then in theory caching behavior can be controlled at a finer granularity—for example, dynamically adjusting the step-size allocation across multiple cached steps based on headroom, or combining other error metrics for more precise scheduling. The adaptive jumping and step-size shrinking mechanisms demonstrated in the paper are only one implementation of this idea, and future research may develop richer caching strategies on this basis.

From an engineering perspective, SoftCache's training-free nature makes it easy to integrate into existing diffusion model inference pipelines. For application scenarios that need to deploy high-quality generative models under limited compute, this type of training-free acceleration method provides a pragmatic path.

The Relationship Between Error Predictors and Scheduling Strategies

Treating the staleness signal as a continuous error predictor is essentially building a cheap surrogate model to estimate the generation error introduced by reusing current activations. This idea is similar to adaptive step-size solvers (such as adaptive order adjustment in DPM-Solver), but the key difference is that adaptive solvers usually need to compute local truncation error, whereas SoftCache relies only on the accumulated signal of input probes, with almost negligible computational overhead.

This surrogate error signal makes finer-grained scheduling possible. For example, one can design a step-size allocation function such that in regions with large headroom it continuously jumps multiple nodes, when headroom is moderate it jumps only one node, and when headroom approaches zero it returns to the original step size. Furthermore, headroom can be linked to the accumulated error across multiple cached steps to achieve globally optimal jump-sequence planning. The adaptive jumping and step-size shrinking demonstrated in the paper are only an initial implementation of this framework; future work could explore more complex mapping functions or combine other lightweight error metrics to maintain generation quality at higher speedup ratios.

One-line Summary

SoftCache converts the headroom of the cache staleness signal into continuous step-size modulation, achieving training-free 3.5x to 6.9x acceleration with leading fidelity.

Paragraph Summary

Diffusion models and flow models have high generation costs because iterative steps require full forward passes of large networks, and training-free caching techniques accelerate by reusing activations. Traditional dynamic caching methods perform hard caching: cheap probes are accumulated into a staleness signal, and when it crosses a threshold they refresh, otherwise they reuse, with every step advancing by a preset step size. SoftCache observes that the headroom by which the signal falls below the threshold can predict reuse error, so it proposes soft caching: letting the headroom determine how far each cached step advances. When headroom is large, it skips multiple predetermined nodes; as a refresh approaches, the step size shrinks back to the original step size. The method is completely training-free, adds no parameters specific to a backbone network, and can be directly applied to diffusion Transformers such as FLUX, HunyuanVideo, and Wan2.1. Experiments show that at 3.5x to 6.9x acceleration, SoftCache leads baselines on all four fidelity metrics, redefining how caching signals are used from binary triggers to continuous error predictors.