Conceptdynamic caching
Dynamic Caching
A diffusion model inference acceleration strategy that monitors the degree of change in the input to dynamically decide whether to reuse cached activation values, rather than refreshing according to a fixed schedule.
Timeline (last 90 days)
Oct 16
Dynamic caching methods accumulate cheap probes of the network input into a staleness signal and perform a new evaluation when the signal crosses a threshold, thereby deciding when to refresh; such methods perform hard caching.
Unverified50%