ConceptSWA 有界重放
SWA Bounded Replay
A prefill optimization technique for model layers that use sliding window attention. Instead of computing the entire window from the start, it replays only the short window at the end of the prompt, substantially reducing prefill work.
Timeline (last 90 days)
Oct 21
Some of DeepSeek's model layers use sliding window attention, attending only to the most recent 128 tokens; SWA bounded replay replays only a short window at the end of the prompt, which is said to nearly eliminate total prefill work.
Unverified50%