DeepSeek V4.1
Timeline (last 90 days)
The deployment scale of V4.1 is prefill EP2 to EP8 and decode EP64 to EP128; DeepInfer adds intra-group voting and a second round of confirmation in the scheduler to coordinate card pacing under wide expert parallelism.
V4.1 adopts SWA bounded replay: it recomputes only the last 128 tokens, and attention is also confined to within these 128 tokens, yielding an approximate window cache; the recomputation cost becomes constant, and the Decoder's 20 layers, which originally had to process the last 2,560 tokens, now only need to process 128, reducing total prefill computation by nearly half.
The agents the uploader uses include DeepSeek V4.1, WorkBuild, and local small models (such as 1825B Flashnex).
DeepSeek V4.1 uses local attention for some layers, looking only at the most recent 128 Tokens, and short-term notes are not archived but kept only in local memory and cleared after about three minutes, with the hit rate still above 98%.
DeepSeek V4.1's SWA Bound replay technique reduces the number of Tokens that need to be recomputed when reading the question from 2560 to 128, roughly halving the total computation for reading the question.
The earlier DeepSeek V4.1 version tested on Great Wall coding capability can now be upgraded, but the dataset appears to be somewhat contaminated.
The V4.1 architecture is robust to quantization, with accuracy loss on the MMLU benchmark of less than 0.2% when switching from FP16 to FP8.
Using Campbot DSL, the team developed 8 fused operators for the DeepSeek V4.1 network, including index, prolog, kingram, and git, among others.
The Attention layer of DeepSeek V4.1 can be divided into three stages: pre-processing, Attention computation, and post-processing.
These 8 fused operators achieved significant performance gains compared to the serial computation of small operators.
17 more timeline events
All Facts (20)
Liu Shengyu is a DeepSeek researcher who participated in writing the main Attention operator for DeepSeek V4.1.
60%UnverifiedThe deployment scale of V4.1 is prefill EP2 to EP8 and decode EP64 to EP128; DeepInfer adds intra-group voting and a second round of confirmation in the scheduler to coordinate card pacing under wide expert parallelism.
50%UnverifiedV4.1 adopts SWA bounded replay: it recomputes only the last 128 tokens, and attention is also confined to within these 128 tokens, yielding an approximate window cache; the recomputation cost becomes constant, and the Decoder's 20 layers, which originally had to process the last 2,560 tokens, now only need to process 128, reducing total prefill computation by nearly half.
50%UnverifiedThe agents the uploader uses include DeepSeek V4.1, WorkBuild, and local small models (such as 1825B Flashnex).
50%UnverifiedDeepSeek V4.1 uses local attention for some layers, looking only at the most recent 128 Tokens, and short-term notes are not archived but kept only in local memory and cleared after about three minutes, with the hit rate still above 98%.
50%UnverifiedDeepSeek V4.1's SWA Bound replay technique reduces the number of Tokens that need to be recomputed when reading the question from 2560 to 128, roughly halving the total computation for reading the question.
50%UnverifiedThe earlier DeepSeek V4.1 version tested on Great Wall coding capability can now be upgraded, but the dataset appears to be somewhat contaminated.
50%UnverifiedThe V4.1 architecture is robust to quantization, with accuracy loss on the MMLU benchmark of less than 0.2% when switching from FP16 to FP8.
50%UnverifiedUsing Campbot DSL, the team developed 8 fused operators for the DeepSeek V4.1 network, including index, prolog, kingram, and git, among others.
50%UnverifiedThese 8 fused operators achieved significant performance gains compared to the serial computation of small operators.
50%UnverifiedThe Attention layer of DeepSeek V4.1 can be divided into three stages: pre-processing, Attention computation, and post-processing.
50%UnverifiedDeepSeek V4.1 claims to achieve 74 on SWE, but the blogger found that this was the best result cherry-picked from multiple tests, with open code testing at 65 and PI testing at 66.
50%UnverifiedIn a typography replication test, GPT-6 Sol was the fastest, subjectively completing the task in about four to five minutes; Luna took approximately three times as long as Sol (close to twenty minutes); DeepSeek was the slowest at about half an hour.
50%UnverifiedThe test conclusions come from a single tester focused on the specific scenario of WeChat Official Account typography replication; the scores are subjective and do not represent the models' full capabilities in dimensions such as programming, reasoning, or long-context handling.
50%UnverifiedThe thinking intensity levels set for the test were: Max for Sol and DeepSeek, and High for Luna.
50%UnverifiedIn the style-preservation step after copying and pasting into the WeChat Official Account platform, Sol and DeepSeek retained styles largely intact, while Luna's styles became disordered at the end.
50%UnverifiedAfter being asked to rework and fix the output, Sol completed the multi-image layout and added decorative text beside images, while Luna and DeepSeek produced the multi-image layout but did not add the decorative text.
50%UnverifiedAll three models (Sol, Luna, and DeepSeek) failed to accurately reproduce multi-image layouts, which was a common shortcoming.
50%UnverifiedIn terms of token consumption, Sol was the lowest; Luna's consumption was slightly more than twice that of Sol; DeepSeek's was higher than Luna's.
50%UnverifiedDeepSeek V4.1 ranked first in tests including Terminal 2.1, software engineering, cybersecurity, office automation, and agent terminal examinations.
50%