Unverified50% confidenceBenchmarkExact time
Under high power operation, the 35B MoE model reaches 78 to 107 tokens/s, while the dense 27B model only reaches 22 tokens/s.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
UnverifiedA user runs the 397B model on 4× ASUS GB10 units achieving 30 tokens/s at only 400W of power consumption, compared to 50–60 tokens/s at 6kW previously.79% similarUnverifiedMost models have a single output limit of between 4,000 and 8,000 tokens, with some high-end models supporting up to 32,000 tokens.76% similarVerifiedTaking LLaMA-2 70B as an example, when processing a 4096-token context with a batch size of 32, the KV Cache can consume over 80 GB of memory.74% similarUnverifiedUnder this configuration, the 35B model achieves a measured single-user peak speed of approximately 80 tokens/second, with concurrent throughput exceeding 500 tokens/second.74% similarUnverifiedThe author considers 12–30 tokens/s to be the comfort zone, with above 30 tokens/s being good; an M5 Max with 48–64 GB can run at approximately 20 tokens/s.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1015148API
curl https://kongchang.com/api/v1/knowledge/claims/1015148MCP
get_claim(id=1015148)