Expired50% confidenceBenchmarkExact time
A sandbox engine with 8 million parameters (8.3M) achieves a decoding speed of approximately 2,000 tokens/second on a 6-core desktop CPU released in 2019.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/17/2026
First Seen
Valid until: 10/15/2026(expired)
Sources
Related Claims
Partially VerifiedFor the 4-bit quantized version of a 7B parameter model, at least 8GB of memory and a modern multi-core CPU are typically required for smooth operation, with inference speeds of approximately 10–30 tokens per second69% similarUnverifiedApple M2/M3 chips adopt a Unified Memory Architecture (UMA), and llama.cpp can achieve approximately 30–50 tokens/second inference speed for 7B models via the Metal API.67% similarUnverifiedTaking Llama-3 70B as an example, the prefill stage for processing 32K tokens on an A100 80GB GPU takes approximately 800–1200ms, while processing 2K tokens takes only about 50ms.67% similarUnverifiedIn prefill speed tests, Halo achieved approximately 650 tokens/s while Spark achieved approximately 2000 tokens/s, with Spark being nearly three times faster.67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/538383API
curl https://kongchang.com/api/v1/knowledge/claims/538383MCP
get_claim(id=538383)