Unverified50% confidenceTradeoffExact time
Below 64k, the phone cannot accelerate generation speed; generation speedup is achieved by custom kernels in the fork, including SME2 instructions on the M4 CPU, Metal fused operators, and DFlash2 speculative decoding, pushing generation from the original llama.cpp's 11.3 tok/s to approximately 25 tok/s at around 30k context.
1
Sources
50%
Confidence
Long-term
Relevance
10/2/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe UP host tested MiniCPM-2B on a computer with 4G VRAM and achieved a generation speed of 57 tokens per second, with a first-token latency of only 0.22 seconds.71% similarUnverifiedCollapse推理速度：CPU上单条10-token上下文约0.23毫秒，MPS在batch 1024下约230万token/秒，最近邻查询约0.48毫秒70% similarUnverifiedAfter INT4 quantization, the Qwen3-0.6B model file is only about 400 MB, retains approximately 95% of baseline capability, and an ordinary laptop CPU can achieve a generation speed of 10–30 tokens per second.69% similarUnverified在配备96GB显存的M2 Max上以1-bit精度运行时，简单任务生成速度稳定在约28-30 tokens/秒68% similarUnverifiedSince 2023, the NPU computing power of flagship smartphone chips has exceeded 40 TOPS, and combined with quantization techniques, 7B and even 13B parameter models can be inferred on smartphones at a speed of several tokens per second.68% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/968232API
curl https://kongchang.com/api/v1/knowledge/claims/968232MCP
get_claim(id=968232)