Unverified50% confidenceFactExact time
Handwired README's speed claims are aimed at batched evaluation on GPU, for example the JSON search demo processing 100,000 documents at a time.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/16/2026
First Seen
Valid until: 1/14/2027
Sources
Related Entities
Related Claims
Unverified传统GPU如NVIDIA A100/H100在推理阶段面临内存带宽瓶颈，模型权重需要在每次生成Token时从显存反复读取69% similarUnverified自回归生成中每生成一个token需读取全部权重一次，内存带宽而非算力峰值才是真正的性能瓶颈65% similarUnverifiedOn an Intel i5-9500, the Qwen2.5-7B-Instruct model reads at about 8 tokens per second, and reading the prompt accounts for about 90% of the time of each model call.65% similarUnverified大模型推理服务的快照数据量可能达到数十GB甚至上百GB64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1048086API
curl https://kongchang.com/api/v1/knowledge/claims/1048086MCP
get_claim(id=1048086)