Unverified50% confidenceFactExact time
Vocabulary choice matters more than backbone size: 10M labels with an 8k vocabulary (8.0M total parameters, bpb 1.168) beats 50M labels with a 2k vocabulary (33.5M total parameters, bpb 1.436) — 4.2 times fewer parameters but 18.7% lower bpb.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe 2,103-token vocabulary is directionally wrong: at the 10M scale, the 2k vocabulary has a bpb about 19% higher than the 8k one.78% similarUnverifiedOn the held-out set, the 8k–16k vocabularies had the lowest bpb across all three model sizes, while the 2k vocabulary consistently performed worst.77% similarUnverified对于7B至13B参数级别的模型，4-bit量化后的输出质量与全精度版本差异很小64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1019331API
curl https://kongchang.com/api/v1/knowledge/claims/1019331MCP
get_claim(id=1019331)