Unverified50% confidenceBenchmarkExact time
On the held-out set, the 8k–16k vocabularies had the lowest bpb across all three model sizes, while the 2k vocabulary consistently performed worst.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedVocabulary choice matters more than backbone size: 10M labels with an 8k vocabulary (8.0M total parameters, bpb 1.168) beats 50M labels with a 2k vocabulary (33.5M total parameters, bpb 1.436) — 4.2 times fewer parameters but 18.7% lower bpb.77% similarUnverifiedThe 2,103-token vocabulary is directionally wrong: at the 10M scale, the 2k vocabulary has a bpb about 19% higher than the 8k one.74% similarUnverified对于7B至13B参数级别的模型，4-bit量化后的输出质量与全精度版本差异很小66% similarUnverifiedThe degree-3 and degree-4 polynomial coefficients in the paper are obtained by minimizing the maximum error (minimax approximation) in the BF16/FP16 target format.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1019329API
curl https://kongchang.com/api/v1/knowledge/claims/1019329MCP
get_claim(id=1019329)