Unverified50% confidenceFactExact time
With a vocabulary size of 12,000, the loss under uniform random guessing for the next token is approximately ln(12000) ≈ 9.39
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Claims
UnverifiedAn EMNLP study found that 4-bit models suffer an average retrieval loss of 23% at 128,000 tokens, with 70B LLaMA losing one-third of its score while QWEN of the same scale loses almost none.66% similarUnverifiedThe order of answer options causes the Decisions API's confidence to drift by approximately 13 percentage points.65% similarUnverifiedThe power-law exponent between test loss and compute is approximately 0.05, meaning that every 10x increase in compute yields approximately a 12% improvement in model performance.65% similarUnverifiedIn 11 of the 12 models, between 87.5% and 100% of capitulation cases stopped at the exact wrong number provided by the user, meaning capitulation typically involves switching sides entirely.64% similarUnverifiedA model with 11,000 parameters occupies less than 50KB of memory.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1004158API
curl https://kongchang.com/api/v1/knowledge/claims/1004158MCP
get_claim(id=1004158)