Unverified50% confidenceFactExact time
The literature previously reported a correlation of up to 0.989 between unigram-normalized loss and BPB, but after correcting for confounders the correlation plummets to 0.057.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedStarting from the same data, BPB and unigram-normalized loss select different vocabularies at every model scale.73% similarUnverifiedIt is generally believed that quality loss is small when the average KLD is below 0.1, while degradation may be noticeable above 0.5.68% similarUnverifiedIn the fixed-vector self-similarity experiment, when the offset goes from 0 to 1024, the mean of the raw scores always fluctuates within ±0.38, while the mean of the absolute values only drops from 3.17 to 3.02, a change of about 4%.67% similarUnverifiedThe study gives a negative answer: lower BPB does not mean a better model.67% similarUnverifiedValidation loss started at 9.48 and dropped to 4.27 after approximately one complete pass through the corpus66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134080API
curl https://kongchang.com/api/v1/knowledge/claims/1134080MCP
get_claim(id=1134080)