Unverified50% confidenceFactExact time
BPB is computed as tokens per byte multiplied by bits per token, and these two factors do not cancel each other out.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedBits per byte (bpb) is a measure of a language model's compression efficiency: the model's negative log-likelihood on text divided by the number of original bytes; since the byte sequence after detokenization is the same length for all tokenizers, bpb does not vary with how tokens are split, making it suitable for cross-vocabulary comparison.67% similarVerifiedBPE的核心思路是从字符级别出发，反复合并出现频率最高的相邻字符对，直到词汇表达到预设大小65% similarUnverifiedDuring inference, tokenization must be executed strictly in the same BPE merge rank order as used during training; otherwise, the same text may be split into different token sequences.63% similarUnverifiedBPE builds a shared vocabulary covering common roots and affixes by statistically counting the co-occurrence frequency of adjacent characters and iteratively merging high-frequency subword units.61% similarUnverifiedHeld-out BPB (bits per byte) is a metric that measures the compression efficiency of a language model, where a lower value indicates more accurate prediction of unseen text.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134075API
curl https://kongchang.com/api/v1/knowledge/claims/1134075MCP
get_claim(id=1134075)