Unverified50% confidenceFactExact time
Held-out BPB (bits per byte) is a metric that measures the compression efficiency of a language model, where a lower value indicates more accurate prediction of unseen text.
1
Sources
50%
Confidence
Long-term
Relevance
10/27/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedBits per byte (bpb) is a measure of a language model's compression efficiency: the model's negative log-likelihood on text divided by the number of original bytes; since the byte sequence after detokenization is the same length for all tokenizers, bpb does not vary with how tokens are split, making it suitable for cross-vocabulary comparison.75% similarUnverifiedThe optimization objectives of the mainstream tokenization algorithms BPE and Unigram are essentially both compression: minimizing the corpus encoding length under a limited vocabulary.67% similarVerifiedBPE的核心思路是从字符级别出发，反复合并出现频率最高的相邻字符对，直到词汇表达到预设大小65% similarUnverifiedBPE 的核心思想是将高频共现的字符对反复合并为新的子词单元，能处理未登录词（OOV）同时保持词表规模可控65% similarVerifiedThe BPE algorithm originates from the field of data compression and builds a vocabulary by iteratively merging high-frequency character pairs.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1102614API
curl https://kongchang.com/api/v1/knowledge/claims/1102614MCP
get_claim(id=1102614)