Unverified50% confidenceFactExact time
Bits per byte (bpb) is a measure of a language model's compression efficiency: the model's negative log-likelihood on text divided by the number of original bytes; since the byte sequence after detokenization is the same length for all tokenizers, bpb does not vary with how tokens are split, making it suitable for cross-vocabulary comparison.
1
Sources
50%
Confidence
Long-term
Relevance
10/13/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe core idea of BLEU is to compute n-gram overlap between machine translations and reference translations; its limitations include insensitivity to word order changes, inability to capture semantically equivalent synonym substitutions, and dependence on the quality and quantity of reference translations.68% similarUnverifiedBM25 is a bag-of-words model that focuses only on term frequency without understanding word order or semantics, but is more reliable for exact term matching and does not suffer from the semantic drift problem of vector retrieval.66% similarUnverifiedBits/byte is the standard evaluation metric for byte-level language models, essentially equivalent to cross-entropy with base 2, with a theoretical upper bound of 8 bits/byte for a uniform distribution model.65% similarVerifiedBPE的核心思路是从字符级别出发，反复合并出现频率最高的相邻字符对，直到词汇表达到预设大小65% similarUnverifiedThe optimization objectives of the mainstream tokenization algorithms BPE and Unigram are essentially both compression: minimizing the corpus encoding length under a limited vocabulary.64% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1019326API
curl https://kongchang.com/api/v1/knowledge/claims/1019326MCP
get_claim(id=1019326)