Bits-per-byte

An evaluation metric for language model compression efficiency, defined as the model's negative log-likelihood for a text divided by the number of original bytes. It is insensitive to vocabulary size, making it suitable for comparison across tokenizers, and avoids the artificial influence of vocabulary size that affects perplexity.