Concept字节级 BPE / Byte-level BPE
Byte-level BPE
A tokenization scheme that first converts text into a UTF-8 byte sequence and then runs the BPE merge algorithm at the byte level. It inherently covers any Unicode character without an UNK token, and vocabulary size can be set freely.
Timeline (last 90 days)
Oct 12
The 26.2M model uses an 8-layer encoder-only Transformer, d_model 384, 6 attention heads, FFN dimension 1536, a maximum of 160 tokens, a self-developed 16k byte-level BPE tokenizer, and 15,000 edit labels.
Unverified50%