Expired65% confidenceFactExact time
The BPE tokenizers of early GPT series were optimized primarily for English corpora, and often applied byte-level segmentation to Chinese text, causing a single Chinese character to be split into multiple tokens.
3
Sources
65%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
DeepSeek vs Claude Hands-On Comparison: Is the 80x API Price Gap Actually Worth It?
bilibili枫叶边城6/27/2026
Related Claims
Unverified中文、日文等表意文字因BPE词表以英语语料为主，往往需要更多Token表达相同信息量80% similarUnverified不同分词算法（BPE、WordPiece等）导致同一段中文文本在不同模型中消耗的Token数量可能相差20%-40%78% similarVerifiedIn GPT-series models, one Chinese character typically corresponds to 1 to 2 tokens, and one English word corresponds to approximately 1 to 1.5 tokens.76% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/61325API
curl https://kongchang.com/api/v1/knowledge/claims/61325MCP
get_claim(id=61325)