Unverified50% confidenceFactExact time
In the GPT-2 vocabulary, the emoji 🙂 (4 UTF-8 bytes) becomes 2 tokens; GPT-4o's larger vocabulary learned the entire sequence, so it is 1 token.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/13/2026
First Seen
Valid until: 1/11/2027
Sources
Related Entities
Related Claims
Unverified不同模型（如 GPT-4、GPT-3.5、text-embedding-ada-002）使用的是不同的 tokenizer，同一段文本在不同模型下的 token 数量可能不同。73% similarVerifiedIn the GPT series of models, one token corresponds to approximately 4 English characters or 0.75 English words.69% similarUnverifiedAn equal-token-width font is inherently tied to a specific tokenizer, and a font generated for GPT loses its meaning when applied to a different model.68% similarVerifiedOpenAI's GPT series increments by integer version numbers (GPT-3, GPT-4) and uses letter suffixes to distinguish variants, such as "o" representing the optimized version (GPT-4o) and "mini" representing the lightweight version66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1022419API
curl https://kongchang.com/api/v1/knowledge/claims/1022419MCP
get_claim(id=1022419)