Unverified50% confidenceFactExact time
cl100k_base is the specific vocabulary used by models such as GPT-4 and text-embedding-ada-002.
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedThe GPT-4 series uses the cl100k_base vocabulary (approximately 100,000 tokens), while the GPT-4o series upgrades to o200k_base (approximately 200,000 tokens).78% similarUnverifiedGPT-4 使用 cl100k_base tokenizer，与 Claude 的 tokenizer 对中文、代码等内容的切分策略不同74% similarUnverifiedThe GPT series uses the BPE algorithm with vocabularies such as cl100k_base, the Llama series uses SentencePiece, and Gemini has its own proprietary tokenization scheme.71% similarUnverified文中将 Codex 描述为 GPT-4o 的代码增强版本71% similarVerifiedThe cl100k_base vocabulary used by GPT-4 contains approximately 100,000 token units, and the number of tokens in a text can be precisely calculated using the tiktoken library.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/977588API
curl https://kongchang.com/api/v1/knowledge/claims/977588MCP
get_claim(id=977588)