Partially Verified65% confidenceFactExact time
OpenAI的tiktoken从GPT-2时代的5万词表扩展至GPT-4的10万词表
3
Sources
65%
Confidence
Long-term
Relevance
7/6/2026
First Seen
Sources
Claude Sonnet 5实测：隐藏的Token陷阱让「低价」成幻觉
bilibili浮生千山路w6/30/2026
Related Claims
UnverifiedOpenAI's GPT series uses the tiktoken library, with vocabulary sizes typically ranging between 50,000 and 100,000 Tokens84% similarVerifiedGPT-4使用的cl100k_base词表包含约10万个Token单元，可通过tiktoken库精确计算文本的Token数量77% similarVerifiedGPT-4系列采用cl100k_base词表（约10万词元），GPT-4o系列升级至o200k_base（约20万词元）76% similarUnverified以OpenAI的GPT系列为例，仅定义工具调用规范的系统提示词就可能消耗2000到5000个Token75% similarUnverifiedGPT系列使用tiktoken分词器，LLaMA系列使用SentencePiece分词器73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/114167API
curl https://kongchang.com/api/v1/knowledge/claims/114167MCP
get_claim(id=114167)