Unverified50% confidenceFactExact time
tokenizer 指的是将文本切分为 token 的完整算法流程,而 encoding 是 OpenAI 在 tiktoken 中给每一套具体 tokenizer 规则起的名字,如 cl100k_base、o200k_base、p50k_base。
1
Sources
50%
Confidence
Long-term
Relevance
10/12/2026
First Seen
Sources
Related Entities
Cite This Claim
Stable URI
https://kongchang.com/claim/1009190API
curl https://kongchang.com/api/v1/knowledge/claims/1009190MCP
get_claim(id=1009190)