Unverified95% confidenceFactExact time
In large language models, a Token is the smallest unit of text that a model processes
1
Sources
95%
Confidence
Long-term
Relevance
7/28/2026
First Seen
Sources
Related Entities
Related Claims
Unverified大模型处理文本以Token（词元）为最小处理单元，由分词器完成切分85% similarUnverifiedToken是大模型计费的基本单位，本质上是文本被分割后的最小语义片段83% similarUnverifiedToken是模型处理文本的最小计量单位，OpenAI使用BPE算法将高频字符组合编码为单一Token，约75个英文单词≈100个Token76% similarUnverifiedAI模型调用的计费单位通常是Token，即文本被分词器切割后的最小语义单元75% similarUnverified语言模型生成 token 本质上是内存带宽受限任务而非纯算力受限73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/680482API
curl https://kongchang.com/api/v1/knowledge/claims/680482MCP
get_claim(id=680482)