Unverified50% confidenceFactExact time
OpenAI 使用的 tiktoken 以及 Google 的 SentencePiece 均以 BPE 为基础
1
Sources
50%
Confidence
Long-term
Relevance
7/20/2026
First Seen
Sources
Related Claims
UnverifiedGoogle的Gemma系列分词器采用SentencePiece框架实现，支持BPE和Unigram两种算法70% similarUnverifiedPage Agent的技术路径与OpenAI的Computer Use、Google的Project Mariner、Anthropic的Claude Computer Use高度相似61% similarUnverifiedGoogle的Gemini采用SentencePiece框架配合Unigram语言模型算法进行分词60% similarUnverifiedOpenAI、谷歌等头部机构均在积极研究文本水印标准60% similarUnverifiedSentencePiece由Google开发，无需预分词步骤，直接在原始文本上学习子词单元60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/570184API
curl https://kongchang.com/api/v1/knowledge/claims/570184MCP
get_claim(id=570184)