Unverified50% confidenceFactExact time
Zero-width characters (such as U+200B and U+200D) are invisible in browsers, but the tokenizer still encodes them, so they may be used for steganography or to bypass keyword filtering.
1
Sources
50%
Confidence
Long-term
Relevance
11/3/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedZero-width characters are a set of special control characters in the Unicode standard, including zero-width space (U+200B), zero-width joiner (U+200D), zero-width non-joiner (U+200C), and left-to-right mark (U+200E); they are completely invisible visually but exist at the byte level78% similarUnverifiedThe same passage of text may be split into different numbers of tokens under different tokenizers, while API billing and context window limits are both measured in token counts.64% similarUnverifiedInput text in a Transformer first goes through a Tokenizer for segmentation, splitting the text into N tokens, which are the smallest units by which the language model understands semantics.63% similarUnverifiedA token is the smallest unit of measurement for text processed by a model; OpenAI uses the BPE algorithm to encode high-frequency character combinations as single tokens, with approximately 75 English words equaling roughly 100 tokens.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1129669API
curl https://kongchang.com/api/v1/knowledge/claims/1129669MCP
get_claim(id=1129669)