#BERT

BERT Tokenization Bottleneck Moves Upstream: Normalization Swallows 88% of Encoding Time

BERT tokenization profiling reveals normalization consumes 88% of English encoding time, WordPiece only 8%. Inside Simon Brandeis's optimization: SIMD skips ASCII, page cache speeds non-ASCII scripts, and why to measure tokenization separately in RAG and embedding services.