Unverified50% confidenceFactExact time
In BERT, the word embedding matrix has dimensions V×d, where V is the vocabulary size and d is the hidden dimension 768, mapping discrete token IDs to dense real-valued vectors.
1
Sources
50%
Confidence
Long-term
Relevance
9/6/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedAn embedding layer is essentially a learnable matrix of size V×D (where V is the vocabulary size and D is the embedding dimension) that looks up the corresponding row vector when given a token ID.78% similarUnverifiedpoly核将原始d维输入隐式映射到O(n^d)维特征空间，维度随degree指数级增长，计算代价急剧膨胀65% similarUnverifiedFor a model with L layers, H attention heads, and per-head dimension d, caching n tokens requires VRAM of 2 × L × H × d × n × (precision bytes).60% similarUnverifiedEmbedding vectors are essentially high-dimensional numerical vectors output by intermediate layers of a neural network, typically having hundreds to thousands of dimensions, and represent the coordinates of raw input data in a learned semantic space.60% similarUnverified标准 Scaled Dot-Product Attention 中 Q 与 K 做矩阵乘法得到 n×n 注意力分数矩阵，计算量为 O(n²·d)59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/866059API
curl https://kongchang.com/api/v1/knowledge/claims/866059MCP
get_claim(id=866059)