Unverified50% confidenceFactExact time
Each token is converted into a d_model-dimensional vector by looking up the Embedding Table, and N tokens form an N × d_model input matrix.
1
Sources
50%
Confidence
Long-term
Relevance
9/15/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe self-attention mechanism transforms each character vector into Query, Key, and Value respectively through three learnable weight matrices, transforming N × dm into N × dk.66% similarUnverified大模型推理的自回归生成需要逐Token串行输出，生成N个Token需要N次前向传播63% similarUnverifiedMLA通过低秩分解将KV Cache显存开销从O(n·d_model)压缩到O(n·d_latent)，其中d_latent约为d_model的1/862% similarUnverifiedAttention 采用 QKV 框架，每个 Token 的 Embedding 经三个线性变换矩阵生成 Query、Key、Value 向量61% similarUnverifiedThe model adopts N-Gram Embedding, combining multiple embeddings into a local context lookup table, which can significantly increase parameter capacity with almost no increase in per-token computation.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/923464API
curl https://kongchang.com/api/v1/knowledge/claims/923464MCP
get_claim(id=923464)