Unverified50% confidenceFactExact time
The decoder adds two mechanisms on top of the encoder: masked self-attention and cross-attention; masked self-attention ensures that during generation only previously produced tokens can be seen, while cross-attention allows the decoder to query the encoder's output.
1
Sources
50%
Confidence
Long-term
Relevance
10/8/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe decoder-only architecture uses a causal attention mask to ensure each token can only attend to its preceding context, making it suitable for autoregressive text generation tasks.75% similarUnverifiedOffline file transcription can use bidirectional attention mechanisms to integrate information from both preceding and following context, and can perform multiple rounds of decoding and reranking to improve accuracy.72% similarUnverified自注意力本身对顺序不敏感，打乱词语顺序注意力权重计算结果相同，因此需要引入位置编码；原始Transformer论文使用正余弦函数编码70% similarUnverifiedPositional Encoding solves the problem of the attention mechanism being inherently orderless and unaware of word sequence by attaching positional information to each word.69% similarVerifiedA Cross-Encoder concatenates a query with each candidate document and feeds them jointly into the same model, capturing word-level interactions through the attention mechanism; its ranking precision is superior to vector similarity, but latency grows linearly with the size of the candidate set.69% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/987733API
curl https://kongchang.com/api/v1/knowledge/claims/987733MCP
get_claim(id=987733)