Unverified50% confidenceFactExact time
各向异性现象的根源在于Transformer的自注意力机制和残差连接结构，使高频词的梯度信号主导嵌入空间整体方向
1
Sources
50%
Confidence
Long-term
Relevance
7/7/2026
First Seen
Sources
分散损失：破解小语言模型嵌入坍缩的关键技术
hackernewshackernews7/3/2026
Related Claims
UnverifiedBecause the Transformer self-attention mechanism exhibits positional bias, earlier input content has a stronger anchoring effect on subsequent generation, which is the underlying reason for the context anchoring effect of system prompts81% similarVerifiedThe self-attention mechanism of the Transformer can capture dependencies between any two positions in a sequence, and regardless of how far apart the tokens are, the computational cost of calculating attention weights is the same77% similarUnverified推理令牌聚集现象可能与Transformer架构中的注意力熵坍缩现象存在内在关联74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/189249API
curl https://kongchang.com/api/v1/knowledge/claims/189249MCP
get_claim(id=189249)