Unverified50% confidenceOpinionExact time
推理令牌聚集现象可能与Transformer架构中的注意力熵坍缩现象存在内在关联
1
Sources
50%
Confidence
Long-term
Relevance
7/6/2026
First Seen
Sources
GPT-5.5 Codex Reasoning-Token Clustering Explained: Causes of Performance Regression and How to Respond
hackernewshackernews7/4/2026
Related Claims
Unverified各向异性现象的根源在于Transformer的自注意力机制和残差连接结构，使高频词的梯度信号主导嵌入空间整体方向74% similarUnverifiedIn the attention mechanism of the Transformer architecture, as sequence length increases, the attention scores of early tokens are diluted by subsequent tokens after softmax normalization, forming an attention sinking effect.74% similarUnverifiedBecause the Transformer self-attention mechanism exhibits positional bias, earlier input content has a stronger anchoring effect on subsequent generation, which is the underlying reason for the context anchoring effect of system prompts74% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/123357API
curl https://kongchang.com/api/v1/knowledge/claims/123357MCP
get_claim(id=123357)