Feed Entities Podcasts Deep Dives Developers About

[KongchangAI]

Real-time AI knowledge base. Every claim is traceable to its source, with data freshness under 24 hours.

Powered by 240+ first-hand AI sources

Knowledge

Feed
Hot Entities
Podcasts

Developers

CLI
MCP Server
REST API
About

© 2026 KongchangAI. The freshest, most accurate AI knowledge

Verified65% confidenceFactExact time

Attention公式中除以√d_k的原因是防止点积过大导致softmax梯度消失

3

Sources

65%

Confidence

Medium-term (~90 days)

Relevance

7/2/2026

First Seen

Valid until: 9/30/2026

Sources

The Complete Skill Map for AI Application Engineers: A Progressive Roadmap from Fundamentals to Production

bilibili大模型AI应用6/22/2026

Related Claims

Unverified注意力稀释的根本原因在于Softmax归一化：序列中token越多，每个位置分配到的有效注意力权重随序列长度近似成反比例缩小79% similar Unverified计算 softmax 函数时指数运算可能导致浮点数溢出，标准做法是先减去输入向量的最大值再取指数68% similar UnverifiedPrompt中token数量过多会稀释注意力机制的聚焦能力，降低每个描述的实际影响力67% similar Unverified缩放点积将点积除以√d（d为嵌入维度），使点积方差保持在合理范围，让Softmax输出更平滑的概率分布以稳定训练66% similar Verified多头潜在注意力机制（MLA）通过对Key-Value缓存进行低秩压缩，显著降低推理阶段的KV Cache显存占用65% similar

Cite This Claim

Stable URI

https://kongchang.com/claim/42572

API

curl https://kongchang.com/api/v1/knowledge/claims/42572

MCP

get_claim(id=42572)