Unverified50% confidenceFactExact time
Entropy regularization techniques have been widely applied in the MoE domain, such as the load balancing loss in GShard and Switch Transformer.
1
Sources
50%
Confidence
Long-term
Relevance
10/8/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe original paper on Switch Transformer discusses the design of introducing auxiliary loss functions to enforce uniform distribution among experts.63% similarUnverifiedSAC（2018）引入最大熵正则化，在连续控制任务中表现卓越，鼓励策略在保持高回报的同时尽可能随机62% similarUnverifiedSAC引入熵正则化在探索与利用之间取得平衡，是当前连续动作空间的主流选择60% similarUnverifiedThe introduction of Transformer architectures such as Decision Transformer provides new tools for handling extremely long temporal dependencies.58% similarUnverifiedAkyürek等人2022年的工作从理论上证明，Transformer在执行上下文学习时其前向传播等价于在注意力层中隐式运行一步梯度下降更新58% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/986861API
curl https://kongchang.com/api/v1/knowledge/claims/986861MCP
get_claim(id=986861)