Unverified50% confidenceFactExact time
The gating network is usually a lightweight linear layer that maps a token's hidden state to a score for each expert, and after Softmax takes the top-K experts by score.
1
Sources
50%
Confidence
Long-term
Relevance
10/21/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedA learned selector is usually a lightweight network (such as a small MLP or low-rank projection) that takes the current query vector and the key vectors of candidate tokens, quickly scores them, and selects the Top-k most relevant tokens to participate in the actual attention computation.66% similarUnverifiedThe 'depth' in deep learning refers to a network containing multiple hidden layers.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1083394API
curl https://kongchang.com/api/v1/knowledge/claims/1083394MCP
get_claim(id=1083394)