Unverified50% confidenceEventExact time
In an experiment with a 32-expert miniature character model, when Token Choice routing had no load balancing loss, a single expert swallowed 57% of the tokens in the early stage of training, and 17 of the 32 experts were almost ignored.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Understanding MoE Routing: Three Core Problems and Two Fixes
redditr/deeplearning10/15/2026
Related Entities
Related Claims
UnverifiedIn three rounds of controlled experiments on a 32-expert tiny character model, Expert Choice routing achieved the lowest evaluation loss among the three rounds.80% similarUnverifiedAnthropic's research shows that even mild alignment training can cause a 2–8 percentage point decline in model performance on certain reasoning-intensive tasks.74% similarUnverifiedThe gap between selection and generation exceeds 50 percentage points, revealing that reliability is far lower than benchmarks suggest when users need the model to directly write out kinship terms.73% similarUnverifiedWhen sampling eight candidates without constraints from the model, 10.2% of inputs cannot produce any valid output under limited training data, and this proportion drops to 0.4% when using the full training data72% similarUnverifiedAn Anthropic randomized trial had 52 engineers (mostly junior) learn an unfamiliar Python library; the AI-assisted group averaged 50% on the comprehension quiz, while the handwritten-code group averaged 67%, with the largest gap on debugging-type questions; the speed advantage brought by AI was about 2 minutes and was not statistically significant.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1035486API
curl https://kongchang.com/api/v1/knowledge/claims/1035486MCP
get_claim(id=1035486)