Unverified50% confidenceOpinionExact time
Cross-entropy is a per-token average metric and does not additionally penalize the structural defect that some tokens are never processed at all, so when comparing routing strategies one cannot look only at a single average evaluation loss.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Understanding MoE Routing: Three Core Problems and Two Fixes
redditr/deeplearning10/15/2026
Related Entities
Related Claims
UnverifiedAfter temperature scaling, none of the training objectives is clearly superior to cross-entropy in terms of calibration.60% similarUnverifiedBenchmark tests used for cross-generational comparison of alignment quality must not drift along with model iterations; otherwise, alignment decay will be undetectable.57% similarUnverifiedThe study argues that Adam's optimization capability may stem from the balance between structural approximation error and momentum smoothing, rather than from precise tracking of the natural gradient path.57% similarUnverifiedAn unbiased Monte Carlo estimator introduces no systematic bias into the expected estimate of the integral, so that gradients point in the correct optimization direction in a statistical sense without impairing the convergence of gradient descent training.57% similarUnverifiedIn small language models, confidence signals based on token entropy almost entirely fail to distinguish between correct and incorrect answers.57% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1035491API
curl https://kongchang.com/api/v1/knowledge/claims/1035491MCP
get_claim(id=1035491)