Unverified50% confidenceFactExact time
Theoretical analysis shows that under MuonGS the weight norm grows by at most a factor of √t after t steps, guaranteeing a cumulative effective learning rate C_T = Ω(√T).
1
Sources
50%
Confidence
Long-term
Relevance
10/28/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe square root scaling formula for the learning rate is: learning rate multiplied by sqrt(M/N); when scaling from 10 environments to 1000, the learning rate is adjusted to 0.0003 * sqrt(100) = 0.003, but this may be too aggressive in ML-Agents, so it is recommended to start by trying 1.5-2 times.66% similarUnverifiedWhen T=1 the output is a standard softmax distribution; when T>1 the probability distribution becomes more uniform—the higher the temperature, the flatter the output distribution, and the more fine-grained information the student model captures.63% similarUnverifiedIn robust regression, replacing the likelihood function from Gaussian to a t-distribution is equivalent to reducing the weight of outliers, because the log-likelihood of the t-distribution penalizes extreme values more slowly.62% similarUnverifiedThe rule of thumb given by the Chinchilla scaling law is that the number of training tokens should be approximately 20 times the number of parameters61% similarUnverifiedOn dense models, compared with MuonH, MuonGS is estimated to achieve about a 15% token-equivalent speedup when matching validation loss.61% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1105999API
curl https://kongchang.com/api/v1/knowledge/claims/1105999MCP
get_claim(id=1105999)