Unverified50% confidenceFactExact time
In the teacher-student model, the continuous-time limit of online stochastic gradient descent (online SGD) as the learning rate tends to zero is described by a set of ordinary differential equations.
1
Sources
50%
Confidence
Long-term
Relevance
10/22/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn practice, almost no one uses raw batch gradient descent; instead, stochastic gradient descent (SGD) or mini-batch gradient descent (Mini-batch SGD) is adopted64% similarUnverifiedKnowledge distillation has the student network learn the soft probability distribution output by the teacher network, enabling the student model to retain core reasoning capabilities while reducing its parameter count by more than 10 times.63% similarUnverifiedThis separation result echoes a series of computational lower bound works in online learning and online convex optimization in recent years, confirming the general phenomenon that "statistical optimality does not necessarily imply computational feasibility."63% similarUnverifiedThe square root scaling formula for the learning rate is: learning rate multiplied by sqrt(M/N); when scaling from 10 environments to 1000, the learning rate is adjusted to 0.0003 * sqrt(100) = 0.003, but this may be too aggressive in ML-Agents, so it is recommended to start by trying 1.5-2 times.61% similarUnverifiedStudent's t-distribution is a heavy-tailed distribution whose tails decay more slowly than the Gaussian distribution, making it more robust to outliers.60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1083918API
curl https://kongchang.com/api/v1/knowledge/claims/1083918MCP
get_claim(id=1083918)