Unverified50% confidenceBenchmarkExact time
In experiments with a frozen teacher-reachable state, the gap reduction achieved by the 4B student trained with CoRD within 16 steps was still not reached by online distillation after 80 steps.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/6/2026
First Seen
Valid until: 2/4/2027
Sources
Related Entities
Related Claims
UnverifiedThe 6.5x reduction reported by JOLT is measured in tokens; if it requires an additional teacher training phase, more training steps, or multiple forward passes per step (teacher evaluation + student generation), the total FLOPs savings may be noticeably smaller than 6.5x, or may even invert when the model is small.71% similarUnverifiedTraining uses batch size 256, 400,000 steps, EMA 0.9999, linear learning rate decay over the final quarter, and torch.compile achieves a 2.4× speedup over eager mode.70% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1141701API
curl https://kongchang.com/api/v1/knowledge/claims/1141701MCP
get_claim(id=1141701)