Unverified50% confidenceFactExact time
In non-private training, μP has been shown to enable zero-shot learning-rate transfer from narrow to wide on Transformers.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedUnsloth通过内核级优化实现了相比原生Hugging Face Transformers训练流程的训练速度大幅提升，同时降低了显存占用72% similarUnverifiedA new research work proposes that a pretrained Transformer can be turned into a single elastic model without retraining, capable of adapting on the fly at inference time to a range of computation budgets from high to low.71% similarUnverifiedGoogle's Switch Transformer demonstrated in 2021 that routing each token to only a single expert per layer (Top-1 routing) within a Transformer can achieve stable training, improving pre-training speed by more than 7× compared to a dense model of equivalent scale.71% similarUnverifiedThe work of REPA and RECON shows that the semantic information learned by the early layers of a diffusion Transformer may not automatically flow to the later layers, and that designing supervision signals hierarchically and in stages may be an underestimated direction for improving training efficiency.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1068355API
curl https://kongchang.com/api/v1/knowledge/claims/1068355MCP
get_claim(id=1068355)