Unverified50% confidenceFactExact time
Introducing random depth during pretraining induces contractive residual dynamics, pulling the output of each loop toward a common fixed point so that representations across loops become increasingly similar.
1
Sources
50%
Confidence
Long-term
Relevance
10/23/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedOver-reliance on the fine-tuned part during fine-tuning may be precisely the root cause of rare-class capabilities being obscured; directing the model's computation moderately toward the pretrained components may be a more robust fine-tuning route.74% similarUnverified训练时不能一开始就追求快，若直接压步数模型容易陷入死循环反复生成重复字符，只做第一步的版本碰上难题会卡死，需先练准再练快73% similarUnverified即便固定所有随机种子，在不同硬件或并行度下，训练结果仍可能出现被大模型放大为显著能力差异的细微偏差73% similarUnverifiedWith a fixed learning_rate, when batch_size scales linearly with the number of environments and num_epoch remains unchanged, the total displacement pushing the policy toward a new distribution in a single iteration may increase, manifesting as violent oscillation or even collapse of rewards in the early stages of training.72% similarUnverified域随机化的代价是早期训练信号更加嘈杂、收敛更慢，但换来策略在整个参数空间内均匀分布的泛化能力72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1088245API
curl https://kongchang.com/api/v1/knowledge/claims/1088245MCP
get_claim(id=1088245)