Unverified50% confidenceFactExact time
Experimental conclusion two: some self-distillation methods give attractive scores at the end of training, but during training they exhibit severe performance fluctuations or even collapse followed by recovery, and these problems cannot be found by looking only at the final metric.
1
Sources
50%
Confidence
Long-term
Relevance
10/15/2026
First Seen
Sources
Related Claims
UnverifiedThe first attempt at the reproduction experiment failed to observe the degradation phenomenon because too few training epochs were used (the model was still in an underfitting state), indicating that sufficient convergence during training is a prerequisite for observing the degradation phenomenon.80% similarUnverifiedExperiments show that repeating training on the same data for more than 4 epochs yields diminishing performance gains and can sometimes result in slight performance degradation.75% similarUnverifiedAblation testing diagnoses whether a feature is effective by setting its weight to zero one at a time and observing performance changes: if performance is nearly unchanged after disabling it, the feature design is problematic or redundant; a significant drop proves it is contributing.73% similarUnverifiedWithin the same training batch, rollout groups where all attempts either fail or succeed may appear; in such cases the group-relative reward signal becomes invalid, as there is no gradient information reflecting relative quality among samples within the group.73% similarUnverifiedFor agentic workflows involving multi-step tool calls, even slight degradation at any single step can cause the final result to collapse, and without step-by-step evaluation it is nearly impossible to pinpoint the problematic stage.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1038114API
curl https://kongchang.com/api/v1/knowledge/claims/1038114MCP
get_claim(id=1038114)