SGUID
A filtering method for skill library distillation in LLMs that retains only skills that consistently produce effective learning signals during training. It supports co-evolution of the model and the skill library, using an iterative loop so that the library and the model's capabilities drive mutual improvement.
Timeline (last 90 days)
The SGUID study points out that in on-policy distillation, fewer than 25% of the retrieved skills can provide effective learning signals.
The core filtering logic of SGUID is: only when a certain skill continuously produces effective learning signals during training is it retained.
SGUID is designed to support the co-evolution of the model and the skill library: after each round of distillation, a new candidate skill library is extracted from the rollouts of the updated model, and then the skills to be internalized in the next round are filtered, iterating in a loop.
On four models from the Olmo and Qwen series, after distilling 6 selected skills, three of the models match or exceed the effect of distilling the complete skill library on the average avg@12.
After the second round of distilling 3 newly selected skills, all four models reach or exceed the level of the complete skill library, while at this point the size of the complete skill library is at most 11 times that of the skills selected by SGUID.
On Qwen3-8B, after the second round of the loop selected 3 new skills, model performance improved from 64.3% to 66.3%.
In the Qwen3-4B comparative experiment, updating the model directly with unfiltered skills without using SGUID filtering decreased performance by 0.3 percentage points on HMMT25.
When using SGUID, Qwen3-4B improved by 0.5 percentage points on HMMT25 after the first round, and by a cumulative 1.1 percentage points after the second round.
In model-skill co-evolution, if a certain round mistakenly admits a low-quality skill, the error will be amplified and contaminate all subsequent rounds.
Skill filtering is not an optimization option, but a necessary mechanism for stable co-evolution.
All Facts (10)
The SGUID study points out that in on-policy distillation, fewer than 25% of the retrieved skills can provide effective learning signals.
50%UnverifiedThe core filtering logic of SGUID is: only when a certain skill continuously produces effective learning signals during training is it retained.
50%UnverifiedSGUID is designed to support the co-evolution of the model and the skill library: after each round of distillation, a new candidate skill library is extracted from the rollouts of the updated model, and then the skills to be internalized in the next round are filtered, iterating in a loop.
50%UnverifiedOn four models from the Olmo and Qwen series, after distilling 6 selected skills, three of the models match or exceed the effect of distilling the complete skill library on the average avg@12.
50%UnverifiedAfter the second round of distilling 3 newly selected skills, all four models reach or exceed the level of the complete skill library, while at this point the size of the complete skill library is at most 11 times that of the skills selected by SGUID.
50%UnverifiedOn Qwen3-8B, after the second round of the loop selected 3 new skills, model performance improved from 64.3% to 66.3%.
50%UnverifiedIn the Qwen3-4B comparative experiment, updating the model directly with unfiltered skills without using SGUID filtering decreased performance by 0.3 percentage points on HMMT25.
50%UnverifiedWhen using SGUID, Qwen3-4B improved by 0.5 percentage points on HMMT25 after the first round, and by a cumulative 1.1 percentage points after the second round.
50%UnverifiedIn model-skill co-evolution, if a certain round mistakenly admits a low-quality skill, the error will be amplified and contaminate all subsequent rounds.
50%UnverifiedSkill filtering is not an optimization option, but a necessary mechanism for stable co-evolution.
50%