Conceptresponse-length inflation
Response Length Inflation
The phenomenon in which the output length of the student grows continuously during distillation. Because the token-level reverse KL rewards the student for imitating the teacher's verbose style, the average response length increases, driving up FLOPs during the rollout stage and inference latency.
Timeline (last 90 days)
Nov 6
In the external-teacher setting, OPD frequently suffers from response length inflation, leading to higher inference costs.
Unverified50%
Nov 6
Distillation from external teachers requires careful assessment of the hidden costs brought by response length inflation.
Unverified50%