JOLT
A post-training method based on joint teacher-student reinforcement learning in which the same model acts simultaneously as teacher and student and is trained through on-policy sampling. On the MATH benchmark it reaches accuracy comparable to GRPO with fewer tokens.
Timeline (last 90 days)
JOLT's absolute accuracy on MATH, the model scale at which it matches GRPO, whether the 6.5x token reduction comes at the cost of more training steps or a more complex teacher training phase, and whether there is a risk of capability degradation or mode collapse under shared parameters are all open questions still to be answered.
The core design of JOLT is to let the same model play both the teacher and student roles at once, with the teacher role specially trained to generate feedback signals that the student can exploit.
The currently public JOLT efficiency comparison is limited to the single MATH benchmark, so its performance on code generation, multi-turn dialogue, or open-domain reasoning cannot yet be inferred.
The 6.5x reduction reported by JOLT is measured in tokens; if it requires an additional teacher training phase, more training steps, or multiple forward passes per step (teacher evaluation + student generation), the total FLOPs savings may be noticeably smaller than 6.5x, or may even invert when the model is small.
If JOLT's efficiency advantage generalizes, the most direct beneficiaries are small and mid-sized teams and academic labs constrained by token budgets.
JOLT is not the first work to attempt to compress post-training costs; previous directions include rejection sampling fine-tuning, the direct preference optimization (DPO) family of methods, and various self-rewarding mechanisms.
The specific mechanism by which JOLT prevents reward hacking is not elaborated in the original material, and this may be a technical detail in the paper that warrants close attention.
Research teams from Perplexity and NVIDIA jointly proposed a post-training method called JOLT, which lets the same model play both the teacher and student roles and trains it with joint reinforcement learning.
On the MATH mathematical reasoning benchmark, JOLT achieved the same accuracy as GRPO, but processed only about one-sixth as many training tokens as GRPO.
JOLT achieved roughly a 6.5x improvement in token efficiency (relative to GRPO).
4 more timeline events
All Facts (14)
JOLT's absolute accuracy on MATH, the model scale at which it matches GRPO, whether the 6.5x token reduction comes at the cost of more training steps or a more complex teacher training phase, and whether there is a risk of capability degradation or mode collapse under shared parameters are all open questions still to be answered.
50%UnverifiedThe core design of JOLT is to let the same model play both the teacher and student roles at once, with the teacher role specially trained to generate feedback signals that the student can exploit.
50%UnverifiedThe currently public JOLT efficiency comparison is limited to the single MATH benchmark, so its performance on code generation, multi-turn dialogue, or open-domain reasoning cannot yet be inferred.
50%UnverifiedThe 6.5x reduction reported by JOLT is measured in tokens; if it requires an additional teacher training phase, more training steps, or multiple forward passes per step (teacher evaluation + student generation), the total FLOPs savings may be noticeably smaller than 6.5x, or may even invert when the model is small.
50%UnverifiedJOLT is not the first work to attempt to compress post-training costs; previous directions include rejection sampling fine-tuning, the direct preference optimization (DPO) family of methods, and various self-rewarding mechanisms.
50%UnverifiedThe specific mechanism by which JOLT prevents reward hacking is not elaborated in the original material, and this may be a technical detail in the paper that warrants close attention.
50%UnverifiedJOLT's currently public information covers only the MATH benchmark, and its generalization ability still needs more verification.
50%UnverifiedResearch teams from Perplexity and NVIDIA jointly proposed a post-training method called JOLT, which lets the same model play both the teacher and student roles and trains it with joint reinforcement learning.
50%UnverifiedWhat JOLT's 6.5x token efficiency improvement mainly compresses is the large-scale repeated sampling cost paid to estimate relative advantages.
50%UnverifiedOn the MATH mathematical reasoning benchmark, JOLT achieved the same accuracy as GRPO, but processed only about one-sixth as many training tokens as GRPO.
50%UnverifiedJOLT achieved roughly a 6.5x improvement in token efficiency (relative to GRPO).
50%UnverifiedJOLT uses on-policy training, where the training data is generated by the model's current policy rather than coming from a fixed offline dataset.
50%UnverifiedJOLT's design eliminates the overhead of maintaining two sets of parameters and lets the teacher signal evolve in sync with the student policy, avoiding the teacher lagging behind the student's current capability level due to being frozen.
50%UnverifiedIf JOLT's efficiency advantage generalizes, the most direct beneficiaries are small and mid-sized teams and academic labs constrained by token budgets.
50%