Perplexity and NVIDIA Launch JOLT: A New Teacher-Student Joint Reinforcement Learning Method

Researchers from Perplexity and NVIDIA have jointly proposed a post-training method called JOLT, whose core idea is to have a single model play both teacher and student roles simultaneously, completing training through joint reinforcement learning. According to MarkTechPost, on the MATH mathematical reasoning benchmark, JOLT achieves the same accuracy as GRPO while processing only about one-sixth the number of training tokens—a 6.5x token efficiency improvement that is the biggest selling point of this work.

One Model, Two Identities

Traditionally, distillation or teacher-student frameworks require two separate models: the teacher generates high-quality signals, and the student learns from them. JOLT breaks this division of labor by having a single model simultaneously assume both teaching and learning responsibilities during training. This design eliminates the overhead of maintaining two sets of parameters and allows the teacher signal and student policy to evolve in sync—the teacher won't fall behind the student's current capability level due to being frozen.

From a reinforcement learning perspective, this amounts to internalizing the question of "who provides the reward signal": another facet of the model itself becomes the reference frame, with policy updates and signal generation completed within the same loop.

What 6.5x Token Efficiency Means

GRPO (Group Relative Policy Optimization) is one of the mainstream post-training methods for current reasoning models, widely used in tasks requiring long-chain reasoning such as math and code. Its cost is the need for extensive sampling to estimate relative advantages, keeping token consumption high.

JOLT claims to achieve equivalent accuracy on MATH with 6.5x fewer training tokens. If this figure can be reproduced on more tasks, it directly translates to compressed training costs and iteration cycles. For teams that need to repeatedly conduct post-training experiments, token efficiency often determines research pace more than absolute accuracy. However, it should be noted that currently available public information covers only the MATH benchmark, and generalization capability still requires further validation.

GRPO was proposed by the DeepSeek^Ad team in their DeepSeekMath work. Its core approach is to sample a group of multiple responses to the same question and use the relative magnitude of rewards within the group (subtracting the group mean, dividing by the group standard deviation) to estimate the advantage value of each response, thereby eliminating the overhead of separately training a value network (critic). Precisely because of this, its sampling volume grows linearly with group size—for long-chain reasoning tasks, a single sample can run to thousands of tokens, keeping training token consumption high. Understanding this is essential to seeing exactly which portion of cost JOLT's claimed 6.5x efficiency improvement compresses—primarily the large-scale repeated sampling required to estimate relative advantages.

Where Do Teacher Signals Come From

The key challenge of joint training is: if the teacher and student are the same model, will the signal degenerate into self-confirmation? JOLT's solution lies in "on-policy" training—training data is generated by the model's current policy rather than coming from a fixed offline dataset. The teacher role provides judgments based on these immediately generated trajectories, and the student role updates accordingly. This closed loop keeps the training distribution always close to the model's current capability boundary, avoiding distribution drift between offline data and online policy.

This design aligns with the recent trend in reinforcement learning post-training that emphasizes on-policy data, but how to prevent signal collapse under the teacher-student unified setting is an engineering detail worth attention in the method design that the original paper does not elaborate on.

The difference between on-policy and off-policy: On-policy means the trajectories used for training are sampled in real-time by the current policy, and old data is discarded after policy updates; off-policy allows reuse of historical data or fixed datasets, offering higher sample efficiency but carrying the risk of distribution mismatch. Under the teacher-student unified setting, on-policy is particularly critical: if the teacher scores based on outdated trajectories, its judgments will deviate from the student's current capability boundary, causing signal distortion. The trade-off is that resampling is required after each round of updates, which is why JOLT must offset sampling overhead by increasing information density per token.

Background and Open Questions

Perplexity has visibly invested in developing its own reasoning models in recent years, while NVIDIA has continued to build out training infrastructure and model research. The post-training methods produced by their collaboration point to the practical need to "trade less compute for stronger reasoning."

Several points that warrant careful consideration:

• The comparison baseline includes only GRPO and the MATH benchmark, lacking horizontal comparisons with other distillation methods and more task types;
• The specific statistical basis for the 6.5x token efficiency (whether teacher-side overhead is included) remains unclear;
• The boundary conditions of teacher-student unification on training stability have not been sufficiently ablated in publicly available materials.

JOLT offers an interesting direction: compressing the teacher-student relationship from "two models" to "two facets of one model." Whether it can evolve from a single benchmark to a general post-training pipeline depends on reproduction results on more tasks.

Knowledge distillation and teacher-student frameworks: Classical distillation involves an independent, typically frozen teacher model outputting soft labels or reasoning trajectories, with the student model fitting these signals. Its limitation is that teacher capability is fixed, signal gains diminish as the student catches up, and two sets of parameters bring memory and inference overhead. JOLT's "unified" design specifically addresses this limitation: the teacher evolves in sync with training, and the signal always stays slightly above the student's current level, forming a continuously learnable gradient. However, this also introduces the risk of self-confirmation, which needs to be constrained through sampling diversity or judgment mechanism design.