LongZero: Zero-Data Reinforcement Learning Boosts LLM Long-Context Reasoning

Long-context reasoning has long been a weak spot for large language models (LLMs). Reinforcement learning (RL) has proven to be an effective means of improving this capability, but traditional RL methods rely heavily on high-quality training data—data that must simultaneously be long, diverse, challenging, and verifiable. Human annotation is costly, natural data is scarce, and existing synthesis methods depend on external corpora or stronger teacher models—a dependency that becomes increasingly difficult to sustain as the model being trained itself grows stronger. The recently proposed LongZero method offers an elegant solution: let the model improve its long-context reasoning entirely using data it generates itself, with no external data required.

The Bottleneck of Zero-Data RL and LongZero's Breakthrough

The core idea of "RL from zero data" is to let a model learn from tasks it proposes itself, and in recent years it has achieved impressive results on short-context tasks. However, transferring this paradigm to long-context scenarios faces a fundamental contradiction: if the model directly writes question-answer pairs based on existing documents, the difficulty of the questions and the distribution of evidence will be constrained by the model's own existing capabilities, making it difficult to produce genuinely challenging long-context reasoning tasks. LongZero's approach is clever—it does not require the model to directly create questions over long documents. Instead, it first has the model synthesize questions, answers, and their supporting evidence in a compact structured form, and then separately expands the evidence into multi-document long contexts. This two-stage "compress first, expand later" design enables the model to autonomously construct complex tasks requiring cross-document multi-hop reasoning without relying on any ready-made long-text corpus.

Multi-hop Reasoning refers to a form of reasoning that requires chaining together multiple scattered pieces of evidence to reach a conclusion. For example, answering "Which university did a certain company's founder attend?" requires first finding out who the founder is, and then looking up their educational background. In long-context scenarios, the difficulty of multi-hop reasoning lies in the fact that evidence may be distributed across different documents or paragraphs, and the model needs to first locate and then integrate it. LongZero, through its approach of "first structurally synthesizing questions, answers, and evidence, then expanding into multi-document contexts," ensures that generated tasks naturally contain cross-document evidence chains, thereby systematically creating multi-hop requirements rather than relying on multi-hop relationships the model happens to discover in existing long texts.

Dual-Task Self-Generation and GRPO Training

LongZero constructs two types of self-generated tasks: one is multi-hop reasoning tasks, i.e., the structured synthesis and expansion described above; the other is multi-turn retrieval tasks, where the model must write questions that require multiple rounds of retrieval to answer. These two types of tasks are mixed together, and the model is trained using the GRPO (Group Relative Policy Optimization) algorithm. GRPO is a policy optimization method that does not require a value network and is well suited to scenarios with self-generated reward signals. By combining multi-hop reasoning with multi-turn retrieval, LongZero is in effect training the model to possess both "deep reasoning" and "broad retrieval"—the two core long-context capabilities—rather than favoring one over the other.

GRPO (Group Relative Policy Optimization) is a reinforcement learning algorithm proposed by DeepSeek^Ad. Its core feature is that it does not use an independent value network (critic) to estimate a baseline; instead, it samples a group of responses to the same question and uses the mean reward within the group as the baseline to compute the advantage function. This significantly reduces memory and computational overhead while preserving the stability of policy gradients. In scenarios like LongZero where rewards are self-generated, the reward signal comes from the verifiability of the tasks the model constructs itself (e.g., whether the answer is consistent with the evidence). GRPO can effectively optimize the policy without training an additional value model, making it particularly well suited to the zero-data RL training pipeline.

Experimental Performance: Surpassing External-Corpus Synthesis Baselines

The researchers evaluated LongZero on two LLMs and five long-context reasoning benchmarks, comparing it with baseline methods that use RL training on data synthesized from external corpora. The results show that LongZero not only generalizes to real tasks but also outperforms both baselines. The significance of this result is that it demonstrates that long-context RL training entirely based on the model's self-generated data is feasible, and its performance is on par with or even better than methods relying on external corpora. For practical applications, this means that as model capabilities grow, training data no longer becomes a bottleneck—the model can continuously self-evolve without constantly searching for harder external data sources.

Impact on the Long-Context RL Training Paradigm

The value of LongZero lies not only in a specific method but also in opening up a path: improving long-context reasoning capability can be freed from dependence on external data supply. The traditional view holds that long-context RL requires "longer, harder, more diverse" data, and such data is naturally scarce. LongZero takes the opposite approach, letting the model define difficulty and length itself and ensuring task quality through structured synthesis and expansion mechanisms. Of course, this method also has its boundaries—the quality ceiling of self-generated tasks is constrained by the model's own capabilities, and how to ensure the diversity and verifiability of self-generated data remains an issue requiring continued exploration. But at the very least, LongZero proves that zero-data long-context RL is a path worth pursuing.

Verifiability is a key prerequisite for zero-data RL to work. RL training requires reliable reward signals, and if self-generated tasks cannot be automatically judged as correct or incorrect, they cannot provide a stable learning signal. LongZero ensures through structured synthesis that every question comes with explicit supporting evidence and an answer, so that rewards can be computed based on rules or model self-checks, thereby bypassing human annotation. But this also introduces an inherent tension: the stronger the verifiability of self-generated data, the more restricted the task format often becomes. How to improve task diversity and difficulty while maintaining verifiability is an engineering and algorithmic problem that must be solved for zero-data long-context RL to mature.

One-line Summary

LongZero enables LLMs to improve long-context reasoning using only self-generated data, through structured multi-hop tasks and GRPO training.

Paragraph Summary

LongZero proposes a zero-data reinforcement learning method that enables large language models to improve long-context reasoning entirely through self-generated data, without external corpora or stronger teacher models. Its core is a two-stage "compress first, expand later" design: the model first synthesizes questions, answers, and supporting evidence in a structured form, then expands the evidence into multi-document long contexts, thereby autonomously constructing cross-document multi-hop reasoning tasks; it also mixes in multi-turn retrieval tasks and trains with the GRPO algorithm. Experiments on two LLMs and five long-context benchmarks show that LongZero generalizes to real tasks and surpasses RL baselines based on external-corpus synthetic data, demonstrating that zero-data long-context RL is feasible and providing a new path for continuous model self-evolution.