Tev Cracks Long-Horizon Agent Training with Dense Progress Rewards

Open-source agents struggle with long-horizon tasks for two core reasons: a lack of training data with verifiable intermediate states, and a reward signal that can guide multi-hour execution without leaking the final goal. Tev-Gym and its proposed Progress Potential Reward Shaping (PPRS) scheme attempt to solve both at once. This infrastructure targets economically valuable workflows—portfolio rebalancing, earnings analysis, even recursive self-improvement loops spanning multiple days.

The Sparse Reward Dilemma

Most agent benchmarks provide only binary feedback: task complete or failed. Suppose a trading agent needs to collect market data, run valuation models, check compliance rules, and execute trades across four exchanges—until the final step, the signal you get is zero. This makes credit assignment nearly impossible.

Traditional reinforcement learning relies on dense rewards, but hand-designing intermediate rewards for every possible agent task isn't scalable. You need a systematic way to measure progress without encoding the solution path.

Credit assignment is a classic problem in reinforcement learning: when an agent receives only a single reward signal at the end of a task, it struggles to determine which of the dozens or even hundreds of preceding steps actually contributed to the final success or failure. This attribution difficulty is exponentially amplified in long-horizon tasks. There are three mainstream approaches: first, reward shaping, which artificially introduces intermediate rewards to guide exploration; second, hindsight relabeling, which uses successful trajectories to infer key steps; third, hierarchical reinforcement learning, where a high-level policy handles subgoal decomposition and a low-level policy handles concrete execution. PPRS belongs to the first category, but unlike hand-designed intermediate rewards, it leverages a latent world model to automatically compute progress, thereby avoiding hardcoding the solution path into the reward function.

A Latent-World-First Construction Approach

Tev-Gym built 1,170 verified task environments using a three-layer approach:

1. Latent world specification: A complete hidden model of the task domain, workflow, and expected state after each step. This is the ground truth.
2. Progressive rendering: Environments are generated from the specification, so complexity is controllable and every intermediate state is verifiable.
3. Progress metric extraction: Since the specification knows the full workflow, it can measure how far the agent has advanced without leaking the solution.

This inverts the usual benchmark design philosophy. Rather than collecting tasks first and reverse-engineering evaluation criteria, you start with a formal task model and render environments from it.

Progress Potential Reward Shaping

PPRS converts progress metrics into per-step rewards. The key insight: reward the agent for reaching states with higher potential to complete the task, not for specific actions.

The reward function looks roughly like this:

def pprs_reward(state, next_state, task_spec):
    progress_current = task_spec.measure_progress(state)
    progress_next = task_spec.measure_progress(next_state)
    potential_delta = progress_next - progress_current
    return potential_delta + sparse_terminal_reward(next_state)

This satisfies reward shaping invariance: the optimal policy under dense rewards is also optimal under sparse terminal rewards. You haven't changed what the agent should do—you've just made the gradient less noisy.

Potential-based reward shaping originates from a theorem proved by Ng et al. in 1999: if the shaping reward takes the form F(s, s') = γΦ(s') − Φ(s), where Φ is a potential function over states, then the optimal policy obtained under this shaping is exactly identical to the optimal policy of the original sparse-reward problem. This is known as reward shaping invariance. PPRS's potential delta is precisely an instance of this form, with Φ being the progress measure provided by the task specification. Its practical value lies in allowing engineers to confidently design arbitrarily complex progress functions as guidance without worrying about introducing bias or steering the agent toward the wrong objective—policy optimality is theoretically guaranteed, and all that's shortened is the number of samples needed for convergence.

The Checkpoint Granularity Trade-off

Long-horizon tasks require checkpoints, but storage and evaluation costs rise with checkpoint frequency. Tev uses task-aware checkpoints based on the latent world model.

Checkpoint Strategy	Storage Cost	Evaluation Overhead	Catastrophic Forgetting Risk
Every step	High (100% of states)	High (continuous validation)	Low
Fixed interval (every N steps)	Medium	Medium	Medium
Task-aware (workflow milestones)	Low (5-10% of states)	Low (only at decision points)	Low
Terminal only	Very low	Very low	High

Task-aware checkpoints save state only when the latent world model indicates a workflow boundary: data collected, model trained, compliance check passed. This reduces storage by roughly 90% while retaining the ability to resume from meaningful intermediate states.

Catastrophic forgetting is particularly thorny in long-horizon agent training: when a workflow spans thousands of steps and involves multiple sub-skills, fine-tuning the policy for one stage late in training can easily overwrite capabilities learned in earlier stages—manifesting as the agent remaining proficient at data collection but showing clear degradation in model training. Task-aware checkpoints are marked as "low forgetting risk" in the table because they anchor states to semantic boundaries of the workflow, allowing replay and retraining to be isolated by stage and preventing different skills from interfering with each other. In contrast, terminal-only checkpoints, while the most storage-efficient, leave you with no way to resume from intermediate states or conduct staged curriculum training—you can only start from scratch, which actually drives up hidden costs.

Training Recipe and Results

Tev-27B was trained entirely with PPRS on the ALE (Agents' Last Exam) computing and mathematical sciences domain. Despite being open-source and smaller, it matches GPT-6 Astra and Claude Opus 5 on Pass@5 for ALE-CM.

Generalization is the real test. Without seeing any Terminal-Bench data during training, Tev-27B surpassed GLM 5.2 and Gemini 3.7 Flash on TB 3.0 and 4.0 subsets. On medium-horizon tasks (TB 2.1), it reached Claude Opus 4.8 performance.

This suggests the PPRS recipe teaches transferable task decomposition ability, not just domain-specific heuristics.

Deployment Shape for Financial Workflows

A portfolio rebalancing agent using this approach would operate as follows:

1. Define the latent world: asset pool, rebalancing rules, compliance constraints, execution venues.
2. Render the environment: market data feeds, order management systems, compliance APIs.
3. Embed progress probes: checkpoints at data collection, valuation completion, compliance approval, and execution initiation.
4. Train with PPRS: reward advancement through each workflow stage without hardcoding trading logic.
5. Deploy with state persistence: save checkpoints at workflow boundaries so the agent can resume after interruption.

The latent world model becomes your observability layer. You can inspect whether the agent is stuck, regressing, or genuinely advancing toward the terminal objective.

Possible Failure Modes

Reward hacking: If progress metrics are poorly designed, the agent will find shortcuts. If the progress function only counts API calls, a trading agent might mark compliance as complete without actually checking the rules.

Checkpoint bloat: Even with task-aware checkpoints, storage can explode if a workflow has many decision points. You need retention policies and compression.

Generalization boundary: Tev-27B generalizes across task lengths but still within similar domains (computing, math). Jumping to entirely new domains (legal research, hardware design) will likely require retraining.

Latent world drift: If the real environment changes (new compliance rules, deprecated APIs), the latent world model becomes stale. You need a process to update specifications and re-render environments.

Technical Conclusions

When to use Tev-style PPRS:

• Tasks have clear intermediate milestones (data collection, model training, approval gates).
• You need open-source models to handle multi-hour workflows.
• You can invest resources in building a latent world model for your domain.
• Storage and compute budgets allow for task-aware checkpoints.

When to avoid it:

• Tasks are genuinely unstructured with no natural progress metric.
• You need sub-second latency for real-time execution (checkpoint overhead is too high).
• The environment changes faster than you can update the latent world specification.
• You already have a closed-source model that handles long-horizon tasks and don't need training infrastructure.

The real contribution isn't the specific model—it's the recipe: latent-world-first design plus progress potential reward shaping. For any domain that needs agents to learn multi-day workflows, this is reusable infrastructure.

Source Links

• Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards (arXiv:2610.18868v1)
• PDF

One-line Summary

Tev-Gym uses latent world specifications and progress potential reward shaping to solve the training bottleneck for open-source agents on long-horizon tasks.

Paragraph Summary

Tev-Gym addresses two bottlenecks in long-horizon agent training—the lack of verifiable intermediate state data and reward signals that guide multi-step execution—by proposing a three-layer construction method and PPRS reward shaping. It first formalizes latent world specifications, then progressively renders 1,170 verifiable environments, and extracts progress metrics from them without leaking solutions. PPRS rewards reaching states with higher task-completion potential rather than specific actions, satisfying reward shaping invariance without altering the optimal policy. Combined with task-aware checkpoints, storage is reduced by roughly 90%. Tev-27B matches GPT-6 Astra and Claude Opus 5 on ALE-CM, and surpasses GLM 5.2 and Gemini 3.7 Flash on unseen Terminal-Bench subsets, demonstrating transferable task decomposition ability.