Continual Harness: AI Automatically Builds Scaffolding to Beat Pokémon RPGs

Introduction: When AI Starts Rebuilding Its Own Toolchain

Princeton University and Google DeepMind have jointly published a groundbreaking paper proposing a continual self-improvement framework for embodied agents — Continual Harness. The core achievement of this research is quite impressive: instead of relying on human-designed scaffolding, the AI automatically constructs and optimizes its own tools and strategies during gameplay, ultimately beating multiple Pokémon RPG games.

This is not just a breakthrough in game AI — it's an important milestone on the path toward general embodied intelligence. It answers a key question: Can AI, starting from the most basic environment interfaces, continuously improve its capabilities without resetting or relying on human intervention?

Why Embodied Intelligence Needs Continual Harness

In the coding domain, LLM-based agents already have mature Harness frameworks — tools like Claude Code and Open Hands that let models invoke various tools, maintain cross-interaction state, and complete complex coding tasks. But in the embodied intelligence domain — scenarios requiring continuous interaction with physical or virtual environments — there's no equivalent general-purpose solution yet.

Embodied Intelligence refers to AI systems that don't just perform abstract reasoning but must continuously interact with physical or virtual environments through perception and action to accomplish tasks. Unlike pure text conversations, the core challenge for embodied agents is that the environment state changes due to their own actions, and decisions must account for temporal dependencies and long-term consequences. Scaffolding originally referred to temporary auxiliary code structures in software engineering, but in AI it has evolved into the tool layer and prompt layer built around large models — providing structured input/output interfaces, tool-calling capabilities, memory management, and task decomposition mechanisms that enable models to handle complex tasks far beyond single-inference capabilities.

The prior PokeAgent challenge already demonstrated that without domain-customized scaffolding, even frontier multimodal models can barely make any progress in RPG games like Pokémon. Pokémon RPGs serve as an ideal testbed for embodied intelligence because they combine multiple cognitive challenges: spatial navigation (pathfinding in complex maps), strategic decision-making (type-advantage turn-based battles), long-term planning (team development and resource management), and natural language understanding (NPC dialogues and quest clues). The PokeAgent challenge revealed a harsh reality: even top multimodal models like GPT-4V and Gemini often can't complete basic town navigation without specially designed auxiliary frameworks, let alone beat an entire game. This is exactly the core pain point that Continual Harness aims to solve.

Gemini Plays Pokemon: From Manual Iteration to Automatic Evolution

Project Evolution

The Gemini Plays Pokemon (GPP) project initially used a Human-in-the-loop approach to optimize the Harness. Human-in-the-loop is a human-AI collaboration paradigm where human experts continuously monitor, correct, and guide model behavior during system operation. In GPP's early stages, researchers needed to repeatedly watch the AI's gameplay recordings, identify stuck points, and then manually adjust the scaffolding design — iterating step by step from a simple interface with just screenshots and button inputs to a complex multi-agent architecture.

Multi-agent architecture decomposes a complex task across multiple specialized AI roles: for example, a battle agent for combat decisions, a pathfinding agent for map navigation, and a resource agent for inventory management. This division of labor borrows from the microservices philosophy in software engineering, where each sub-agent only handles the sub-problem it specializes in, reducing the cognitive load on any single model.

Later, the research team made a bold decision: remove all manually written sub-agents, providing the model only with Primitive Tools, and letting it build sub-agents and reusable scripts on its own during gameplay.

The results of this approach are impressive:

• May 2025: Beat Pokémon Blue
• August 2025: Beat the high-difficulty Pokémon Yellow Legacy Edition
• November 2025: Beat Pokémon Crystal

Becoming the first AI system to complete multiple Pokémon RPGs.

Spontaneous Continual Optimization Behavior

More interestingly, during the high-difficulty phases of Yellow and Crystal, the model had already begun spontaneously iterating strategies through long-context memory — this is essentially a form of spontaneous continual Harness behavior. What the paper does is formalize and automate this process.

Continual Harness Core Architecture Explained

Four Core Components

Agentive Harness is the scaffolding layer between the large model and the environment, comprising four core components:

1. System Prompt: Provides instructions and strategic guidance for each reasoning step
2. Sub-agents: Handle specific tasks such as battles, strategy, puzzles, and self-reflection
3. Skills: Include heuristic rules for reasoning and executable programs like pathfinders and tool wrappers
4. Memory: Accumulates facts, strategies, and observations throughout the trajectory

Crucially, the Harness also provides primitive tools that let the AI directly add, delete, or modify these four components.

Comparison of Three Harness Types

The paper compares three different levels of Harness:

Type	Characteristics
Minimal Harness	Only environment interfaces and generic prompts; no sub-agents, memory, or custom skills
Expert Harness	Fully human-designed with built-in pathfinding algorithms, type matchup tables, damage calculators, etc.
Primitive Harness	Only provides primitive tools; the model builds everything it needs on its own

Dual-Loop Architecture Design

Continual Harness employs an elegant dual-loop architecture:

• Inner Loop: Standard agent interaction steps where the model outputs actions based on observations
• Outer Loop: Harness optimization where the Refiner periodically reads recent trajectory windows, identifies failure patterns (navigation loops, tool call failures, goal stagnation, etc.), and generates corresponding edit operations

The optimization process has four steps corresponding to the four components: rewrite system prompt → create/edit/delete sub-agents → codify successful action sequences into new skills → supplement and update memory entries.

Core Advantage: No Reset Required

This design has two key advantages:

1. Optimization quality accumulates over time: Unlike reset-based methods that need to re-accumulate failure information after each update
2. Can handle failure patterns that only appear later: High-difficulty battles, multi-step puzzles, long dialogue chains — things that reset-based methods simply never encounter

In practice, many long-cycle tasks (continuously running coding tasks, embodied robot tasks, operations and maintenance tasks) simply cannot reset their environments at will, making this no-reset approach more aligned with real-world application scenarios.

Experimental Results: Automatic Optimization Approaches Human-Designed Performance

Cost and Efficiency Comparison

Experiments on Pokémon Red and Emerald show that Continual Harness, compared to Minimal Harness, significantly reduces the operational cost per milestone, closing most of the efficiency gap between Minimal and Expert Harness — all without accessing the game's decompiled code or milestone lists.

For the strongest model, Gemini 3 Pro:

• Continual Harness spent approximately $130 to complete 100% of milestones
• Minimal Harness spent $215 to complete only 90%
• Cost reduced by approximately 40%, with higher completion rate

Model Capability Threshold: Not All Models Benefit

An important finding is that Continual Harness has a clear model capability threshold for effectiveness.

• Gemini 3 Pro (strong model): Significant and stable benefits
• Gemini 3 Flash (medium model): High variance in benefits, unstable performance
• Gemini 3 Flash Lite (weak model): All Continual Harness variants actually performed worse than Minimal Harness

This indicates that models with insufficient capability become distracted by the Harness optimization tasks, diluting their core interaction abilities and leading to overall performance degradation. Not every model benefits from this framework. This finding aligns with the widely discussed "tool-use capability threshold" phenomenon in AI — models need to reach a certain baseline reasoning ability to effectively leverage external tools rather than being dragged down by their complexity.

Joint Learning Validation with Open-Source Models

Researchers also validated joint learning of model weights and Harness state using the Gemma 4 series of models. Key findings include:

• Supervised fine-tuning and offline reinforcement learning warm-up alone cannot produce milestone progress
• Only after entering the joint learning loop does sustained game progress improvement begin to appear
• Whether training starts from the game's beginning or a mid-game checkpoint, the improvement curves have very similar shapes

Here it's important to understand the difference between two training methods: Supervised Fine-Tuning (SFT) adjusts model parameters using labeled input-output pairs to improve performance on specific tasks; Offline Reinforcement Learning (Offline RL) learns policies from pre-collected historical data without real-time environment interaction. The key finding of this research is that neither SFT nor Offline RL alone is sufficient for the model to make substantial progress — only by combining model weight updates with Harness state optimization into a joint learning loop can sustained capability improvement be achieved. This suggests a deep coupling between model capabilities and tool capabilities.

This joint learning approach echoes the concept of co-evolution in machine learning: just as genes and environment mutually shape each other in biological evolution, a model's weights and its toolchain also adapt to each other. A stronger model can write better tools, better tools let the model handle more complex tasks, which generates higher-quality training data, forming a positive feedback loop.

Direct Evidence of Skill Self-Optimization

To confirm that skill optimization is indeed the cause of performance improvement, researchers used Dijkstra's algorithm as a pathfinding Oracle for comparison. Dijkstra's algorithm is a classic shortest-path algorithm in graph theory, proposed by Dutch computer scientist Edsger Dijkstra in 1956, capable of finding the absolute shortest path from a source to all other nodes in a weighted graph. In this study, it serves as the theoretically optimal baseline with complete game map information — comparing AI-generated pathfinding skills against Dijkstra's optimal solution quantitatively measures how far the AI's pathfinding ability is from the theoretical optimum.

Results show that the from-scratch version's path cost gap rapidly dropped from nearly 50% to single digits and remained consistently low. This means the AI's self-learned pathfinding strategy is already very close to the optimal algorithm with perfect information.

The frozen-optimization version's path cost gap remained essentially flat, directly proving that skills are indeed continuously self-optimizing, gradually approaching the optimal solution.

Current Limitations and Future Research Directions

This work also has some noteworthy limitations:

1. Model capability threshold: Models with insufficient capability actually perform worse with this framework. Current open-source models cannot simultaneously serve as both Refiner and Agent
2. Unknown convergence limits: Joint learning experiments only demonstrate continuous improvement within the training period, without finding the convergence limit. Whether the two interdependent optimization processes (model weights and Harness state) necessarily converge to a stable state currently lacks rigorous mathematical proof
3. Lack of comparison with traditional methods: No head-to-head comparison with reset-based batch training has been conducted

Conclusion: An Important Step Toward General Embodied Intelligence

The core value of this research lies in demonstrating that: embodied agents can start from the most basic environment interfaces and automatically construct and optimize their own scaffolding, achieving most of the efficiency of human-designed expert systems without human intervention or environment resets.

This continual self-improvement framework can be extended in the future to home service robots, continuous operations and maintenance systems, open-world game AI, and many other long-cycle embodied interaction scenarios. When AI can not only execute tasks but also continuously optimize its own tools and strategies during execution, we've taken one more step toward truly general embodied intelligence.

Key Takeaways

• Princeton and DeepMind propose the Continual Harness framework, enabling embodied agents to automatically build and optimize scaffolding within a single game session without human intervention or environment resets
• The AI system powered by this framework becomes the first to beat multiple Pokémon RPGs, with auto-generated Harness achieving most of the efficiency of human-designed expert systems at approximately 40% lower cost
• The framework uses a dual-loop design: the inner loop executes agent interactions while the outer loop uses a Refiner to analyze trajectories and automatically optimize four components — system prompts, sub-agents, skill libraries, and memory stores
• Experiments reveal that Continual Harness has a model capability threshold — only sufficiently strong models benefit, while weaker models perform worse due to optimization tasks diluting their interaction capabilities
• Open-source models achieve continual evolution through joint learning of weights and Harness state, validating the feasibility of co-optimizing models and scaffolding