RSIGame: Letting LLMs Autonomously Develop Games with Recursive Self-Improvement

Recent advances in large language models and multimodal models have enabled AI agents to tackle increasingly complex real-world tasks — from software engineering to computer-use automation. Autonomous game development is emerging as a particularly promising new frontier, requiring AI to simultaneously design mechanics, write code, produce multimodal assets, and integrate these heterogeneous components into an executable, playable project. The paper RSIGame: Autonomous Agentic Game Development With Recursive Self-Improvement presents a complete framework specifically designed to meet this challenge.

The Real Difficulty of Automated Game Development

Existing benchmarks have demonstrated that vision-language model-based agents can translate natural language requirements into playable games with recognizable mechanics and visual content. But producing a "working initial version" is far from the finish line.

Game development is fundamentally a complex, multimodal engineering process. Even experienced human teams regularly encounter unexpected bugs, missing features, and overlooked edge cases. The industry therefore universally adopts iterative development workflows: repeatedly testing the current build, collecting feedback, and revising the project until it reaches the desired quality. This process is itself a form of Recursive Self-Improvement (RSI).

The problem is that prior RSI approaches tend to converge on "brittle solutions" — ones that appear to broadly satisfy the goal but leave behind untested bugs, missing behaviors, and unresolved edge cases. The development loop overfits to a small set of test cases without genuinely improving overall game quality, causing performance to degrade sharply when faced with a broader range of player interactions.

Recursive Self-Improvement (RSI) in AI broadly refers to a system iteratively enhancing its own capabilities using its own outputs. In code/software generation, RSI typically manifests as: generate a draft → run tests → revise based on failures → test again, in a continuous loop. The appeal of this paradigm is that it requires no additional human annotation — execution feedback replaces human judgment. However, "overfitting to a small test suite" is a well-known pain point of RSI in code generation: when the test set is small or insufficiently comprehensive, models learn to "pass these specific tests" rather than "genuinely solve the problem," leaving them brittle in broader settings. Game development is especially prone to exposing this flaw, since the space of player interactions far exceeds what any finite test suite can cover.

RSIGame's Core Architecture: Local Loops and Global Loops

The authors' approach is to strengthen the testing phase, extensively exploring remaining failure modes and potential improvement opportunities, with the goal of "developing a high-quality game, not just a playable one." To this end, RSIGame organizes development into complementary local and global loops.

Local Loop: Explore–Diagnose–Improve

The local loop performs fine-grained recursive refinement through three phases. In the autonomous exploration phase, two complementary agents collaborate: a controller decides which aspect of the game to explore next, while an explorer actually interacts with the executable game, gathering behavioral evidence and discovering specific bugs, missing behaviors, and improvement opportunities.

The problem diagnosis and improvement phase also uses a dual-agent structure: an editor translates discovered issues into concrete modifications and applies them, while a validator interacts with the updated project to determine whether the target issue was resolved and whether any regressions were introduced.

Running throughout the entire process is an evolving development checklist. It records discovered issues, improvement opportunities, priorities, and validation results, allowing all agents to collaborate from a consistent view of the development state. The controller reads the checklist to determine direction; the explorer expands it with new evidence; the editor addresses high-priority items; the validator updates item statuses. Each iteration thus builds on verified progress rather than starting from scratch.

Global Loop: Quality Monitoring and Convergence Control

While the local loop focuses on improvements within a single round, the global loop manages overall coordination across phases. A game quality monitor tracks cumulative quality and preserves the best checkpoint; a convergence control mechanism decides whether development should continue, terminate, or enter a new phase with additional high-level guidance.

Unlike the local validator — which evaluates individual edits — the quality monitor assesses the game's accumulated quality holistically and is completely isolated from the benchmark evaluator, with no access to scoring rubrics or feedback, to ensure fairness. When successive global evaluations fail to produce a better checkpoint, the system treats this sustained lack of progress as convergence, either returning to the best-preserved version or introducing strategic high-level guidance from a human or stronger model (rather than specific editing instructions).

From Training-Free Improvement to Experience Internalization

Beyond training-free self-improvement, RSIGame also introduces training-based "experience internalization": successful development trajectories, diagnostic decisions, and verified revisions are converted into supervised training data for the underlying generator, boosting its fundamental capabilities.

The authors collected three types of supervised data — generation trajectories from GPT 4.5 building games, successful planning trajectories, and improvement trajectories produced by running RSIGame. They retained only content that was executable and interactively verified not to break existing functionality, ultimately obtaining 2,213 generation trajectories, 2,108 planning trajectories, and 2,013 rounds of verified improvement data from 4,003 candidates, used for supervised fine-tuning of Qwen3-8B/27B.

This step expands self-improvement from "in-context optimization" to "parametric optimization," enabling stronger initial generation, reducing the number of refinement iterations needed to reach high quality, and opening the door to continuous RSI — where newly accumulated experience can be repeatedly internalized.

Supervised Fine-Tuning (SFT) is the standard method for adapting pretrained large models to specific tasks: collect high-quality input-output sample pairs and directly optimize the model's prediction probability on those samples using cross-entropy loss. Compared to reinforcement learning, SFT requires no reward function design, is training-stable, and has relatively manageable computational costs. In this paper's context, "experience internalization" means treating the successful trajectories accumulated during RSIGame runs as SFT data, encoding into model weights the knowledge of "how to make a good game" — not just relying on in-context prompting at inference time, but consolidating that knowledge into the parameters themselves. Qwen3-8B/27B is Alibaba's family of open-source language models at multiple scales; the authors chose them as the fine-tuning base because open-source models permit parameter updates, whereas commercial APIs (like the GPT series) generally do not expose fine-tuning interfaces.

Experimental Results: Consistent Gains Across Engines and Generators

Experiments were conducted on GameCraft Bench, a benchmark comprising 140 game development tasks spanning 15 game series, implemented on both the Godot and Phaser engines. Evaluation replays game demonstration trajectories and scores them across four dimensions — mechanics, depth, visuals, and art — according to hidden rubrics.

On the Godot engine, RSIGame improved overall scores by 10.7 to 20.3 points over frozen initial games across all five generator configurations, covering all dimensions. Under the same compute budget, it outperformed the comparison method Play2Code by 7.2 to 13.8 points — the authors emphasize that the gains come from "how development is organized," not from iteration per se.

The gains from experience internalization are particularly striking: fine-tuning lifted Qwen3-8B/27B's one-shot score from 37.07 to 48.22, closing the gap with Codex+GPT 4.5 to within 2 points, while reducing generation tokens by 11× (from 6.41M to 570K). Adding RSIGame on top further raised the score to 61.38, surpassing Codex+GPT 4.5's one-shot generation score of 50.26, with total token usage reduced by 2.6×. On the Phaser engine, overall scores across three generators also improved by 8.8 to 13.4 points over the baseline, validating the method's cross-engine robustness.

GameCraft Bench is an evaluation benchmark designed specifically for automated game development, requiring agents to generate executable projects in real game engines based on natural language descriptions. Godot is an open-source, cross-platform game engine centered on GDScript scripting and a scene-tree architecture, widely used for 2D/3D indie game development; Phaser is a 2D web game framework based on JavaScript/TypeScript, running in the browser. The two engines differ significantly in language paradigm, asset organization, and runtime environment, so validating on both effectively demonstrates generalizability rather than engine-specific adaptation. Evaluation uses replayed demonstration trajectories rather than live interaction to ensure reproducibility and fair comparison — fixed input trajectories eliminate variance introduced by random player behavior.

Key Findings: Why It Can Keep Improving

Further analysis reveals several valuable insights. First, simply scaling up the budget does not guarantee better games — Play2Code quickly plateaus or even regresses, while RSIGame consistently converts more development rounds into higher quality. The global quality monitor prevents later edits from erasing earlier gains by preserving the best state, and the saturation-aware stopping mechanism achieves comparable quality in fewer rounds.

Second, adaptive evolution outperforms fixed scheduling. Dynamically reallocating effort based on current game state (shifting to visuals after build failures, adding art rounds when visuals lag) was preferred 59% of the time in pairwise evaluations, versus only 37% for fixed round-robin scheduling.

Third, agent-based validation is critical. Evidence-driven pre-improvement validation raised precision from 58.6% to 72.3%, and unsupported objectives per round dropped from 1.93 to 0.50; replay-based validation detected 76.2% of unsuccessful modifications, while build-check-only approaches detected none of these behavioral failures. Furthermore, combining planning and verified improvement experience transfer yielded gains of 14.9 and 14.1 points on the mechanics and depth dimensions respectively, suggesting these trajectories provide reusable knowledge that goes beyond simply imitating successful games.

Beyond Games: A Transferable Generation Paradigm

RSIGame addresses the core problem that has plagued prior recursive self-improvement approaches in game development: overfitting to a small number of tests while falling short on overall quality. Its value lies not in introducing iteration itself, but in reliably sustaining iteration over the long term — pairing local evidence-driven improvement with global best-checkpoint tracking and saturation detection, then using experience internalization to form a positive feedback loop.

This "iterative development + global control + experience internalization" approach balances inference-time iterative optimization with training-time capability improvement: it solves the problem of insufficient single-pass generation quality while avoiding the inefficiency and regression that come from blind iteration. For complex software development, industrial design, and other generation scenarios requiring high reliability and quality, RSIGame offers a highly instructive paradigm — one that stops pursuing perfect one-shot results and instead progressively approaches high quality through exploration, diagnosis, and improvement.