AI Builds JAX Environment in 2.5 Days, Achieves 85% Win Rate for Getting Over It Agent

From Inspiration to Reality: A Breakthrough in AI-Built Game Environments

In the field of AI agent training, a long-standing challenge is: how can skills learned by agents in simulated environments be effectively transferred to real games? A recent experiment by a Reddit user provides a remarkable answer: with the help of Opus 5.5, they successfully built a JAX-based Getting Over It game environment, and the trained agent achieved a win rate of about 85% in the real game.

This experiment was inspired by a post from u/Gold-Deal8281. The author decided to test whether Opus 5.5 could generate a JAX environment sufficiently aligned with the real game. The entire pipeline took about 2.5 days, consumed about 13 GPU hours on a 3090 GPU, reached 1.32B training steps, and used 1B tokens. Ultimately, the agent's performance in the real game was impressive.

JAX is an open-source high-performance numerical computing library developed by Google, designed for machine learning and scientific computing. It combines automatic differentiation (Autograd) with the XLA (Accelerated Linear Algebra) compiler, enabling just-in-time compilation of Python functions into efficient GPU/TPU kernels. It also supports function transformations such as jit, vmap, and pmap, making large-scale parallelism and vectorization convenient. In reinforcement learning settings, JAX is often used to write batched physics simulators—the entire environment state is represented as arrays, and thousands of parallel copies can advance synchronously in a single compiled call, pushing simulation throughput to tens of thousands of steps per second. This is precisely the technical foundation that allows the environment in this article to reach 1600x real-time speed.

Technical Details: How Opus 5.5 Accelerated Environment Construction

The author had previously applied similar methods to several games, including Brotato, Cuphead, and Kingdomino, with good results. But Getting Over It has a more complex physics engine and interaction logic, and traditionally required substantial manual reverse engineering to build a high-fidelity simulated environment.

This time, the author handed the "dirty work" to Opus 5.5. He explicitly required the environment to have sufficient parity so that the trained agent could perform well in the real game. Opus 5.5 delivered: it increased the simulation environment's speed from initially about 6k steps/s (about 200x real-time) to about 50k steps/s (1600x real-time). This improvement greatly increased training efficiency.

Parallel Training and Policy Distillation

To further accelerate training, the author used 10 parallel game copies. Building on the environment constructed by Opus 5.5, training speed reached 1600x real-time, providing a solid foundation for subsequent reinforcement learning training. After about 11 hours of training, the author distilled multiple specialist policies and ultimately obtained a high-performing winning policy.

Parity in reinforcement learning environment construction refers to the degree to which the simulator and the real target system agree on state transitions, physics rules, reward signals, and observation representations. If parity is insufficient, the agent may exploit simulator biases to "cheat" (i.e., reward hacking), learning policies that are ineffective or even harmful in the real environment, causing the so-called sim-to-real gap. Measuring parity usually requires feeding the same sequence of actions into both the simulator and the real game, then comparing trajectories, collision outcomes, and score curves. The requirement in this article that the LLM-generated environment have sufficient parity is precisely to ensure that the distilled policy still has about an 85% win rate in the real Getting Over It, rather than performing well only in a high-fidelity simulator.

Policy distillation is a technique for compressing the knowledge of multiple "expert" policies into a single student policy. The usual approach is to have each expert produce actions or action probabilities over the state distributions it excels at, and have the student network imitate them by minimizing KL divergence (or cross-entropy) with the experts' outputs, thereby obtaining the combined capabilities of multiple experts without significantly increasing inference cost. In this article, multiple specialist policies are first trained for different levels or different starting states, and then distilled into a unified winning policy. This improves robustness and also makes it easier to deploy a single model in the real game. Combined with parallel environments, distillation can also greatly reduce the total number of samples needed to reach the target win rate.

Results and Implications: Lessons Behind the 85% Win Rate

The agent's win rate in the real game is about 85%, a result that validates the high parity between the simulated environment and the real game. For AI agent training, this means we can use large language models (such as Opus 5.5) to rapidly build high-quality simulated environments, greatly reducing training cost and time.

The core advantage of this method is that it does not require manual reverse engineering of the game engine; instead, it relies on an LLM to generate a JAX environment with sufficiently high fidelity. This opens a new path for agent training in more complex games in the future. The author says he has already successfully applied similar methods to multiple games and hopes this approach can be extended to a broader range of domains.

Reverse engineering a game engine means inferring and reproducing its physics and interaction logic without source code by observing game behavior, memory data, or disassembly, in order to build a simulator usable for training. The traditional process requires manually annotating collision bodies, fitting gravity and friction parameters, and reconstructing input latency, which is time-consuming and error-prone—especially difficult for games like Getting Over It that rely on custom physics (such as articulated constraints between the character and the hammer). LLM-driven environment generation turns this process into a loop of "describe requirements—generate code—verify parity," freeing human effort from manual reverse engineering. This is also a key reason the method in this article can scale to multiple games.

Future Outlook: LLM-Driven Environment Generation

This experiment demonstrates the enormous potential of LLMs in code generation and environment construction. As model capabilities improve, we may see more training environments automatically generated by AI in the future, accelerating agents' learning in various tasks. For the game AI and reinforcement learning communities, this is undoubtedly a direction worth watching.

Note: This article is based on experimental results shared by a Reddit user; specific implementation details and code were not provided in the original post.

One-line Summary

Using Opus 5.5 to generate a highly aligned JAX environment, the agent achieves an 85% win rate in the real Getting Over It.

Paragraph Summary

A Reddit user validated the feasibility of using the large language model Opus 5.5 to automatically build a high-fidelity game simulation environment: for Getting Over It, with its complex physics and interactions, Opus 5.5 generated a JAX-based environment and increased simulation speed from about 6k steps/s to about 50k steps/s (1600x real-time). With 10 parallel copies, about 11 hours of reinforcement learning, and multi-expert policy distillation, the entire pipeline took about 2.5 days, 13 GPU hours, 1.32B training steps, and 1B tokens, and the final agent achieved about an 85% win rate in the real game. The author had previously applied similar methods to Brotato, Cuphead, and Kingdomino, showing that LLM-driven environment generation can significantly lower the barrier and cost of agent training.