Adapt-1 Without PPO: Learning to Clear Super Mario Bros 1-1 in 15 Minutes

An Approach That Diverges from Mainstream Reinforcement Learning

In deep reinforcement learning, PPO (Proximal Policy Optimization) is almost the default choice for training game agents — from OpenAI's Dota project to various Atari benchmarks, it's inescapable. Rei Labs' Adapt-1 takes a different path: it's neither LLM-based (non-LLM) nor does it use PPO. The developers shared a straightforward validation scenario on Reddit: having Adapt-1 learn from scratch and clear the classic World 1-1 of Super Mario Bros.

Mario 1-1 is a classic proving ground in game AI research. Deceptively simple, it actually demands continuous decision-making across jump timing, enemy avoidance, pipe navigation, and pit detection — making it a reliable benchmark for whether a learning system can form effective strategies under limited feedback.

PPO (Proximal Policy Optimization), introduced by OpenAI in 2017, limits update magnitude by clipping the objective function during each policy update, striking a balance between learning efficiency and training stability. Compared to earlier policy gradient methods, PPO is simpler to implement, requires fewer hyperparameters, and scales well in parallel environments — which has made it the de facto standard for game AI training. Its main limitation is relatively low sample efficiency: it typically requires millions to billions of game frames to reach human-level performance, and it's sensitive to hyperparameter choices and environment details. By abandoning this route, Adapt-1 needs an alternative mechanism to handle the exploration-exploitation tradeoff — which is the context for understanding the design rationale behind its two operating modes.

Two Modes: Reactive Policy and Sequence Search

According to the developers, Adapt-1 demonstrated two distinct approaches in this test. They differ in both methodology and time cost, so it's worth examining each separately.

Reactive Policy

The first approach involves learning a reactive policy through self-play. The system starts untrained, accumulates experience over roughly 36 minutes of actual gameplay, forms a policy, and then disables learning — using only the acquired policy to clear the level. This separation of "learn first, then act" means the final demonstration is driven entirely by previously learned behavior, not by real-time adjustment.

Machina Sequence Engine

The second approach uses Adapt-1's sequence engine, Machina. Also starting from scratch, it searches for a sequence of button inputs capable of reaching the flagpole. According to the developer's data, this took 403 attempts and approximately 11 wall-clock minutes to find a working sequence.

The "15 minutes" mentioned in the headline roughly corresponds to this sequence search pathway — a notably lower time cost compared to traditional RL approaches that often require hours or more of training. That said, it's worth clarifying: the sequence found is a fixed solution tailored to a specific level, which is fundamentally different in nature from the reactive policy, which aims to generalize across varying situations.

The fundamental difference between sequence search and reinforcement learning lies in how the objective is framed. RL attempts to learn a general state→action mapping (a policy) that makes reasonable decisions across any encountered state. Sequence search instead directly seeks a complete sequence of actions that leads from the initial state to the goal — closer in spirit to path planning or Monte Carlo Tree Search (MCTS). In deterministic, fixed-layout levels, the latter is often more efficient: finding one viable path counts as success, with no need to model generalization. Completing 403 attempts in 11 minutes suggests Machina evaluates each attempt extremely efficiently — possibly relying on fast simulation or early termination rather than running full game episodes.

Putting the Numbers in Perspective

From a technically rigorous standpoint, a few things about these results deserve honest consideration.

Mario 1-1 is a relatively deterministic environment — fixed layout, limited randomness — which makes sequence search (finding a fixed sequence of inputs to clear the level) not just feasible but highly efficient. Completing 403 attempts in 11 minutes indicates that each attempt is executed and evaluated very quickly, but this is closer to efficient search optimization than to "learned generalization" in the traditional sense.

The reactive policy pathway (36 minutes of self-learning followed by policy-only execution) is more aligned with the spirit of "learning," since it produces a reusable decision-making strategy. However, a single-level, single-demonstration result isn't sufficient to evaluate how that strategy would perform on other levels or in more complex environments.

In other words, this is a convincing proof of concept — not a comprehensive benchmark of Adapt-1's general capabilities. Its true value can only be assessed after testing across more levels, more games, and eventually non-game tasks.

Commendable Transparency

One clear strength of this project is its openness. The developers have published the code, full experimental data, run traces, demo footage, and a step-by-step guide including cost information — all hosted on GitHub at hsrvc/adapt1-mario.

For an AI system claiming to take a non-mainstream route, reproducibility is especially critical. Publishing complete data and traces means other researchers can independently verify specific claims — like 403 attempts and an 11-minute clear — rather than relying solely on a demo video. In an era where many AI announcements are impressive but impossible to reproduce, this level of openness stands out.

Takeaway

Adapt-1's performance on Mario 1-1 offers a concrete window into a non-PPO, non-LLM learning system. It cleared the level through two distinct approaches — reactive policy and the Machina sequence engine — within reasonable time frames, and made all materials publicly available for verification.

The significance of this kind of exploration isn't in how fast a single level gets cleared, but in the reminder that beyond PPO and large language models, other architectures are worth taking seriously. Whether Adapt-1 can move from a polished Mario demo to broader real-world tasks is a question the community will need to answer through further experimentation. Interested readers can head directly to the GitHub repository to reproduce and evaluate these results.