Training a Mario AI with NEAT Neuroevolution: An Evolutionary Experiment from Three Failures to Completion

Training a Mario AI with NEAT Neuroevolution: An Evolutionary Experiment from Three Failures to Completion

Teaching a program to play Super Mario Bros. is nothing new in the world of AI gaming — but using a neuroevolution algorithm (NEAT) to tackle the higher-difficulty challenges of ROM Hack levels is still largely uncharted territory. A Reddit developer shared his ongoing project: evolving a Mario controller capable of clearing the gap near the end of Bowser's Crown 4-1 — a spot that had repeatedly caused failures — and ultimately completing the level.

How NEAT Teaches Mario to Play Itself

NEAT (NeuroEvolution of Augmenting Topologies) is a method that uses evolutionary algorithms to simultaneously optimize both the structure and weights of neural networks. Unlike traditional reinforcement learning, which relies on gradient descent, NEAT simulates natural selection: generate a population of networks, evaluate their performance, select the best performers, apply mutations, and repeat — generation after generation.

In this project, the neural network reads nearby game state information (such as enemies, terrain, and Mario's position) directly from emulator memory, then outputs the corresponding button inputs. Each "controller" is essentially an evolved network. The developer made clear that the replays shown in the video are not the training process itself — the network does not learn during playback. What's shown is a finished, evolved controller running through the level.

This project doesn't start from scratch — it builds on prior work. It continues in the tradition of SethBling's famous MarI/O project and draws from the work of Akisame and Electra. The video shows replays from three earlier-generation genomes that failed, followed by the controller that ultimately cleared the level. These are not consecutive training attempts — they were selected from different stages for comparison purposes.

NEAT was introduced by Kenneth Stanley and Risto Miikkulainen in 2002. Its core innovation is a speciation mechanism designed to protect innovative structures in their early stages. A newly mutated network topology often performs poorly at first, but competing directly against mature networks would cause it to be eliminated immediately. Speciation groups structurally similar networks together so they compete within their own group, giving novel architectures room to survive. NEAT also uses historical markings to track gene homology, enabling meaningful crossover between networks with different topologies — rather than randomly splicing unrelated connections. This gives NEAT stronger structural exploration capabilities compared to fixed-topology evolutionary strategies, at the cost of greater implementation complexity and sensitivity to hyperparameter tuning (such as speciation thresholds and mutation rates).

The Real Challenges of Tackling ROM Hack Levels

The developer's core interest lies in pushing the NEAT approach against harder SMB1 ROM Hack levels. Compared to the original Mario, ROM Hacks typically feature more devious terrain design and more demanding execution requirements. The gap near the end of the level (around the staircase) repeatedly "stopped" various controllers cold, becoming a genuine obstacle in the evolutionary process.

The real challenge isn't simply moving forward — it comes in two forms: backtracking (where the level requires the player to move backward before advancing), and non-linear, complex layouts. These scenarios are deeply hostile to networks evolved purely on a "rush right" reward signal, because they break the intuitive logic of linear progression.

SMB1 ROM Hacks are custom levels created by modifying the original Super Mario Bros. ROM file, typically using tools like LunarMagic. Bowser's Crown is one of the community's high-difficulty hacks, deliberately featuring precise jumps, tight corridors, and counter-intuitive routes that far exceed the original game's margin for error. For NEAT-based approaches, the challenge lies in reward signal design: if the reward function is simply tied to "distance moved to the right," the network has no incentive to move left at all — even when moving left is a prerequisite for clearing a particular obstacle. Solving backtracking problems typically requires more sophisticated reward shaping or the introduction of temporal memory mechanisms, and the feedforward or simple recurrent networks used in standard NEAT already have inherent limitations when handling long-term temporal dependencies.

An Honest Technical Demo, Not a Universal Proof

To his credit, the developer maintains a notably restrained characterization of what the results actually show. He explicitly states that each new level is trained separately, that this successful run does not demonstrate the network's ability to generalize to unfamiliar levels, and that no quantifiable success rate data is provided.

In other words, this is a controller that "can clear one specific level" — not a general-purpose agent that has "learned to play Mario." This distinction is especially important in AI gaming projects. Many demos create the impression that an AI has mastered a general skill, when in reality the result is often overfitting to a single scenario.

Additionally, the project's implementation is currently private, making this more of a gameplay demonstration with high-level methodology discussion than a reproducible open-source release. The developer also openly acknowledges that his ongoing development process "relied heavily on AI tools," and that the reactions and captions in the video are purely for humor — the in-game completion itself is real.

The Value of Neuroevolution in Game AI

In an era where deep reinforcement learning dominates the field, evolutionary algorithms like NEAT still hold a unique place. They don't require carefully designed gradients or differentiable loss functions, making them naturally suited for game environments with discrete action spaces and sparse rewards. They also automatically evolve network topology, eliminating the need to manually design architectures.

The significance of this project may not lie in "yet another AI completing Mario," but in its honest illustration of where evolutionary methods hit their limits when facing genuinely increased difficulty: single-level training is feasible, generalization remains an open problem, and backtracking with non-linear layouts is still stubbornly hard. For developers looking to get started with neuroevolution or game AI, this kind of transparent, practical sharing is more valuable than sensationalized "AI conquers game" headlines.

The core differences between NEAT and deep reinforcement learning (such as PPO or DQN) manifest across several dimensions: deep RL relies on backpropagation and differentiable loss functions, requiring large amounts of samples but learning efficiently from continuous reward signals; NEAT explores the solution space through parallel population search, making it naturally robust to sparse or non-differentiable rewards, though the computational cost of maintaining a population and evaluating it generation by generation is similarly non-trivial. A 2017 OpenAI study showed that on certain Atari games, evolutionary strategies (ES) can achieve performance comparable to deep RL, with a training speed advantage due to large-scale parallelism. Part of NEAT's long-running popularity in the game AI community also stems from SethBling's MarI/O video bringing it to mainstream attention — lowering the barrier to entry for beginners and fostering a rich ecosystem of community implementations and tutorials.