League of Legends AI 1v1: How an Ezreal Reinforcement Learning Agent Beat a Previous Version

A researcher trained two non-scripted AIs with reinforcement learning and had them play an Ezreal vs. Ezreal mid-lane 1v1 inside the League of Legends client. The blue side is the main agent (trained for 45,600 updates), and the red side is an "exploiter" trained specifically to beat an older version of the blue agent. The details of this match reveal a viable path for training game AI today.

Match Setup: No Healing, the First Trade Decides It

Both sides are level 6 Ezreal with fixed abilities Q3/W1/E1/R1, running Flash and Barrier. The rules have no passives, no items, no leveling, and no potions, recalls, or lifesteal — once you lose health, you never get it back. This means the first trade almost directly decides the match, and that is exactly what happens in the video.

There are three win conditions: first blood, first to 100 CS, or first to destroy the outer turret, with a 15-minute cap. Minion waves and outer turrets are present, but according to the author, under these rules reaching 100 CS is far more worthwhile than grinding down a 9,000-HP outer turret.

The Two Agents: Main Training and the "Exploiter"

The blue side is the main agent, trained to update 45,600, the policy the author actually wants to make stronger. The red side is the "exploiter," trained for only about 2 hours (half of a 4-hour cycle), whose sole job is to beat a frozen copy of the main agent at step 43,800. It starts from an earlier checkpoint, "exploit 2600," and only plays against that 43,800 snapshot and its own historical versions.

During training it wins about 85% against that snapshot. Early on it wins by fast all-ins, later shifting to CS denial — in its wins, the CS count is 67 to 9. In the video, the red side crushes the blue side exactly this way.

Training Environment: From Replays to a Numba Simulator

The game rules come from real match replays, but for training they were ported into a compiled Numba simulator with orders of magnitude higher throughput. The agent makes a decision every 4 server ticks, about 133 ms; each side observes the game from its own perspective, and the reward is zero-sum.

The algorithm uses PPO with an LSTM policy; the input covers units such as heroes, minions, and turrets, with about 1.5M to 1.65M parameters. The action space includes move/stop, basic attack, QWER, Flash, and Barrier, plus a pointer head for target selection, with illegal actions masked out. Each update runs one PPO epoch, with Adam learning rate 3e-4, discount factor γ=0.9997 (about a 7-minute horizon, matching OpenAI Five), and a win reward of ±10.

The compute is a single Lambda GH200 (64 ARM cores plus an H100 96GB), running 15,360 games in parallel, with throughput of 1.2 to 1.9 million samples/sec depending on whether an exploiter shares the GPU. Each update is about 5.5 million samples, totaling about 250 billion samples by update 45,600; the author mentions that the agent itself reported about 200 billion at step 36,300.

PPO and LSTM Policy

PPO (Proximal Policy Optimization) is one of the most commonly used reinforcement learning algorithms in game AI today and belongs to the policy gradient family. Its core idea is to limit how far the new policy can deviate from the old one on each update (by clipping the importance sampling ratio), thereby preventing training collapse. The article mentions "each update runs one PPO epoch," meaning only one round of gradient updates per batch of data, a common practice for PPO in large-scale distributed training.

LSTM (Long Short-Term Memory) is used to handle sequential decision-making. In League of Legends, the agent needs to remember information such as the minion wave state a few seconds ago and the opponent's ability usage habits; the LSTM hidden state gives the policy network short-term memory. PPO handles "how to update," LSTM handles "how to remember," and together they form the algorithmic backbone of the agent described in the article. OpenAI Five also used a PPO+LSTM combination, which is why this approach has been repeatedly validated in complex multi-agent environments.

Training Lineage: From Mirror Self-Play to an AlphaStar-Style League

The training path has several stages. The earliest was mirror self-play and laning training with minion waves, producing the first checkpoints around October 10. On October 19, "racing" rules were added — the 100 CS win condition, win reward, and long horizon — so the agent could from then on see both sides' CS counts.

On the same day, turret training started from the "exploit 2600" checkpoint. The newly added turret input was wired in with zero-initialized weights (OpenAI Five's "surgery" trick) to guarantee that the old checkpoint's behavior is completely unchanged at the start. During training, some turrets start with random health. From October 19 to 23, this part of training ran in 4- to 12-hour segments, against fixed opponents plus the main agent's own past versions.

Starting October 23, training shifted to a league structure loosely modeled on AlphaStar: in each 4-hour cycle, 6 of the main agent's 8 groups play against fixed opponents (older exploiters plus early main checkpoints), and the other 2 play against their own past; meanwhile a new exploiter trains against the latest main agent and becomes one of the fixed opponents at the end of the cycle.

AlphaStar League and the "Exploiter" Mechanism

AlphaStar is DeepMind's AI for StarCraft II, and one of its biggest innovations is the "league" training framework. Simply put, a league maintains an opponent pool of different agent versions; the main agent not only self-plays against the current version but also plays against historical versions and "exploiters" trained specifically against it. The exploiter's sole goal is to find and exploit the main agent's weaknesses, forcing the main agent to evolve more robust strategies.

The "exploiter" in the article is a simplified implementation of this idea: the red side trains for only about 2 hours, specifically against the frozen copy of the blue side at step 43,800, and already reaches 85% win rate. This asymmetric matchup makes training far more efficient than pure self-play — the main agent is forced to face a specifically countering opponent rather than grinding through mirror matches. The league structure also includes two roles, "fixed opponents" and "historical versions": the former provides a stable baseline, the latter prevents the agent from forgetting old strategies.

Zero-Initialized Weight Wiring and the "Surgery" Trick

When training enters a new stage (such as adding turret input), directly modifying the network structure would break the old checkpoint's behavior and cause training to collapse. OpenAI Five's "surgery" trick solves this: the weights corresponding to the new input are initialized to zero, so at the start of training the new input has no effect on the network's output and the old policy's behavior is fully preserved. As training progresses, gradients gradually make these weights nonzero, and the agent learns to use the new information.

The article mentions that "the newly added turret input was wired in with zero-initialized weights," which is exactly this trick. It lets the author smoothly start turret training from the "exploit 2600" checkpoint without retraining from scratch. This incremental expansion capability is an important part of engineering large-scale reinforcement learning, and it also lets the stages of the training lineage connect seamlessly.

Current Level and the Ultimate Goal

At recording time, the main agent's win rate against fixed opponents is about 81% to 83%, plateauing from around step 39,000; it leads heavily in CS at 48 to 17; it almost never attacks full-health turrets. For reference, its win rate against the scripted DuelBot is 98% to 99%, while a behavior-cloned agent trained on ranked replays wins only 0% to 6% against these checkpoints.

The author's ultimate goal is clear: build an AI strong enough to beat T1/Faker (or the then-world champion) in a BO5. Judging from this Ezreal 1v1, there is still an obvious way to go before that goal, but league-style training plus the exploiter mechanism has already let a non-scripted agent show interpretable strategy evolution under restricted rules.

One-line Summary

A non-scripted RL agent demonstrates strategy evolution through league training and an exploiter mechanism in a League of Legends Ezreal 1v1.