ClashRoyaleAi
An open-source reinforcement learning simulator for Clash Royale that rewrites the game's battle engine in C++17 and exposes a Python interface via pybind11. The simulator conforms to the Gymnasium API standard and supports training of PPO agents as well as self-play league mechanisms.
Timeline (last 90 days)
A developer created a mini Clash Royale AI agent lab page that can be trained directly in the browser.
The simulator engine used in the browser lab is compiled from C++ to WebAssembly and is the same engine used to train the full agent.
The mini agent uses the REINFORCE algorithm with a per-spawn baseline and has a total of 5,629 weights.
The entire learner consists of approximately 350 lines of pure JavaScript, with every gradient hand-implemented and no dependency on any machine learning library.
The full Clash Royale AI employs recurrent PPO (recurrent Proximal Policy Optimization).
In the Giant vs. Cannon matchup, placing the Cannon directly in the attack path yields only about 75% of the optimal score; the optimal solution is to place it in the center to draw the Giant into the attack range of two towers.
When using a constant entropy reward, the network got stuck in a suboptimal solution in 5 out of 6 runs; after annealing the entropy coefficient from 0.1 to 0.005 over the first 10,000 attempts, the number of times it got stuck dropped to only 1 out of 6.
The Chariot vs. Valkyrie matchup was intentionally locked; every configuration the author tried yielded only 55% of the optimal score, and the problem remains unsolved and open to the community.
The full agent plays entire matches, manages 4 hand cards and elixir resources, uses recurrent PPO, and is trained on CPU for several days.
The core loop of the mini version and the full version is identical: act → let the engine score → update.
14 more timeline events
All Facts (20)
The code for both the lab and the full project is open-sourced and hosted in the GitHub repository itzik123/ClashRoyaleAi.
50%UnverifiedThe simulator engine used in the browser lab is compiled from C++ to WebAssembly and is the same engine used to train the full agent.
50%UnverifiedThe mini agent uses the REINFORCE algorithm with a per-spawn baseline and has a total of 5,629 weights.
50%UnverifiedThe entire learner consists of approximately 350 lines of pure JavaScript, with every gradient hand-implemented and no dependency on any machine learning library.
50%UnverifiedThe full Clash Royale AI employs recurrent PPO (recurrent Proximal Policy Optimization).
50%UnverifiedIn the Giant vs. Cannon matchup, placing the Cannon directly in the attack path yields only about 75% of the optimal score; the optimal solution is to place it in the center to draw the Giant into the attack range of two towers.
50%UnverifiedWhen using a constant entropy reward, the network got stuck in a suboptimal solution in 5 out of 6 runs; after annealing the entropy coefficient from 0.1 to 0.005 over the first 10,000 attempts, the number of times it got stuck dropped to only 1 out of 6.
50%UnverifiedThe Chariot vs. Valkyrie matchup was intentionally locked; every configuration the author tried yielded only 55% of the optimal score, and the problem remains unsolved and open to the community.
50%UnverifiedThe full agent plays entire matches, manages 4 hand cards and elixir resources, uses recurrent PPO, and is trained on CPU for several days.
50%UnverifiedThe core loop of the mini version and the full version is identical: act → let the engine score → update.
50%UnverifiedThe project's docs/DECISIONS.md document records pitfalls encountered during development, including reward hacking, biased self-play null hypotheses, and discount horizons shorter than match duration.
50%UnverifiedThe ClashRoyaleAi simulator exposes its C++ engine to Python via pybind11.
50%UnverifiedThe project code has been open-sourced on GitHub (github.com/itzik123/ClashRoyaleAi); most of the card logic was built with the help of Ambash, and AI coding tools were used as a pair programming assistant.
50%UnverifiedThe simulator covers all cards in the game as of July 2026, including 132 regular cards and 41 evolution cards.
50%UnverifiedThe simulator guarantees determinism and supports setting a random seed.
50%UnverifiedRunning a complete match on a single notebook CPU core takes only about 10 milliseconds, and the snapshot() interface can fork the game state within 7–20 microseconds.
50%UnverifiedThe environment follows the Gymnasium API standard; the observation space is a 13,977-dimensional float vector and incorporates the historical record of cards played by the opponent.
50%UnverifiedThe action space requires the agent to first select a card and then choose a placement position from 612 grid cells, with a legality mask to constrain illegal actions.
50%UnverifiedThe project's agent uses a recurrent PPO scheme with a CNN + LSTM network architecture of approximately 1.85 million parameters, running on CPU only.
50%UnverifiedTraining strategies include curriculum learning (teacher opponents mastering 16 mainstream ladder decks) and PFSP prioritized fictitious self-play league.
50%