Pokémon Red
A turn-based RPG released by Nintendo in 1996 for the Game Boy platform and one of the first entries in the Pokémon series. It features 8 gyms and 151 species of Pokémon. Its limited state space and deep decision tree have led to its use as a benchmark for evaluating AI capabilities.
Timeline (last 90 days)
PokéBench maintains a leaderboard of agents playing "Pokémon Red", with turn logs and the cost of each run.
In the same game at the same token price, Opus 5.5 completed it in 271 turns, costing $8.44.
On the day of its release, Muse Wonder livestreamed playing "Pokémon Red"; after entering Viridian Forest at turn 208, it spent the next 792 turns entirely inside it, the six-hour livestream cost $96.06, and it never beat the first gym leader, Brock.
Sonnet 5.5 became the first Sonnet model to complete Pokémon Red solely by reading and operating through screenshots.
The difficulty of Pokémon Red for AI agents lies not in reaction speed but in long-term planning and state memory.
Developer Christian Mat created an AI system called Jev and attempted to have it complete Pokémon Red.
The developer candidly admitted that it is currently uncertain whether Jev will be able to obtain all badges in Pokémon Red.
A Reddit user had an AI model named Jev play Pokémon Red in real time, with the harness required to run the game being built simultaneously by Opus 5 during the process.
The advantage of Pokémon Red as an AI benchmark vehicle lies in its limited state space but extremely deep decision tree, where completing the game requires hundreds of effective actions.
Pokémon Red was released in 1996 and is a turn-based RPG for the Game Boy, featuring 8 Gyms, 151 catchable species, and a linear main storyline.
2 more timeline events
All Facts (9)
Sonnet 5.5 became the first Sonnet model to complete Pokémon Red solely by reading and operating through screenshots.
60%UnverifiedPokéBench maintains a leaderboard of agents playing "Pokémon Red", with turn logs and the cost of each run.
50%UnverifiedIn the same game at the same token price, Opus 5.5 completed it in 271 turns, costing $8.44.
50%UnverifiedThe difficulty of Pokémon Red for AI agents lies not in reaction speed but in long-term planning and state memory.
50%UnverifiedThe advantage of Pokémon Red as an AI benchmark vehicle lies in its limited state space but extremely deep decision tree, where completing the game requires hundreds of effective actions.
50%UnverifiedPokémon Red was released in 1996 and is a turn-based RPG for the Game Boy, featuring 8 Gyms, 151 catchable species, and a linear main storyline.
50%UnverifiedOn the day of its release, Muse Wonder livestreamed playing "Pokémon Red"; after entering Viridian Forest at turn 208, it spent the next 792 turns entirely inside it, the six-hour livestream cost $96.06, and it never beat the first gym leader, Brock.
50%UnverifiedDeveloper Christian Mat created an AI system called Jev and attempted to have it complete Pokémon Red.
50%UnverifiedThere have been projects in the community that used reinforcement learning to complete Pokémon Red, sparking widespread discussion
50%