GPT-6 Astra Plays Anno 117 Autonomously: A 1,000-Resident City Built in 6 Hours with Zero Instructions

A Reddit user ran a fascinating experiment: they let GPT-6 Astra play the city-building strategy game Anno 117: Pax Romana entirely from scratch — no plugins, no MCP, no game guides — relying only on screenshots to perceive the game state and mouse controls to interact with it. After roughly 6 hours of autonomous play, the AI had built a thriving city with 1,000+ residents, spanning 4 islands, with established trade routes and most buildings unlocked.

What makes this case worth examining isn't the game itself, but what it reveals about a general-purpose AI agent's ability to plan, execute, and self-iterate in an open-ended environment.

Experimental Setup: Pure Visual Control Under Minimal Constraints

The experimenter deliberately stripped away every convenience that could have given the AI an edge. By their account, the entire run met the following conditions:

• Screen perception via screenshots only: The AI captured game frames through the ChatGPT desktop app to understand the current state — no reading of internal game data.
• Mouse control only: All interactions were simulated mouse clicks, identical to how a human player would operate.
• No plugins, MCP, or game documentation: The AI received no prior knowledge of the game and had no access to any specialized tool interfaces.
• Minimal instructions: The experimenter simply asked it to "keep the economy healthy" and "improve its own gameplay skills as it progresses."

The key insight of this setup is that it closely approximates the challenges a real-world agent faces — completing a complex task through visual observation and general-purpose controls alone, without complete information or dedicated interfaces.

The screenshot capability of the ChatGPT desktop app is the technical foundation of this experiment. The app allows the model to periodically capture screen content with user authorization, passing pixel-level images as visual input to the multimodal model. The model then outputs the next action command, which is translated into actual mouse coordinate clicks. This "perceive → reason → act" loop forms the most basic agentic execution framework — the Agentic Loop. Compared to traditional AI gaming approaches that directly call game APIs or read memory data, a pure-vision approach loses significantly more information: the model must infer the semantics of UI elements, the meaning of numbers, and the causal relationships between buildings from raw pixels. But precisely because of this, the methodology transfers far more readily to real-world software environments.

Why Anno 117 Is a High-Difficulty Test

The Anno series is a classic city-building and economic management simulation, and its core challenge lies in supply chain balancing. Players must manage multi-tiered supply chains — from raw material extraction through production processing to resident consumption — where any broken link leads to economic collapse or population loss.

The experimenter specifically noted that "this is a complex game to balance correctly with all supply chains." Establishing trade routes across 4 islands adds another layer: the AI must understand the logic of routing resources between different geographic nodes, not just stacking buildings on a single map.

Sustaining economic health and continuing to expand in a multi-variable environment that demands long-term planning suggests the AI isn't just doing local pattern matching — it's performing some form of global resource planning.

The Anno series enforces strict production ratio constraints in its supply chain design: each advanced good typically depends on multiple raw inputs, and each production building has a fixed output rate. Players must precisely calculate building ratios to avoid bottlenecks or shortages. Bread production, for example, requires a three-tier chain of wheat fields → mill → bakery, where the number of wheat fields must match the mill's processing rate. As a city expands across multiple islands, each island's terrain determines which crops can be grown, making trade routes the only means of cross-island resource allocation — poor route design creates regional shortages of specific goods. This combination of multi-tiered, geographically distributed constraints makes it an excellent high-pressure scenario for testing long-horizon planning.

What the Results Say About Agent Capabilities

The final outcome — 1,000+ residents, 4 islands, trade routes, and most buildings unlocked — is genuinely impressive for an autonomous run with no game-specific instructions. The experimenter themselves admitted to being "genuinely impressed."

One detail worth noting: the experimenter suggested the AI "could have gone further," but was cut off because they "ran out of their weekly token quota." This reveals two things: first, the AI's progress wasn't stalling due to capability limits — it was interrupted by an external resource constraint; second, long-horizon autonomous tasks consume a substantial amount of compute and tokens, meaning cost remains a real barrier to scaling such applications.

What This Experiment Means

Using games as AI capability test beds isn't new, but past AI game-playing typically relied on specialized training, direct game state access, or API calls. What's different here:

• General-purpose, not specialized: A general-purpose large language model was used, with no fine-tuning or tool adaptation for this specific game.
• Pure vision + general controls: The AI used the same information channels and control methods as a human player, making it far more generalizable.
• Self-iteration: The AI was asked to "improve its own skills," meaning it needed to summarize experience and adjust strategy over the course of play.

If this capability is reliable and reproducible, it has direct implications for GUI automation, software testing, digital assistants, and more — an agent that can read any screen, operate a mouse, and autonomously pursue complex goals over extended periods has an application boundary that extends far beyond gaming.

GUI automation is the most immediate transfer scenario for this kind of capability. Traditional RPA (Robotic Process Automation) tools depend on pre-recorded action scripts or fixed UI element coordinates, and break down the moment an interface changes — making maintenance costs extremely high. A new generation of GUI agents built on vision-language models can theoretically "read" any interface the way a human does, deciding actions based on semantic intent rather than pixel coordinates, resulting in far greater robustness and cross-application generalizability. The Anno 117 experiment effectively simulates an "extremely complex GUI environment": dense UI elements, continuously changing state, and delayed feedback on actions. If an AI can sustain autonomous progress in this scenario, it offers strong positive signal for more structured use cases like enterprise software operation and browser automation.

Reasons for Caution

It's worth noting that this is a single personal account shared on Reddit, without complete screen recordings, failure case logs, or a reproducible experimental protocol. Figures like "6 hours" and "1,000+ residents" cannot be independently verified, and the extent of trial-and-error or human intervention during the run remains unknown.

The appropriate stance toward demonstrations like this is to acknowledge the direction of potential they point to, while remaining cautious about the specific data and methodology. This is more of a thought-provoking capability snapshot than a rigorous benchmark result.