Kolibri-1 Plays Breakout in Real Time: Under 25ms Per Decision

When a Language Model Picks Up a Game Controller

Aleph Alpha's open-source model Kolibri-1 was recently put through an interesting experiment by a developer — playing the classic arcade game Breakout directly, with no fine-tuning whatsoever. The model relied entirely on its own capabilities to control the game, keeping inference latency per decision under 25 milliseconds.

The experiment's tagline says it all: "Less talk. More Breakout." The core question it set out to answer: can a language-based model, given minimal constraints, produce structured and executable actions — rather than generating a wall of text?

From Probabilities to Actions

The most technically interesting aspect of this experiment is the direct translation of the model's "action probabilities" into actual game inputs. The developer specifically optimized the inference pipeline around this action probability step, achieving a stable per-decision latency of roughly 25 milliseconds.

For a real-time game scenario, latency is a hard constraint. Breakout demands fast reflexes — the paddle has to track the ball's movement. At 25ms per step, the model can theoretically make around 40 decisions per second, which is fast enough to support a smooth, real-time gameplay experience.

The experiment's key design choices were deliberately lean:

• Four movement options: The model chooses from a limited action space
• No text generation: Output is an action, not natural language
• Shared game session: Continuous decision-making within a single environment
• Open weights: The model is open-source — anyone can reproduce this

"Action probabilities" here means the model outputs a probability distribution over available actions (e.g., move left, move right, stay, launch ball) rather than a text description. This is analogous to how a policy network in reinforcement learning works: given the current game frame or state as input, the model produces a probability vector over the action space, and the final action is selected via argmax or sampling. Since language models are architecturally built around next-token probability prediction, aligning the output head to a small set of discrete "action tokens" is technically straightforward — essentially compressing the vocabulary down to just a few tokens. This design bypasses the overhead of text generation entirely by reading probability values directly from the logits layer, and is the key reason 25ms latency is achievable.

Why This Is a Worthwhile Test

Having language models play games isn't new, but what makes this experiment meaningful is what it actually tests: structured output capability. Many large models excel at generating coherent text, but getting them to produce machine-executable instructions under strict constraints — while maintaining low latency — is a different challenge entirely.

Breakout is an ideal test environment for exactly this reason: simple rules, instant feedback, and a limited action space make it easy to surface a model's weaknesses in real-time decision-making. The full pipeline has to work — continuously observing game state, converting observations into sensible action probabilities, and rapidly executing the chosen action. Only when all of that runs end-to-end can you call the test a success.

The developer noted in the original post that the more they explore the Kolibri model, the more potential they see in it. At its core, this kind of experiment probes the boundaries of where a language model can operate beyond language tasks.

Breakout was released on the Atari 2600 in 1976 and has since become one of deep reinforcement learning's most iconic benchmarks. When DeepMind published the DQN (Deep Q-Network) paper in 2013, Atari games — including Breakout — were the central test suite, demonstrating that deep neural networks could learn superhuman game strategies directly from pixel input. Since then, Breakout has become virtually synonymous with "AI learning to play games." Unlike DQN, which requires millions of frames of training data, this experiment emphasizes zero-shot direct inference under minimal constraints — a fundamentally different technical path. The goal isn't to train a game-specific model; it's to see whether a general-purpose model works out of the box.

Open Source Meets Real-Time Inference

Kolibri-1's open weights release strategy allows the community to directly reproduce and extend experiments like this. Compared to closed-source models, open weights lower the barrier to entry — developers don't need to rely on an API and can run inference locally or in self-hosted environments, optimizing for specific tasks. The 25ms action probability inference optimization in this experiment is a direct example of that flexibility.

Zooming out, this experiment points toward a broader direction: expanding large models beyond the role of "conversational assistant" and into "real-time decision engine." When a model can output executable actions at millisecond latency, its application space extends well beyond chat — into control systems, automation, game AI, and more complex agentic frameworks.

For now, this is still a proof-of-concept experiment, and a demo is available on the developer's page. It doesn't establish Kolibri-1's capabilities across complex tasks broadly, but it does demonstrate a genuine possibility: open-source models producing low-latency, structured output.

Aleph Alpha is a German company focused on enterprise AI, known for its emphasis on data sovereignty and European-local deployment. Kolibri is the name of its open-source model series, positioned as lightweight, efficient, and locally runnable. It's worth distinguishing between "open weights" and fully open source: the former makes model parameter files publicly available, allowing free inference and fine-tuning, but doesn't necessarily include full training code, datasets, or commercial licensing; the latter typically includes the complete training pipeline and codebase. For developers looking to deploy in local or private environments, open weights satisfy most customization needs without calling any external API — and that's precisely what makes targeted latency optimization (like real-time action probability inference) far more tractable.

Takeaway

This "less talk, more Breakout" experiment used a lightweight scenario to validate three dimensions of Kolibri-1's capability: task completion without fine-tuning, translation of action probabilities into executable operations, and real-time inference at the 25ms level. For developers following open-source models and real-time AI inference, this is a directional signal worth paying attention to.