Gamer: A General Vision-Language-Action Foundation Model for Long-Horizon Tasks

Gamer unifies perception, memory, and reasoning in an end-to-end architecture, training a general game agent across 2D and 3D with keyboard-mouse trajectories.
Gamer is an end-to-end framework for general game agents that integrates perception, memory management, and multi-step reasoning into a single model and uses one training and deployment pipeline for both 2D GUIs and 3D simulated worlds. Trained on tens of thousands of hours of device-level keyboard-mouse trajectories, it uses continual pre-training, sparse-thinking instruction fine-tuning to cut redundant computation, and stable multi-turn RL to tackle limited context, rollout error accumulation, and sparse rewards. A cloud-edge inference engine enables massive concurrency and unbounded context execution. Gamer shows near-skilled-player generalization on several 3D open-world games and 2D benchmarks, offering a reusable path toward general agents spanning digital and physical worlds.
Game agents are moving from single-task demonstrations toward generality. The newly proposed Gamer framework attempts to unify long-horizon interaction across 2D graphical interfaces and 3D simulated worlds within a single end-to-end architecture, offering a reusable technical path toward general-purpose agents.

An End-to-End Architecture Unifying Perception, Memory, and Reasoning
Gamer's core design integrates perception, memory management, and multi-step reasoning into a single end-to-end model. Unlike common agent approaches that split perception, planning, and execution into separate modules, this unified structure allows the model to continuously maintain an internal representation of the environment during long-horizon tasks, rather than reassembling context at every decision step.
Supporting this architecture is a shared training and deployment pipeline that is compatible with both traditional 2D graphical interfaces and complex 3D simulated worlds. This means the same model does not need rewritten interface logic for different environments, reducing the engineering cost of cross-domain transfer. For game agents that need to run over long periods, the uniformity of the architecture directly determines whether they can maintain behavioral consistency when switching environments.
Learning Environment Dynamics from Tens of Thousands of Hours of Keyboard and Mouse Trajectories
Gamer's training does not start from a carefully annotated task dataset, but rather from keyboard and mouse interaction trajectories collected at the device level, totaling tens of thousands of hours. These trajectories cover simulated games and embodied environments, enabling the model to learn environment dynamics simultaneously in both the physical and virtual domains.
This choice is noteworthy: keyboard and mouse trajectories are highly repetitive and have a low signal-to-noise ratio, and are typically considered unsuitable as a learning signal for general capabilities. But Gamer's results show that as long as data scale and learning algorithms scale together, even repetitive operation trajectories can support the stable acquisition of complex skills. This provides a positive case for the question of whether "data scaling can compensate for insufficient signal quality."
Continual Pre-Training refers to continuing next-token-prediction-style training on a base model that already possesses general capabilities, using domain-relevant data, so that the model acquires the statistical regularities and behavioral patterns of that domain rather than training from scratch. Unlike instruction fine-tuning, it does not rely on task annotations, only on raw sequence data, making it particularly suitable for unlabeled, low-signal-to-noise signals such as keyboard and mouse trajectories. In this stage, Gamer lets the model implicitly learn the causal dynamics of "keystroke—screen change" from massive interaction sequences, providing initialization weights with environment common sense for subsequent instruction fine-tuning and reinforcement learning. This is also the precondition for converting low-quality trajectories into usable capabilities.
Sparse-Thinking Instruction Fine-Tuning: Cutting Redundant Computation
After continual pre-training, Gamer introduces sparse-thinking instruction fine-tuning. Its goal is to compress redundant computation in the reasoning process while retaining the reasoning capability required for complex decisions.
A practical contradiction faced by long-horizon task agents is that the more thorough the reasoning, the higher the decision quality, but the worse the real-time performance. Sparse thinking alleviates this contradiction by having the model take a lightweight path in most steps and expand reasoning only at key nodes. The original text emphasizes that this step "substantially reduces redundant computation," but does not disclose the specific compression ratio, and its actual benefits still need to be verified by subsequent evaluation.
Multi-Turn Reinforcement Learning Solves Three Major Challenges of Long-Horizon Interaction
The policy training stage adopts a stable multi-turn reinforcement learning method that specifically addresses three types of problems in long-horizon interaction: limited context windows, accumulated rollout errors, and sparse rewards.
These three are coupled with one another—a limited context window leads to loss of historical information, errors amplify as they accumulate over turns, and sparse rewards make credit assignment difficult. Gamer's method includes dedicated design for training stability, enabling the model to maintain policy consistency over long episodes rather than drifting in behavior after dozens of steps. This is a key engineering challenge that distinguishes long-horizon task agents from short-horizon task agents.
Rollout error accumulation is a typical problem for reinforcement learning in long-episode tasks: when a policy executes in the environment (rollout), prediction or action deviations at each step change the distribution of subsequent states, causing the agent to gradually drift away from states seen during training. Errors amplify nonlinearly with episode length, ultimately leading to behavioral drift or even collapse. Mitigation approaches usually include shortening the effective credit assignment span, correcting the state distribution, or introducing multi-turn resets and stabilization constraints during training. Gamer has dedicated design for this problem, with the goal of maintaining policy consistency over dozens of steps or even longer episodes, which is precisely one of the core engineering differences between long-horizon task agents and short-horizon task agents.
Cloud-Edge Inference Engine Supports Real-Time Concurrency
To meet real-time interaction requirements, Gamer is accompanied by a cloud-edge collaborative inference engine. This engine can simultaneously serve a large number of simulated game environments, support unbounded context execution, and keep online inference latency at a low level.
Unbounded context execution is the key point here: long-horizon tasks naturally require the model to remember events that occurred long ago, while traditional context windows cannot expand indefinitely. The cloud-edge architecture provides an engineering-level solution for long-episode interaction by pushing part of state management down to the edge side.
Unbounded context execution means that when an agent performs long-horizon tasks, it is not limited by a fixed context window length and can continuously access and utilize historical information from far earlier than the current moment. Common approaches include compressing historical states and storing them in external memory, or pushing part of state management down to the edge side and recalling them on demand via retrieval or summarization mechanisms, rather than stuffing all historical tokens into the model context. For game agents, this means it can remember map information or task progress from hundreds of steps earlier without sacrificing online inference latency, which is the key support that makes long-episode interaction feasible at the engineering level.
Dual Validation on 3D Open Worlds and 2D Benchmarks
In multiple challenging 3D open-world games, Gamer demonstrates generalization ability and operational efficiency comparable to skilled players; on a series of traditional 2D game benchmarks, it also achieves competitive results.
It should be noted that the original text describes performance only in qualitative terms and does not publish specific win rates, scores, or baseline comparison values. These conclusions should therefore be understood as directional validation rather than precise performance rankings. Nevertheless, achieving comparable performance across both 2D and 3D domains itself demonstrates the transfer potential of the unified architecture.
Implications for General-Purpose Agents
Gamer's value lies not only in games themselves. It validates a path: using device-level interaction trajectories as fuel, combined with sparse reasoning and stable multi-turn reinforcement learning, it is possible to train agents that stably acquire complex skills in environments where digital and physical properties coexist.
The original text points out that this method "provides a path toward general-purpose agents capable of interacting between the digital and physical worlds." From games to embodied environments, the environment-dynamics knowledge carried by keyboard and mouse trajectories may be exactly the kind of underlying signal that general-purpose agents need to cross the boundary between the virtual and the real.
One-line Summary
Gamer unifies perception, memory, and reasoning in an end-to-end architecture, using keyboard and mouse trajectories to train a general game agent spanning 2D and 3D.
Paragraph Summary
Gamer is an end-to-end framework for general game agents that integrates perception, memory management, and multi-step reasoning into a single model, and uses the same training and deployment pipeline to support both 2D graphical interfaces and 3D simulated worlds. Its training starts from tens of thousands of hours of device-level keyboard and mouse interaction trajectories; after continual pre-training, it introduces sparse-thinking instruction fine-tuning to cut redundant computation, and then uses stable multi-turn reinforcement learning to address the three major long-horizon interaction challenges of limited context, accumulated rollout errors, and sparse rewards. The accompanying cloud-edge inference engine supports concurrency across many environments and unbounded context execution. On multiple 3D open-world games and 2D benchmarks, Gamer demonstrates generalization ability close to that of skilled players, providing a reusable path toward general-purpose agents that span the digital and physical worlds.
Related articles

GnuPG Discloses Remote Code Execution Vulnerability, Renewed Focus on Encryption Tool Security
GnuPG's official mailing list disclosed a remote code execution (RCE) vulnerability, sparking nearly 100 comments on Hacker News. This article summarizes the disclosure, impact assessment, and security lessons for encryption tools to help developers prioritize incident response.

Together.ai Reportedly Exposes Prompt Leakage Vulnerability: Request Handling Mechanism Raises Concerns
Together.ai is reportedly exposed on Hacker News for request prompt leakage. This article analyzes the incident background, the isolation challenges of multi-tenant inference platforms, and the security implications for developers and the AI infrastructure industry.

Codex for Beginners: 13 Hands-On Projects from Installation to Building Websites and Blender Modeling
Complete Bilibili Codex beginner tutorial: from registration and installation, permissions and model selection, to 13 hands-on scenarios including environment setup, GPT image generation for personal websites, Skill packaging, MCP connection to Blender modeling, CapCut control, scheduled tasks, and mobile control.