Technical Breakdown of Cursor Composer 2's Distributed RL Training

From Application Company to Foundation Model Company

Cursor recently released Composer 2—an agent model designed for long-horizon coding tasks—sparking widespread attention across the industry. This is not just a product upgrade; it marks Cursor's transformation from a pure application company into a foundation model company. In a Sequoia Capital podcast, Cursor's research lead Federico and Fireworks' Dima shared in-depth technical details behind Composer 2's training.

Federico explained the core logic behind training their own model: "You can think of the model as a storage drive—it can store a limited number of information bits in its weights. We only care about one task: software engineering inside Cursor. What if we allocate all the information bits to that single task?" This extreme focus brought Composer's operating cost down by an order of magnitude compared to Opus.

Training Architecture: Mid-Training + Large-Scale Reinforcement Learning

Composer 2 is built on top of Kimi 2.5 (a 1-trillion-parameter MoE model with 30B active parameters), employing a two-stage approach.

About the MoE Architecture: Mixture of Experts (MoE) is a sparsely activated neural network architecture. Unlike traditional dense models, MoE activates only a subset of parameters during each forward pass—a Router layer dynamically selects the most relevant "expert" sub-networks to process the input. Of Kimi 2.5's 1 trillion total parameters, only about 30B are activated per inference, drastically reducing computational cost while retaining the knowledge capacity of an ultra-large-scale model. The trade-off is the non-determinism of routing decisions: tiny numerical perturbations can cause the router to select entirely different expert combinations, leading to severe consistency issues in distributed training—one of the core engineering challenges discussed in detail below.

Mid-Training

The first stage is code-domain continued training at near-pretraining scale. Mid-training is a training phase between pretraining and fine-tuning that has gained prominence with the rise of domain-specific models. Pretraining builds foundational capabilities on massive general corpora, while mid-training continues next-token prediction on large-scale domain-specific data, aiming to "compress" domain knowledge into model weights rather than injecting it at inference time through prompts. For code models, this means continued pretraining on GitHub repositories, technical documentation, code review records, and similar data, enabling the model to develop deeply internalized "intuition" for codebase structures, API call patterns, common bug types, and more. This provides a broader policy search space for the subsequent RL stage—the model needs to first "know" various possible solutions before RL can teach it to select the optimal one. This is similar to the work done by Cursor's Tab autocomplete model—pure next-token prediction.

Reinforcement Learning (RL)

The second stage is large-scale reinforcement learning, which is the core innovation of Composer 2. Unlike pretraining, RL requires the model to execute complete agent sessions (rollouts) in real environments, potentially involving 50 turns of tool calls, code generation, and environment interaction, with reward signals assigned based on the final outcome.

In LLM reinforcement learning training, a rollout refers to the process of the model executing a complete task sequence in an environment. Unlike supervised learning, RL does not rely on human-annotated "correct answers"—instead, it lets the model explore autonomously and receive reward signals based on the final result. For a coding agent, a single rollout might include: reading code files, executing terminal commands, modifying code, running tests, iteratively fixing issues based on error messages, and dozens of other steps. Cursor used the GRPO (Group Relative Policy Optimization) algorithm in its offline RL stage—an RL algorithm variant proposed by DeepSeek that estimates advantage functions by sampling multiple outputs for the same problem and performing intra-group relative comparisons, avoiding the overhead of training a separate value network as in traditional PPO, making it particularly suitable for long-sequence reasoning tasks.

Distributed RL Infrastructure: Core Engineering Challenges

Asynchronous Pipeline Design

RL training faces a fundamental efficiency problem: if the trainer sits idle waiting for rollouts to complete, GPU utilization drops sharply. Cursor and Fireworks adopted an asynchronous pipeline approach—training and inference run continuously like two production lines in a factory. Rollouts always use the latest model version to generate new sessions, while the trainer continuously digests newly produced results and updates weights.

The cost of this approach is introducing "staleness": by the time a rollout completes, the model weights may have been updated several steps. But in practice, the few percentage points of algorithmic loss are far outweighed by the efficiency gains from keeping GPUs fully utilized.

Global Distributed Inference and Weight Synchronization

Composer 2's training used four clusters worldwide. Training was concentrated on a single high-speed interconnected cluster, while inference components were distributed across small clusters around the world—even borrowing production inference GPUs during off-peak user traffic to accelerate training.

This introduced a critical systems challenge: the Kimi model produces approximately 1TB weight snapshots per training step, every 5–15 minutes. How do you quickly synchronize these to clusters around the globe?

The solution was delta compression. Since only a small fraction of weights change per RL training step, transmitting only the delta (the difference) shrinks the data volume by 20x. Combined with sharded uploads and downloads to fully utilize bandwidth, weight exchanges typically complete within one minute, requiring inference services to pause for only about 30 seconds.

The Numerical Alignment Challenge of MoE Models

Floating-point non-determinism is a severely underestimated problem in distributed RL training. The IEEE 754 floating-point standard allows processors to perform operations in any order as long as precision requirements are met, meaning A+B+C and C+B+A can produce different results—differences typically at the 10th+ decimal place, but significantly amplified after billions of cumulative operations in deep neural networks. In distributed training, differences in computation order across GPU nodes, conversions between different precision formats (FP16/BF16/FP8), and reduction ordering in All-Reduce communication are all sources of numerical inconsistency.

For MoE models, the problem is even more severe: the router layer typically uses softmax followed by top-k selection to choose experts. When two experts have extremely close scores, tiny numerical differences in hidden states can flip the selection result (e.g., choosing Expert 7 instead of Expert 9), thereby activating different parts of the model. If inference activates Expert 7 but training tries to update Expert 9, training collapses.

Solutions include: hand-writing GPU kernels to ensure consistent operation ordering, using "route replay"