Nvidia Rubin Boosts Devin's Throughput by 4.8x — But Is It Actually Faster?

On September 30, AI coding company Cognition became the first customer to run its coding agent Devin on Nvidia's latest AI server, Vera Rubin — rented from cloud provider CoreWeave. Cognition claims that compared to the previous generation of servers, Rubin processes up to 4.8x more tokens per second — and tokens are the basic text units that AI models read and write.

Nvidia had promised earlier this year that Rubin would cut token costs by up to 10x. But a key question immediately follows: Will Devin get faster, cheaper, or both? And what exactly is this oft-cited "4.8x" actually measuring?

What the 4.8x Figure Actually Measures

Cognition's own explanation offers the first clue: on Rubin, each user receives answers at the same output speed as the old system. That means a single GPU can now handle far more total work across more users simultaneously. In other words, this is primarily a story about capacity and cost — not raw speed.

Understanding this requires knowing how GPUs work: a GPU serving a model divides its time among many users. The more users connected, the slower each person receives their response. This creates the core trade-off in any such system — total token throughput versus per-user response speed.

A benchmark chart published by CoreWeave illustrates this curve clearly: the horizontal axis shows per-user output speed (from low to high), while the vertical axis shows how many more tokens Rubin processes per GPU compared to the older GB200. At the slow end, Rubin holds roughly a 3x advantage; at the fast end, it climbs above 7x. The 4.8x figure is marked at the midpoint of the curve — the point where both systems deliver identical per-user response speeds, so all the gains are reflected in total tokens produced per GPU across all users.

Notably, when users need fast responses, Rubin's advantage grows even larger — which is precisely the scenario for coding agents, since the agent must wait for each reply before taking its next step.

The core technical concept here is batching. GPUs don't serve one user at a time; they bundle multiple users' requests into a batch and process them together to maximize hardware utilization. Larger batches mean more total tokens per second per GPU, but each user waits longer. This is the fundamental reason why "total throughput" and "per-user latency" trade off against each other. Rubin's architectural improvements mean it spends less time on the same batch size, or equivalently, can support larger batches while maintaining the same user latency — two sides of the same coin. Understanding this is why "4.8x" doesn't mean your answers arrive 4.8x faster, nor that the server handles 4.8x more users simultaneously — it specifically refers to the ratio of total tokens processed per GPU per second at a particular latency point.

Why Coding Agents Are So Sensitive to Token Throughput

A coding agent like Devin works in a loop: the model reads your code request, writes an instruction, a tool like a code editor or test runner executes it, and the result feeds back to the model.

Cognition's example task is having the agent add a shared warning function to a software project and call it everywhere applicable. This kind of task requires multiple round trips through the loop, with each pass sending more text for the model to read and write — and those small code snippets passed back and forth are exactly the tokens the "4.8x" figure is counting.

The model handling this work is SWE-2, built by Cognition on top of Chinese AI company Moonshot AI's Kimi K2 model, released on September 10, with cost as its headline feature. Cognition says SWE-2 scores within one point of Anthropic's Claude models on its own coding benchmarks, at 64% lower cost. Co-founder Silas Alberti put it plainly: "The cost per token determines what we can deliver."

Kimi K2, the foundation of SWE-2, is an open-source Mixture-of-Experts (MoE) model released by Moonshot AI in 2025, with approximately 32 billion active parameters and over 1 trillion total parameters, excelling at code and tool-calling tasks. The MoE architecture activates only a small fraction of parameters per inference, yielding lower computation — and lower inference cost — for equivalent performance. That's the core logic behind Cognition choosing it as SWE-2's base: matching top closed-source models on coding benchmarks while driving down the per-token price. Crucially, Kimi K2 is released with open weights, letting companies deploy it on their own infrastructure — giving Cognition the ability to run it directly on CoreWeave and tune it for the hardware, rather than paying per-API-call to Moonshot AI.

Are the Savings Real? Pricing Hasn't Been Disclosed

The logic is straightforward: if a GPU produces more tokens per hour, and the hourly rental cost of that GPU hasn't risen proportionally, the cost per token falls.

CoreWeave's publicly listed price for the older GB200 is $10.50 per GPU per hour. By that math, as long as Rubin rents for under roughly $50/hour, it delivers cheaper tokens at the 4.8x point. The problem: CoreWeave hasn't published Rubin's price during its limited availability period, and Cognition's own rates are private — so how much is actually being saved remains unknown.

If the savings are real, Cognition has three options: serve more customers, cut costs, or give each user faster responses. The company hasn't indicated how it will balance these three.

Why Devin Won't Be "4.8x Faster"

This is the most important clarification in this piece. Even if Cognition directed all the gains toward speed, tasks you give Devin wouldn't complete 4.8x faster — because part of every coding task happens outside the model.

Consider a simple illustration with round numbers: a 10-minute task where 5 minutes are spent waiting for the model and 5 minutes running builds, tests, and other tools. Even if model time dropped to zero, the tools still take 5 minutes — so no matter how fast the GPU, the task can at most complete twice as fast.

Nvidia is working on the other half too. According to CoreWeave testing, the isolated sandbox environments used to run code start more than 3x faster on Nvidia's new Vera CPU. That's a separate test on a different chip and doesn't stack on top of the 4.8x — but it targets exactly the portion of a task that GPU speed can't touch.

CoreWeave also notes that its engineers worked with the Cognition team to tune the model-serving software, so some of the gains may come from software optimization rather than new silicon alone.

This bottleneck has a formal name in computer science: Amdahl's Law. It states that when you can only speed up one part of a system, the overall speedup is limited by the proportion of time spent on parts that can't be accelerated. In the extreme case, if half a task's time is spent on unaccelerable steps, the total speedup can never exceed 2x — regardless of how much faster the acceleratable portion gets. That's exactly what the illustrative numbers above demonstrate. Nvidia's simultaneous work on sandbox startup times via the Vera CPU, and Cognition's ongoing efforts to reduce tool-call latency within the agent loop, can both be understood as expanding the boundary of "what can be accelerated" — which is precisely why no single hardware metric can be directly equated with the task completion speed users actually experience.

What This Means for Users

For Devin users, 4.8x is genuinely good news for Cognition's capacity — it gives the company room to make Devin faster or cheaper without making any promises yet.

The first signal may come soon: Devin's pricing page shows that SWE-2 is free to use in its desktop app and command line through October 16. How Cognition prices things after that will be the key indicator of how this hardware dividend translates into real-world value.