Mercury 3 Launches on AI Gateway: Diffusion-Based Text Generation Exceeds 1,000 Tokens/s

When Text Generation No Longer Happens Token by Token: Mercury 3's Diffusion Approach

Inception's Mercury 3 model is now available on Vercel AI Gateway, with a 50% launch-period discount. What's most striking about this model isn't its parameter count, but how it generates text—using diffusion for parallel refinement rather than the traditional autoregressive token-by-token output of large models.

According to figures published by Inception, Mercury 3 performs strongly across multiple benchmarks, with generation speeds exceeding 1,000 tokens per second. For latency-sensitive applications, this number represents a step-change in interaction experience.

Where the Speed Comes From: The Engineering Significance of Parallel Refinement

Autoregressive models predict one token at a time, with speed constrained by the inherent bottleneck of serial decoding. Mercury 3 takes a different path: it performs multiple rounds of parallel refinement over an entire passage of text through a diffusion process, which in theory can make fuller use of parallel compute.

This architecture is particularly friendly to latency-sensitive workloads. The officially named scenarios include browser agents, real-time search, interactive education, and customer support—tasks that share a common trait: users can't afford to wait. Take browser agents, for example: they need to rapidly alternate between page operations and decisions, and each generation's latency accumulates into visible stuttering.

Interestingly, diffusion-based text generation is not without its costs; the balance between the number of refinement rounds and output quality requires engineering tuning. Inception claims "strong results" on benchmarks, but the original text did not disclose specific comparison models or score details, so actual performance still needs independent evaluation to verify.

The core difference between autoregressive and diffusion generation lies in the dependency structure of decoding. Autoregressive models generate tokens sequentially from left to right, with the distribution of the nth token depending on the preceding n-1 tokens, making them inherently serial: even if the GPU has substantial idle compute, multiple positions cannot be computed simultaneously. Diffusion models, by contrast, start from a noisy initial sequence and update all positions simultaneously across multiple denoising steps, with each step using the entire context for parallel prediction—thus converting serial dependency along the "depth" (token count) into iteration over "rounds" (refinement steps). The difficulty of text diffusion is that the discrete token space has no continuous Gaussian noise, requiring specially designed noise-adding/denoising mechanisms (such as masking, absorbing states, or embedding-space perturbation). Too few refinement rounds lead to insufficient quality, while too many negate the parallel benefit—this is precisely the engineering tuning mentioned in the article.

Million-Token Context: Room for Long Conversations and Long Documents

Mercury 3 offers a 1 million token context window. For applications like real-time search and multi-turn customer support, long context isn't just about "fitting more content in"—it's about letting the system maintain coherence across extended conversation histories or reducing information truncation in long-document Q&A.

Combined with generation speeds exceeding 1,000 tokens/s, this combination points to a class of applications previously difficult to deploy due to latency constraints: agent workflows that need to handle large amounts of context while responding quickly.

A million-token context does not equal effective utilization of a million tokens. Long-context models typically face the "lost in the middle" problem: retrieval accuracy for information located in the middle of the context is lower than for the beginning and end. In addition, the cost of attention computation grows with sequence length; although inference frameworks commonly employ optimizations such as chunked prefill and KV cache reuse, at million-scale lengths, memory usage and first-token latency still rise significantly. For agent workflows, the value of long context also depends on whether the model can stably maintain state across multiple tool calls and correctly reference information from earlier turns—a factor that often determines practical usability more than window size alone.

How to Access: One Model Name, or One CLI Command

The barrier to calling Mercury 3 on AI Gateway is low. After setting the AI_GATEWAY_API_KEY environment variable, specify the model name as inception/mercury-3. Below is a minimal working AI SDK example:

import { streamText } from 'ai';

const result = streamText({
  model: 'inception/mercury-3',
  prompt: 'Write a brief, friendly reply to a customer whose delivery is late.',
});

for await (const text of result.textStream) {
  process.stdout.write(text);
}

To use it in coding agents like Claude Code, Codex, or Cursor, install the latest version of Vercel CLI and run npx vercel ai-gateway setup, then select inception/mercury-3 in the agent. You can also try it directly in the model playground.

The Value of AI Gateway: Unified API and Zero Markup

AI Gateway's positioning is as a unified entry point for model calls, while also providing usage and cost tracking, retry and failover configuration to improve availability beyond that of any single provider. It also includes built-in custom reporting, Zero Data Retention support, API key budgets, and routing rules.

On pricing, AI Gateway reflects provider prices as-is, with no markup and no platform fees for inference—including BYOK (bring your own key) requests. For teams that need to switch between multiple models, this lowers the cost of experimentation; for a new model like Mercury 3, the 50% launch discount further lowers the barrier to early adoption.

A unified entry layer like AI Gateway solves the engineering fragmentation caused by multi-provider integration: different vendors' APIs differ in authentication methods, request formats, streaming protocols, error codes, and rate limits. If teams integrate directly, they need to maintain multiple sets of adaptation logic in their business code. The gateway layer abstracts each provider's model into a unified model identifier, converging cross-cutting concerns such as authentication, retries, timeouts, and failover into one place. Zero Data Retention addresses scenarios with compliance requirements, ensuring that request content is not retained by the gateway or provider for purposes beyond training or auditing. In BYOK mode, teams bring their own provider keys, and the gateway only handles routing and observability—this is also the premise that makes "no markup" possible, since billing still occurs on the provider side.

A Step Toward Mainstream Diffusion-Based Text Generation

The significance of Mercury 3 landing on AI Gateway isn't just that there's one more model to choose from. It puts "diffusion-based text generation"—a technical path different from the mainstream autoregressive route—onto a distribution channel within developers' reach. Whether the speed advantage can be realized under real business workloads, and whether benchmark performance can withstand third-party reproduction, will be the observation points worth watching next.

For teams building low-latency agents, there is now an option worth testing.

One-line Summary

Inception's Mercury 3 replaces token-by-token autoregression with diffusion-based parallel refinement, now available via Vercel AI Gateway.

Paragraph Summary

Mercury 3 is a text generation model from Inception, now available on Vercel AI Gateway with a 50% launch-period discount. Its core difference lies in how it generates: rather than autoregressive token-by-token decoding, it performs multiple rounds of parallel refinement over an entire passage through a diffusion process. Inception claims generation speeds exceeding 1,000 tokens per second, targeting latency-sensitive scenarios such as browser agents, real-time search, interactive education, and customer support. The model offers a 1 million token context window, which combined with high-speed generation can support long-conversation and long-document workflows. Access is simple: set AI_GATEWAY_API_KEY and specify the model name inception/mercury-3, or enable it in coding agents via CLI. AI Gateway bills at provider list prices with no markup or platform fees. However, Inception has not disclosed the specific comparison models and scores behind its benchmarks, so actual performance still needs independent evaluation to verify.