AI Gateway Adds Private Inference: Unified Access for Self-Hosted Models

Vercel AI Gateway launches Private Inference, bringing self-hosted and fine-tuned models into unified auth, SDK, and observability management.
Vercel's AI Gateway adds Private Inference, allowing enterprises to connect in-house, fine-tuned, or pre-release models to the gateway via "private providers" with the same auth, SDK, and API as catalog models. Private models use `private/<slug>` IDs, are visible only to the owning team, and support encrypted credentials and optional Zero Data Retention. Requests flow into unified logs and reports with per-model metrics, and endpoints can fail over to catalog models.
AI Gateway Adds Private Inference: Unified Access for Self-Hosted Models
Enterprises' in-house or fine-tuned large models have long remained outside unified AI infrastructure. Vercel's AI Gateway now brings these models back into the main workflow through its Private Inference feature—managing all models with the same authentication, SDK, and observability.

Why Private Inference Is Needed
Enterprises running fine-tuned models, self-hosted models, or pre-release models previously had to manage keys, client code, and logs separately for these models. This meant teams saw only a fragmented view on AI Gateway: some traffic went through the gateway, while the rest was scattered across self-built endpoints.
The core idea of Private Inference is to eliminate this fragmentation. According to the official description, private models can now be called like catalog models through the same authentication, SDK, and API, and share the same observability capabilities. Every model a team uses is managed in one place.
AI Gateway is an infrastructure service provided by Vercel that acts as a unified proxy layer between applications and various model providers (such as OpenAI, Anthropic, Google, etc.). Developers can call multiple models through a single gateway endpoint, and the gateway handles authentication, request format conversion, rate limiting, retries and failover, and centrally records call logs and usage metrics. Its core value lies in simplifying model calls from "one set of keys and clients per provider" to "one API, one set of credentials." The catalog models mentioned in this article refer to these publicly available models natively supported by the gateway, whose endpoints and protocol adaptations are pre-configured so users don't need to maintain connection details themselves.
How Private Models Connect
The onboarding process revolves around a "private provider." When adding an endpoint on the Private Inference page, you need to fill in the base URL, select the API format, and add credentials, then register the models served by that endpoint.
Registration can also be done via Vercel CLI, requiring version 65.16.0 or above. The command-line approach is suitable for automation workflows: first create a private provider and specify the base URL, then create a model and associate it with the provider and upstream API model name.
When calling, use a model ID in the form private/<slug>. Whether it's the Chat Completions, Responses, or Messages interface, AI Gateway translates the request into the format required by the endpoint, so switching between private and catalog models only requires changing the model ID.
Zero Data Retention (ZDR) refers to a data governance model in which a service provider commits to not storing or logging user request and response content. In AI inference scenarios, enabling ZDR typically means the request payload is discarded after inference is complete and is not used for model training, log analysis, or human review. For regulated industries such as finance and healthcare, ZDR is a key configuration for meeting data minimization and compliance requirements. Private Inference allows this policy to be enabled at the private provider granularity, enabling enterprises to maintain control over the retention scope of sensitive data while enjoying the convenience of a unified gateway.
Time to First Token (TTFT) is a core metric for measuring large model response speed, referring to the latency from sending a request to receiving the first output token. Unlike total generation time, TTFT more directly reflects the "waiting feel" perceived by users, especially affecting experiences in streaming interactions and coding agents. Breaking down TTFT by model in observability helps distinguish whether latency bottlenecks come from gateway forwarding, upstream endpoint load, or the model itself.
Key Capabilities Breakdown
Team Isolation and Credential Security
Private models are visible and callable only within the owning Vercel team and do not appear in the public catalog. Credentials are stored encrypted, and Zero Data Retention can be enabled per provider. For enterprises handling sensitive data, this layer of isolation is a prerequisite for private inference to be viable.
Directly Usable by Coding Agents
Coding agents connected to AI Gateway, such as Claude Code, Codex, and Cursor, can directly use private models. This means internally fine-tuned code models no longer need a separate agent access layer.
Unified Observability
Private model requests appear in logs, trace drains, and custom reports alongside catalog model traffic. Observability charts break down request counts, token counts, and time to first token by private model. For teams running multiple models simultaneously, this means troubleshooting and cost analysis no longer require piecing together data across systems.
Configurable Failover
If a private endpoint call fails, you can choose to fall back to a catalog model. This design reduces the impact of self-hosted endpoint stability fluctuations on upper-layer applications and provides a buffer for canary validation of pre-release models.
What It Means for Enterprises
Private Inference solves not a model capability problem but a model governance problem. When enterprises use both public catalog models and self-hosted models, the biggest friction often comes from the operational layer: scattered keys, fragmented logs, and inconsistent calling methods. Once private endpoints are brought into the gateway, model switching degrades to a change of one ID, and monitoring and failover strategies can reuse the same mechanism.
For teams putting fine-tuned models into production, this unified access approach reduces the need to build their own intermediate layer. For specific supported model types, onboarding requirements, and billing details, the official recommendation is to consult the Private Inference documentation.
One-line Summary
Vercel AI Gateway launches Private Inference, bringing self-hosted and fine-tuned models into unified authentication, SDK, and observability management.
Paragraph Summary
Vercel's AI Gateway adds Private Inference, allowing enterprises to connect in-house, fine-tuned, or pre-release models to the gateway through "private providers," using the same authentication, SDK, and API calls as catalog models. Private models are called with private/<slug> model IDs, visible only to the owning team, with encrypted credential storage and optional Zero Data Retention. Onboarding options include console configuration and Vercel CLI 65.16.0 or above. Private model requests flow into logs, trace drains, and reports uniformly, with request counts, token counts, and time to first token viewable by model; endpoint failures can fall back to catalog models. This feature addresses model governance and operational fragmentation, not model capability itself.
Related articles

Gemini 4 Neon Leak: A Potential Strong Contender Against Opus 5.5
A Hacker News thread suggests Gemini 4 Neon looks like an Opus 5.5 competitor. This article unpacks the product positioning behind this community signal, shifts in the top-model competitive landscape, and what it means for developers' model selection strategy.

Claude Fable 5.2's Self-Authored Short Film 'Fable' Sparks Discussion
A short film 'Fable,' credited to Claude Fable 5.2 for script, visuals, and directing, appears on Hacker News. This article analyzes the significance of full-pipeline AI creation, community reactions, and what it means for content creators.

Anthropic IPO Pricing: A Signal of AI Giants Going Public
Anthropic's IPO pricing draws attention. This article analyzes its potential impact on AI industry valuation, commercialization paths, and the competitive landscape, and highlights public-market expectations and challenges for AI companies.