Private Inference
A Vercel AI Gateway feature that allows enterprises to connect self-hosted, fine-tuned, or pre-release models through a unified gateway, sharing authentication, SDK, API, and observability capabilities.
Timeline (last 90 days)
Vercel AI Gateway adds a Private Inference feature that allows enterprises to connect self-hosted, fine-tuned or pre-release models to the gateway through the same authentication, SDK and observability.
To connect a private model through Private Inference, you need to add a base URL, choose an API format and add credentials, then register the models served by that endpoint.
Registering a private model via the Vercel CLI requires CLI version 65.16.0 or above.
When calling a private model, use a model ID in the form `private/<slug>`; AI Gateway translates the request into the format the endpoint requires, and switching between private and catalog models requires only changing the model ID.
Private Inference allows enabling a Zero Data Retention policy at the granularity of individual private providers.
Private models are visible and callable only within the owning Vercel team and do not appear in the public catalog; credentials are stored in encrypted form.
Requests to private models appear alongside catalog model traffic in logs, trace drains and custom reports, and observability charts break down request counts, token counts and time to first token by private model.
If a call to a private endpoint fails, AI Gateway can optionally fall back to a catalog model.
The core problem Private Inference solves is model governance (such as scattered keys, fragmented logs and inconsistent invocation methods), not model capability.
Once private endpoints are brought into AI Gateway, switching models degrades to a change of one ID, and monitoring and fallback policies can reuse the same mechanism.
All Facts (10)
Once private endpoints are brought into AI Gateway, switching models degrades to a change of one ID, and monitoring and fallback policies can reuse the same mechanism.
50%UnverifiedTo connect a private model through Private Inference, you need to add a base URL, choose an API format and add credentials, then register the models served by that endpoint.
50%UnverifiedRegistering a private model via the Vercel CLI requires CLI version 65.16.0 or above.
50%UnverifiedWhen calling a private model, use a model ID in the form `private/<slug>`; AI Gateway translates the request into the format the endpoint requires, and switching between private and catalog models requires only changing the model ID.
50%UnverifiedPrivate Inference allows enabling a Zero Data Retention policy at the granularity of individual private providers.
50%UnverifiedPrivate models are visible and callable only within the owning Vercel team and do not appear in the public catalog; credentials are stored in encrypted form.
50%UnverifiedRequests to private models appear alongside catalog model traffic in logs, trace drains and custom reports, and observability charts break down request counts, token counts and time to first token by private model.
50%UnverifiedIf a call to a private endpoint fails, AI Gateway can optionally fall back to a catalog model.
50%UnverifiedThe core problem Private Inference solves is model governance (such as scattered keys, fragmented logs and inconsistent invocation methods), not model capability.
50%UnverifiedVercel AI Gateway adds a Private Inference feature that allows enterprises to connect self-hosted, fine-tuned or pre-release models to the gateway through the same authentication, SDK and observability.
50%