[KongchangAI]
Product私有推理

Private Inference

A Vercel AI Gateway feature that allows enterprises to connect self-hosted, fine-tuned, or pre-release models through a unified gateway, sharing authentication, SDK, API, and observability capabilities.

Timeline (last 90 days)

Nov 16

Vercel AI Gateway adds a Private Inference feature that allows enterprises to connect self-hosted, fine-tuned or pre-release models to the gateway through the same authentication, SDK and observability.

Unverified50%
Nov 16

To connect a private model through Private Inference, you need to add a base URL, choose an API format and add credentials, then register the models served by that endpoint.

Unverified50%
Nov 16

Registering a private model via the Vercel CLI requires CLI version 65.16.0 or above.

Unverified50%
Nov 16

When calling a private model, use a model ID in the form `private/<slug>`; AI Gateway translates the request into the format the endpoint requires, and switching between private and catalog models requires only changing the model ID.

Unverified50%
Nov 16

Private Inference allows enabling a Zero Data Retention policy at the granularity of individual private providers.

Unverified50%
Nov 16

Private models are visible and callable only within the owning Vercel team and do not appear in the public catalog; credentials are stored in encrypted form.

Unverified50%
Nov 16

Requests to private models appear alongside catalog model traffic in logs, trace drains and custom reports, and observability charts break down request counts, token counts and time to first token by private model.

Unverified50%
Nov 16

If a call to a private endpoint fails, AI Gateway can optionally fall back to a catalog model.

Unverified50%
Nov 16

The core problem Private Inference solves is model governance (such as scattered keys, fragmented logs and inconsistent invocation methods), not model capability.

Unverified50%
Nov 16

Once private endpoints are brought into AI Gateway, switching models degrades to a change of one ID, and monitoring and fallback policies can reuse the same mechanism.

Unverified50%

All Facts (10)

Unverified

Once private endpoints are brought into AI Gateway, switching models degrades to a change of one ID, and monitoring and fallback policies can reuse the same mechanism.

50%
Unverified

To connect a private model through Private Inference, you need to add a base URL, choose an API format and add credentials, then register the models served by that endpoint.

50%
Unverified

Registering a private model via the Vercel CLI requires CLI version 65.16.0 or above.

50%
Unverified

When calling a private model, use a model ID in the form `private/<slug>`; AI Gateway translates the request into the format the endpoint requires, and switching between private and catalog models requires only changing the model ID.

50%
Unverified

Private Inference allows enabling a Zero Data Retention policy at the granularity of individual private providers.

50%
Unverified

Private models are visible and callable only within the owning Vercel team and do not appear in the public catalog; credentials are stored in encrypted form.

50%
Unverified

Requests to private models appear alongside catalog model traffic in logs, trace drains and custom reports, and observability charts break down request counts, token counts and time to first token by private model.

50%
Unverified

If a call to a private endpoint fails, AI Gateway can optionally fall back to a catalog model.

50%
Unverified

The core problem Private Inference solves is model governance (such as scattered keys, fragmented logs and inconsistent invocation methods), not model capability.

50%
Unverified

Vercel AI Gateway adds a Private Inference feature that allows enterprises to connect self-hosted, fine-tuned or pre-release models to the gateway through the same authentication, SDK and observability.

50%

Source Articles