One API: The Complete Guide to an Open-Source API Gateway for Managing 30+ LLMs

In today's era of flourishing large language models, developers often need to integrate APIs from multiple LLM service providers simultaneously. Different interface specifications, different authentication methods, and different billing logic turn API management into a nightmare. An open-source project on GitHub called One API is solving this problem elegantly — it has already garnered over 32,800 Stars, making it one of the most popular open-source solutions in the LLM API management space.

What Is One API?

One API is an LLM API management and distribution system created by developer songquanpeng. Its core philosophy is simple: aggregate all mainstream LLM services on the market through a single unified API interface. Whether your backend connects to OpenAI, Anthropic Claude, Google Gemini, or domestic providers like DeepSeek, ERNIE Bot, or Qwen, your frontend application only needs to call one consistent interface format.

In simple terms, One API acts as an "API Gateway" for the large model world. It sits between your application and the various LLM services, handling protocol conversion, key management, traffic distribution, and usage tracking.

An API Gateway is a core infrastructure component in microservices architecture, first popularized by Netflix in its large-scale microservices practice. As the unified entry point for all client requests, it handles cross-cutting concerns such as request routing, protocol conversion, authentication, rate limiting, circuit breaking, and logging/monitoring. In the traditional web services domain, products like Kong, Nginx, and AWS API Gateway are already very mature. In the LLM era, however, the vast differences between model providers' interface specifications — for example, OpenAI uses the Chat Completions format, Anthropic uses the Messages format, and Chinese providers each have their own protocols — have created a pressing need for API gateways specifically designed for large models. One API was born precisely in this context.

The Technical Root Cause of LLM API Fragmentation

The interface differences between LLM service providers go beyond surface-level field naming. The deeper divergence lies in how each provider models the abstract concept of "conversation." OpenAI treats the System Prompt as a message with role=system within the messages array; Anthropic Claude separates the system field from messages entirely, passing it as a top-level parameter. Google Gemini uses a completely different contents structure with nested parts format. These differences mean that even semantically identical requests cannot be directly reused across providers and require structural transformation. Furthermore, each provider has its own error code system, rate limit response headers, and model version naming conventions, making multi-provider adaptation far more complex than it appears on the surface. One API encapsulates these differences within each channel adapter, centralizing the conversion logic and eliminating repetitive adaptation work in business code.

What LLMs Does One API Support?

One API currently supports an impressively wide range of LLM service providers, covering virtually all mainstream options on the market:

International Providers

• OpenAI (GPT-4o, GPT-4, GPT-3.5 series)
• Azure OpenAI
• Anthropic Claude (Claude 3.5 Sonnet, Claude 3 Opus, etc.)
• Google Gemini (Gemini Pro, Gemini Ultra)

Chinese Providers

• DeepSeek (DeepSeek-V3, DeepSeek-R1)
• ByteDance Doubao
• Baidu ERNIE Bot
• Alibaba Qwen
• iFlytek Spark
• Zhipu ChatGLM
• 360 AI Brain
• Tencent Hunyuan

This broad compatibility means developers can configure and switch between different model providers in a single unified management panel without modifying any business code.

The Complexity of Token Metering and Billing Models

Significant differences exist among LLM service providers in token metering methods, which is a major source of API management complexity. OpenAI uses the tiktoken library for BPE (Byte Pair Encoding) tokenization, where Chinese characters are typically encoded as multiple tokens. Chinese providers like Qwen and ERNIE Bot use their own optimized Chinese tokenizers, and the same Chinese text may consume 30%-50% different token counts across providers. Additionally, each provider has different pricing ratios for input vs. output tokens, and some offer discounts for cached tokens (for example, Anthropic's Prompt Caching feature can reduce the input cost of repeated prefixes by approximately 90%). One API's usage tracking feature needs to normalize cost data across channels as accurately as possible under this complexity, which is one of the engineering challenges in its implementation.

One API Core Features Explained

Unified API Protocol Adaptation (OpenAI-Compatible Format)

One API's most core value lies in API protocol unification. It converts all model providers' interfaces into an OpenAI-compatible format. This means any client, SDK, or application that supports the OpenAI API can seamlessly connect to any model behind One API.

OpenAI's Chat Completions API format has become the de facto standard for LLM interfaces. This format uses a messages array (containing role and content fields) as input, a choices array as output, and supports streaming responses (Server-Sent Events) and Function Calling. Since OpenAI was the first to achieve large-scale commercialization, a massive ecosystem has formed around its API format — mainstream frameworks and tools including LangChain, LlamaIndex, Dify, and OpenWebUI all use the OpenAI format as their primary adaptation target. Therefore, converting other models' interfaces to OpenAI-compatible format essentially enables these models to seamlessly integrate into the entire existing AI application ecosystem, rather than merely solving interface difference issues.

Streaming Response Protocol Differences and Unified Handling

Streaming responses are a standard feature of modern LLM applications, allowing users to see the model "output word by word" rather than waiting for the complete response to display all at once. OpenAI uses the Server-Sent Events (SSE) protocol for streaming, with each data chunk prefixed by data: and ending with data: [DONE]. However, different providers have variations in their streaming protocol implementation details: some providers' streaming responses don't include usage statistics, some have error message formats that differ from normal responses, and others send heartbeat packets to maintain connections during network instability. As an intermediary proxy layer, One API needs to handle these edge cases, uniformly converting upstream streaming formats to the standard OpenAI SSE format before forwarding to downstream clients, while also accumulating token usage in real-time during streaming to support billing statistics.

For projects already developed using the OpenAI SDK, the cost of switching to another model provider is virtually zero — you only need to change the API base URL:

# Originally calling OpenAI directly
client = OpenAI(api_key="sk-xxx")

# Switch to calling any model through One API
client = OpenAI(
    api_key="your-one-api-key",
    base_url="https://your-one-api-domain/v1"
)

API Key Management and Secondary Distribution

One API provides a comprehensive key management mechanism. Administrators can configure original API Keys from various service providers in the system, then generate new access tokens to distribute to team members or downstream users.

API Key secondary distribution is essentially a Proxy Token mechanism, with design principles similar to the token delegation pattern in OAuth 2.0. In the traditional approach, if a team has 10 developers who need to use the OpenAI API, they either share a single Key (posing security risks and audit difficulties) or apply for individual Keys for each person (high management cost and difficulty in unified quota control). The secondary distribution mechanism decouples original credentials from user credentials by introducing a middle layer: the system holds the real provider Keys and issues independent access tokens externally, each of which can be bound to independent quotas, rate limits, and model access permissions. This architecture is particularly important in enterprise environments because it satisfies the principle of least privilege and audit trail compliance requirements.

This secondary distribution mechanism brings several