One API Deployment Guide: An Open-Source API Gateway for Managing 30+ LLMs in One Place

Project Overview: Why You Need One API

In today's era of flourishing large language models, developers frequently run into a thorny problem: different LLM providers like OpenAI, Claude, Gemini, and DeepSeek each have their own API format, authentication method, and calling conventions. If your application needs to integrate with multiple models simultaneously, the adaptation work alone can consume enormous amounts of effort.

The root cause of this problem lies in the rapid evolution of the entire industry. When OpenAI released the GPT-3 API in 2020, there was no unified specification — each vendor designed their interfaces independently. As the LLM commercialization wave surged in 2022–2023, dozens of providers including Anthropic, Google, Baidu, and Alibaba launched their own APIs, with inconsistent authentication methods (Bearer Token, API Key, OAuth), request formats (varying JSON structures), streaming response implementations (SSE format differences), and error code definitions — resulting in severe ecosystem fragmentation. This problem is particularly pronounced in enterprise applications: when a product needs to dynamically select models based on cost, capability, and compliance requirements, the engineering cost of maintaining multiple SDKs and adapter code is extremely high.

One API is an open-source project created to solve exactly this pain point. Built by developer songquanpeng, it has earned over 32,800 Stars on GitHub with more than 6,200 forks, making it one of the most popular open-source solutions in the LLM API management space.

In simple terms, One API is an LLM API gateway — it converts dozens of different model interfaces into a unified OpenAI-compatible format, allowing you to write a single codebase to call all major large language models.

An API Gateway is a core component in microservice architecture that acts as a middle layer between clients and backend services, handling cross-cutting concerns such as request routing, protocol translation, authentication, rate limiting, circuit breaking, and log auditing. In the traditional web services domain, Kong, Nginx, and AWS API Gateway are common gateway solutions. One API is essentially a purpose-built API gateway for LLM scenarios. On top of general gateway capabilities, it adds AI-specific features like model format conversion, token metering, and multi-channel load balancing — making it a much better fit for the practical needs of LLM API calls than general-purpose gateways.

Core Features Explained

Unified API Adaptation Layer: One Codebase to Call All Models

One API's core value lies in unifying dozens of LLM APIs into an OpenAI-compatible format. You only need to write your code once following the OpenAI API specification, and you can seamlessly switch to Anthropic Claude, Google Gemini, DeepSeek, ByteDance Doubao, Baidu ERNIE Bot, Alibaba Qwen, or any other supported model.

Why choose the OpenAI format as the unified standard? OpenAI's Chat Completions API format (represented by the /v1/chat/completions endpoint) has become the de facto standard for LLM API calls. This format defines a unified request structure — including parameters like model, messages, and temperature — along with both streaming (SSE) and non-streaming response modes. The establishment of this standard was no accident: the release of GPT-3.5-turbo in March 2023 reduced API calling costs by approximately 90%, triggering explosive growth in the developer ecosystem. Frameworks like LangChain and LlamaIndex were designed from the ground up with the OpenAI format as their core abstraction, building up a substantial collection of plugins and integrations based on this format. OpenAI's design choices — Function Calling, System Prompts, and multi-turn conversation message arrays — have proven to be sufficiently universal abstractions that cover the vast majority of conversational AI scenarios. Competitors like Anthropic and Google, while launching their own APIs, have also successively offered OpenAI-compatible modes, further solidifying this standard's position. This network effect has made the OpenAI format the greatest common denominator across the entire industry. Converting other models' APIs into OpenAI-compatible format means developers can directly reuse the entire OpenAI ecosystem's toolchain without building separate adapters for each model.

Currently supported mainstream models include:

• International Providers: OpenAI, Azure OpenAI, Anthropic Claude, Google Gemini
• Chinese Providers: DeepSeek, ByteDance Doubao, ChatGLM (Zhipu AI), ERNIE Bot (Baidu), iFlytek Spark, Qwen (Alibaba), 360 Zhinao, Tencent Hunyuan

This unified adaptation design dramatically reduces the development cost of multi-model integration. Switching models is as simple as changing a single parameter — no business logic needs to be rewritten.

API Key Management and Redistribution

One API is more than just an API gateway — it's a complete Key management system:

• Multi-channel Management: Consolidate API Keys from different providers under unified management, with support for load balancing and failover
• Key Redistribution: Generate sub-keys for distribution to team members or downstream users, enabling fine-grained permission control
• Usage Tracking: Monitor call volume and consumption for each key, facilitating cost accounting
• Quota Control: Set call quota limits for different users to prevent resource abuse

Among these, load balancing and failover are critical capabilities for ensuring production environment stability, but their implementation in LLM API scenarios is more complex than in traditional web services. LLM call response times range from hundreds of milliseconds to tens of seconds, and streaming responses occupy connections for extended periods, meaning simple round-robin strategies can produce severe long-tail effects. A more reasonable approach is dynamic allocation based on the current number of active requests or estimated token throughput. In terms of failure modes, LLM API failure types are also far more complex than traditional services: beyond network timeouts and