Using Skills to Let AI Automatically Manage LLM API Gateway Stations

What Is the New Skills Paradigm in AI Programming

As AI-assisted programming gradually becomes part of daily development workflows, CLI tools and IDE plugins like Claude Code, Cursor, and OpenCode have become standard equipment for many developers. Yet a common frustration persists: the output quality of large language models varies widely—sometimes brilliant, sometimes off the mark. How to make AI produce high-quality code stably and controllably has become a new core challenge.

This is precisely the problem the Skills paradigm aims to solve. According to Bilibili tech creator Xiaofu Ge in his talk "The New AI Programming Paradigm - Skills Edition," Skills is essentially a mechanism for constraining and guiding large model behavior—through preset scripts, templates, and intent routing, AI agents strictly follow established rules to execute tasks in specific scenarios, thereby "constraining code quality" and avoiding uncertain outputs.

The Skills paradigm grew out of engineering practices in the AI Agent field. An AI agent refers to an LLM application system capable of perceiving its environment, autonomously planning, and executing multi-step tasks. Unlike single-turn Q&A calls, it possesses capabilities such as tool invocation (Function Calling), memory management, and multi-turn reasoning. Since the publication of the ReAct (Reasoning + Acting) framework paper in 2023, agent architectures have rapidly moved from academic concepts to engineering practice.

ReAct Framework Background: The ReAct framework was proposed by Google Research in 2022. Its core idea is to alternate between the reasoning and acting of large language models, forming a "Think → Act → Observe" loop. In each iteration, the model first generates a natural-language reasoning trace (Thought), then outputs structured action instructions (Action) accordingly, and finally injects environmental feedback (Observation) into the next round of context. This framework directly influenced the design philosophy of engineering solutions like Skills: reasoning is handled by the LLM, while actions are carried out by deterministic scripts, with information passed between them through structured interfaces. This gives the entire agent system both the flexibility of natural language and the certainty of script execution. Notably, subsequent frameworks such as Reflexion, AutoGPT, and MetaGPT all adopted this "reasoning-action" layered architecture, further validating the engineering universality of this paradigm in complex task scenarios.

However, traditional LLM invocation relies on Prompt Engineering, which has three core limitations: model version sensitivity, context window pollution, and unstable output formats—even when constrained with JSON Schema, models may still exhibit format drift. These limitations drove the birth of hybrid "prompt + script" architectures like Skills: using prompts to handle natural language understanding, and scripts to ensure execution determinism, since prompts alone are difficult to reuse at scale in production environments. By decoupling intent recognition, script invocation, and constraint templates, Skills essentially constructs a set of deterministic behavioral boundaries, enabling an uncertain probabilistic model to exhibit near-deterministic reliability within a specific task domain.

This resonates deeply with the "Design by Contract" philosophy in software engineering—a theory proposed by Bertrand Meyer in 1986, which requires system components to clearly define their interaction boundaries through preconditions, postconditions, and invariants.

Mapping Design by Contract to AI Engineering: The three core elements of Design by Contract are the precondition (constraints that must be satisfied before a call), the postcondition (the state the system must guarantee after a call), and the class invariant (properties an object should maintain in any observable state). In Skills engineering practice, these three elements are concretely mapped as follows: a skill's input JSON Schema defines the precondition (specifying what format of instructions can trigger the skill), the output template defines the postcondition (specifying what structure the script must return), and the constraint rule set serves as the invariant (e.g., "any channel operation must first pass a health check"). This correspondence means the design of Skills is not merely a summary of engineering experience but is backed by rigorous formal theory. Notably, the Eiffel language (designed by Meyer himself) compiles contracts directly into the language specification, whereas modern-era Skills leverage JSON Schema and structured prompts to achieve similar contract-enforcement capabilities within probabilistic systems in a more lightweight manner—a creative extension of classic software theory by AI engineering.

The constraint templates of Skills are exactly the engineering embodiment of this philosophy in the AI era: they do not attempt to control the LLM's internal reasoning process, but instead set strict format and behavioral specifications at both the input and output ends, incorporating the probabilistic model into a deterministic engineering framework, thereby maintaining overall predictability amid local uncertainty in complex systems.

In short, Skills transforms AI from a "random improvising assistant" into a "professional tool that follows standard procedures."

A Real Pain Point: Managing Multi-Channel LLM APIs

To make the value of Skills more concrete, Xiaofu Ge chose a practical scenario for demonstration—the distribution and management of LLM channels.

Many developers will surely relate to this pain point. In daily development, we often accumulate various LLM resources from multiple platforms: free quotas offered by vendors like Xiaomi and iFlytek, time-limited trial tokens, locally deployed Tongyi Qianwen instances, and various paid API keys. These resources are scattered everywhere—some limited monthly, some billed daily, some with fixed quotas.

Even worse, when using multiple tools like OpenCode, Cursor, and Claude Code simultaneously, each tool requires its own model configuration—tedious to set up, costly to maintain, and requiring manual troubleshooting one by one when a channel fails.

Xiaofu Ge's proposed solution is: integrate all channels into a unified API gateway station, then let the AI agent manage them autonomously. In his own words—"Let your agent maintain its own tokens. You've grown up now, go find your own food."

Architecture Design of the Automated Management Platform

The core of this solution is an "AI Model API Automated Management Platform," whose architecture can be broken down into the following layers:

User Interaction Layer

The top layer is the AI agent entry point. Users issue instructions in natural language, such as "give me a key," "add a channel," "verify available models"—all of which are captured and processed by Skills.

Intent Routing Layer

Skills serve as the core hub, performing intent routing by recognizing keywords in user input and invoking the corresponding execution scripts.

Intent routing technology has evolved through three generations, from rule engines to statistical classification models to large language models. The first generation was based on keyword-matching trees—fragile and hard to maintain; the second generation relied on pretrained models like BERT for intent classification, significantly improving accuracy but requiring large amounts of labeled data; the third generation, represented by the Skills paradigm, leverages the large language model itself to simultaneously perform intent understanding, slot filling, and parameter structuring—requiring no additional training data and capable of handling ambiguous expressions.

Slot filling is a classic NLP task in dialogue systems, referring to extracting the structured parameters needed to fulfill a specific intent from a user's utterance—for example, extracting the date, time, and departure city from "book me a flight from Beijing to Shanghai at 3 PM tomorrow." Traditional approaches relied on sequence-labeling models such as Conditional Random Fields (CRF), requiring large amounts of domain-labeled data. The emergence of large language models completely changed this paradigm: LLMs can perform intent recognition and slot extraction simultaneously in a zero-shot manner and directly output structured formats like JSON. The innovation of the Skills paradigm lies in delegating routing judgment itself to the large model—through carefully designed system prompts, the model acts both as an intent recognizer and a parameter extractor, ultimately triggering the corresponding deterministic script. This "zero-shot intent routing" capability allows developers to build complex multi-intent distribution systems simply by describing each skill's trigger conditions in the system prompt, greatly lowering the engineering barrier for dialogue systems.

From Rasa to LLM: The Paradigm Shift in Dialogue System Intent Architecture: Before large language models became widespread, frameworks like Rasa, Dialogflow, and Microsoft LUIS dominated the field of dialogue system intent routing. These frameworks typically required developers to prepare dozens to hundreds of labeled examples per intent, train dedicated intent classification models, and handle entity extraction through an NLU pipeline. This process was not only time-consuming and labor-intensive but also faced the "cold start" problem—new intents had to be supplemented with sufficient training data before going live. The emergence of LLMs completely overturned this paradigm: by describing each intent's trigger conditions in natural language within the system prompt, models can achieve zero-shot intent recognition and parameter extraction, compressing what was once a weeks-long intent development cycle into a few hours. The Skills paradigm is precisely the engineering embodiment of this trend, and its core insight is that the LLM itself is a general-purpose "semantic router"—given a sufficiently clear "routing table description," it can make routing decisions more flexibly than a dedicated classification model.

This layered architecture of "LLM for understanding, scripts for execution" effectively avoids the hallucination risks of relying entirely on LLM reasoning to perform operations. These scripts support mainstream languages such as Python, TypeScript, and Node.js—as long as they can run locally, offering great flexibility.

Data and Constraint Layer

During script execution, operational data is persistently stored, while constraint templates are introduced to regulate every step of behavior, ensuring output consistency.

Service Wrapping Layer

At the bottom is the encapsulation of One API. One API is a widely used LLM interface management and distribution tool, whose core value lies in wrapping the LLM APIs of different vendors into a unified, OpenAI-compatible interface.

One API and the Industry Impact of the OpenAI Compatibility Standard: The OpenAI API specification gradually became the de facto industry standard after the release of GPT-3 in 2020, and its influence expanded dramatically in 2022–2023 with the explosion of ChatGPT. The reason the OpenAI API specification became the de facto standard lies in its clean and easy-to-use REST+JSON design, the intuitive and universal message-role system of Chat Completions (system/user/assistant), and the widely copied Server-Sent Events (SSE) streaming output implementation. The rise of gateway tools like One API is essentially an engineering response to this standardization trend—through a unified "protocol translation layer," they map the differentiated APIs of dozens of model vendors into a single OpenAI-compatible interface, allowing upper-layer applications to achieve true "integrate once, reuse across multiple models." One API has accumulated over 20,000 stars on GitHub, supports integration with more than 40 model providers, and has become one of the most widely used model gateways in China's AI application development ecosystem. Currently, nearly all mainstream LLM vendors at home and abroad (including Anthropic, Google Gemini, Baidu ERNIE, Alibaba Tongyi, iFlytek Spark, etc.) offer an OpenAI-compatible mode.

Such "Protocol Translation Layer" tools represent an important architectural pattern emerging in the AI infrastructure field. From a software architecture perspective, One API plays the role of the converter between the "target interface" and the "adaptee" in the Adapter Pattern, enabling upper-layer applications to achieve "integrate once, reuse across multiple models." One API supports enterprise-grade features such as load balancing, priority-based grouping, usage-based billing, and automatic retries, and achieves multi-user isolation management through a token mechanism. As the number of LLM vendors continues to grow, such tools are becoming an indispensable "glue layer" in enterprise AI tech stacks, comparable in importance to the API gateways of the early cloud computing era. This Skills solution essentially puts an AI agent's "control shell" over One API, freeing users from having to manually operate its complex management interface.

The core capabilities implemented by the overall architecture include: one-click distribution, automated management, health checks, automatic degradation and recovery, and load balancing. Xiaofu Ge jokingly calls this project the "Free Token Plan"—use free resources when available, and switch to the next one when they run out.

Hands-On Demo: From Deployment to Usage

Xiaofu Ge fully demonstrated the workflow of this Skills setup in the video. Below is a summary of several key steps.

Step 1: Import the Skill Package

Download the skill package and extract it, rename the branch directory, then delete the old skill in tools like Claude Code, re-import it, and save. Once imported, simply ask the AI "what does this skill do," and it will automatically introduce its features—one-click distribution, automated management, health checks, automatic degradation and recovery, load balancing, and more.

Step 2: Deploy One API

The entire demo relies on a deployed One API service, paired with a MySQL database. Xiaofu Ge recommends deploying with Docker first, which works for both cloud servers and local environments, and the official website provides detailed deployment documentation.

Step 3: Add Channels

After deployment, use natural language instructions to have Skills "add a One API channel" and provide the service address. The AI will automatically complete the integration, after which you can progressively add various free or paid model channels.

A Detail Worth Noting: Filling in the base_url

A typical pitfall appeared during the demo: manually entering a base_url with a /v1 suffix causes an incorrect interface address. This issue relates to REST API versioning conventions—OpenAI was the first to use /v1 as the version prefix for its API, and compatible services followed this convention. One API internally hardcodes /v1 into its standard path-concatenation logic, so if a user repeatedly adds this suffix when filling in the base address, the final request path becomes /v1/v1/chat/completions, resulting in 404 errors or routing failures. Such "double version prefix" problems are extremely common in API integration scenarios, and the usual solution is to use a regular expression to strip trailing characters like /v1 or / before writing the configuration. Xiaofu Ge specifically reminds us that One API follows the standard protocol by default, so you only need to fill in the root address—any extra /v1 suffix must be manually removed. These easily overlooked details are very valuable for developers getting started for the first time.

The Evolution of REST API Versioning Strategies: API versioning has long been a controversial topic in web service design, with three mainstream approaches: URL path versioning (e.g., /v1/), header versioning (e.g., Accept: application/vnd.api+json;version=1), and query parameter versioning (e.g., ?version=1). OpenAI chose the most intuitive URL path versioning strategy and established an industry convention on top of it. However, this convention also introduced integration friction: different services define the boundary of "base URL" differently—some expect it to include /v1, while others expect only the protocol and domain. When building multi-model gateway systems, it is advisable to standardize the base_url at the configuration layer: uniformly strip trailing version paths and slashes, and let the gateway layer handle the concatenation, thereby eliminating the impact of downstream service differences on upper-layer configuration. This is also a typical scenario where Skills constraint templates can add value—by building regex validation rules into the input schema, URL format normalization is automatically completed before data is written.

Step 4: Generate a Key and Verify

Once configured, a simple "give me a key" prompts the AI to automatically generate an API key. After generation, you can directly verify model availability in the terminal.

The demo also featured an "invalid token" incident—caused by an incomplete key copy. After re-copying the correct key, verification passed smoothly, reminding us to pay close attention to accuracy in real operations.

Step 5: Model Mapping and Invocation

Finally, import models via methods like AutoModel and run tests. All models are mapped uniformly, with AutoModel ultimately routing to the specific underlying model. Just send a "1+1" to see the output, and the logs will record complete invocation information and model mapping relationships for easy tracking and troubleshooting.

The Value and Insights of This Solution

From this case, we can see several core values of the Skills paradigm:

Reducing configuration costs. In the past, each tool required its own set of model configurations; now, you only need to maintain a single unified API gateway station, managed automatically by the AI agent. In team collaboration scenarios, everyone shares the same set of channel resources, with the AI handling allocation and maintenance—saving a great deal of repetitive work.

Improving service reliability. Through health checks and automatic degradation-recovery mechanisms, failed channels are automatically blocked, effectively guaranteeing overall service availability. Health checks and automatic degradation are classic patterns in distributed system reliability engineering, whose underlying logic can be traced back to Netflix's open-source Hystrix circuit breaker framework—a circuit breaker typically has three states: closed (normal flow), open (blocking requests), and half-open (probing for recovery), automatically switching states based on the error rate within a time window. This shares the same evolutionary lineage as Alibaba's Sentinel in the Chinese context.

The Unique Challenges of Health Checks for LLM Channels: Traditional microservice circuit breakers (such as Hystrix and Sentinel) mainly target HTTP timeouts and 5xx server errors, with relatively simple error-determination logic. However, the failure modes of LLM channels are far more diverse, going well beyond the coverage of traditional operations experience: in addition to network timeouts and server errors, they include balance exhaustion (402 Payment Required), QPS rate limiting (429 Too Many Requests), specific model versions being retired (404 Not Found), request context exceeding the maximum token limit (400 Bad Request with context_length_exceeded), and security review interception (403 Forbidden)—all failure modes unique to LLMs. This requires health check logic to perform semantic-level secondary parsing of HTTP status codes and the error.code field in the response body, precisely distinguishing "transient failures" (such as network jitter and brief rate limiting) from "permanent failures" (such as balance exhaustion and model retirement), and designing differentiated recovery strategies for each error type—the former suited for exponential backoff retries, and the latter requiring automatic channel switching or manual intervention alerts. Encapsulating this fine-grained failure detection and recovery mechanism as a standard tool callable by Skills is a typical practice in AI-driven operations (AIOps), transforming complex operational capabilities that once required senior DevOps engineers to manually configure and maintain into standardized skills that any developer can trigger with natural language.

Opening up development thinking. The real significance of this case lies in its demonstration: Skills can encapsulate any operations work with clear rules and fixed processes into capabilities that AI can directly invoke. Whether it's channel management, code standard checks, or automated deployment, this approach can be applied to implement them.

The Boundaries and Future Evolution of the Skills Paradigm: It is worth pointing out that the Skills paradigm is not a cure-all—it excels within task domains that have "clear rules and fixed processes," but still has limitations in tasks requiring cross-domain creative reasoning. As LLM capabilities continue to improve, the boundary between Skills and LLMs will dynamically shift: some deterministic operations currently handled by scripts may gradually be returned to LLM reasoning as model reliability improves, while newly emerging complex business scenarios will drive the accumulation of new Skills. This collaborative model of "humans defining deterministic boundaries, LLMs handling uncertain understanding" may represent the optimal solution for AI engineering for quite some time to come—it neither blindly trusts every LLM output nor avoids the LLM's unique advantages in natural language understanding, but instead finds the most reasonable division of labor between the two.

Xiaofu Ge said the next session will guide everyone in developing such a Skills setup from scratch. For developers eager to dive deep into the new AI programming paradigm, this is an excellent hands-on entry point: starting from a real pain point, using Skills to teach AI to "manage its own food supply."

Key Takeaways