Skills as Programs: A New Paradigm for LLM Application Development

An Underrated Analogy

In a recent YouTube conversation, a guest offered a deceptively simple yet remarkably penetrating metaphor: skills are the programs of LLMs. In other words, skills are to large models what code is to computers—the key abstraction layer that makes model capabilities structured, reusable, and composable.

The power of this analogy lies in how it shifts the discussion from "what the model can do" back to "how we organize the model to get things done." We're used to treating LLMs as a universal function to be called—feed in a prompt, wait for output. But once the concept of "skills" enters the picture, an application is no longer a patchwork of individual prompts. It becomes a set of skill modules with clear boundaries that can be iterated independently. This shift in perspective directly determines the direction of subsequent engineering practice.

Skill Orchestration: From Solo Work to System Collaboration

Once an application is decomposed into multiple skills, the next question follows immediately: how do these skills collaborate? The answer offered in the conversation is "orchestration." The guest envisions a purpose-built message board designed to efficiently support orchestration and communication among multiple agents.

The message board here is not just a simple message queue—it's a shared space that carries collaboration protocols. Different skills exchange state, pass tasks, and coordinate execution order through it. This design pulls orchestration logic out of hardcoded implementations and turns it into observable, debuggable system behavior. For complex applications, this is far more reliable than cramming all the logic into one giant prompt.

Message Board in agent systems refers to a shared, asynchronous communication medium where multiple agents (or skill modules) collaborate by publishing and subscribing to messages on different "channels" or "topics." It draws on the classic idea of the Blackboard System: modules don't call each other directly but instead read and write state around a shared data structure, with a control mechanism deciding when to trigger which module. Compared to point-to-point direct messaging, a message board decouples senders from receivers, naturally supporting dynamic joining, task broadcasting, and result aggregation—making orchestration logic easier to observe and debug.

Let Experiments Speak: A Mechanism for Selecting the Best Orchestration

A further insight is that orchestration approaches themselves don't have to be decided by gut feeling—they can be tested. The guest proposes running separate experiments for different types of orchestration schemes and seeing which one performs best.

This idea brings A/B testing and performance comparison thinking from software engineering into agent systems. Orchestration strategies—such as serial, parallel, competitive, voting, and routing—each have their applicable scenarios and costs. Rather than debating which is better, it's better to run the same set of skills under different orchestrations and let the data speak. This "experimentability" is an extra dividend of a skill-based architecture: because skills are independent and reusable, the cost of swapping orchestration approaches is far lower than refactoring the entire application.

A/B Testing and Orchestration Strategy Comparison: In agent systems, the performance differences among orchestration strategies (such as serial, parallel, competitive, voting, routing) often depend on the specific task distribution and are hard to judge a priori. A/B testing randomly assigns traffic or task instances to different strategies and compares metrics such as success rate, latency, and cost, enabling data-driven selection. Unlike traditional unit testing, the "experiments" here require running complete task flows in a real or simulated multi-skill collaboration environment, which demands that skills have reproducible interfaces and observable execution traces—precisely why a skill-based architecture lowers the cost of experimentation.

A Community-Driven Testing Ground

The most vital part of this vision is that it isn't confined to a single team. The conversation explicitly mentions that these experiments come not only from within but also from the broader community, ultimately forming a hub for creative experimentation.

Once a platform accumulates a large number of skills and orchestration schemes, it ceases to be a product and becomes an ecosystem. Developers can reuse skills validated by others and contribute orchestration patterns they've discovered themselves. The cost of experimentation is spread thin, and the pace of innovation is amplified. The guest's assessment is direct: "That's a very genius idea."

Implications for Developers

Treating skills as programs of LLMs means we need to rethink several things. First, skills should have clear interface contracts with predictable inputs and outputs so they can be orchestrated and reused. Second, the orchestration layer needs independent observability—otherwise failures in multi-skill collaboration are hard to pinpoint. Third, experimental infrastructure should be put in place early; don't wait until the system becomes complex to start running comparative tests.

This analogy isn't perfect—LLM "programs" carry probabilistic and uncertain characteristics, unlike the determinism of traditional code. But it's precisely this imperfection that makes skill design, orchestration strategies, and experimental methods more worthy of research. When the community treats skills and orchestration as shareable assets to be accumulated, LLM application development will move from craft workshop to true engineering.