How Can Vertical AI Agents Differentiate? A Three-Layer Strategy and Regular Iteration

In a recent conversation with Harry Stebbings, we explored many topics in the AI space, and the discussion about vertical vs. horizontal agents was particularly thought-provoking. I'm clearly bullish on the prospects of vertical agents and believe differentiation advantages can be built at three layers.

The Three-Layer Differentiation Strategy for Vertical Agents

1. Context: Quickly Build Domain Moats

Context includes prompts, skills, and tools, all tailored to a specific domain. Providing domain-specific context is relatively easy, yet it effectively helps the agent understand industry terminology, business processes, and user intent. For example, a legal-domain agent needs built-in statute retrieval, contract templates, and compliance-checking tools—these contexts form its first line of defense.

2. Harness: Adjusting the Agent's Fundamental Behavior

Compared with context, adjusting the execution harness is more difficult, but it can change the agent's fundamental behavioral patterns. The harness involves how the agent plans tasks, calls tools, handles errors, and manages memory. By optimizing the harness, you can make the agent work more efficiently in specific scenarios—for example, reducing unnecessary tool calls or improving the stability of multi-step reasoning.

Agent Harness is the runtime system that connects the large language model to the outside world, responsible for turning model outputs into concrete actions. It typically includes a task planner (decomposing complex goals into subtasks), tool-calling interfaces (executing API requests, running code, etc.), state and memory management (maintaining conversation history and intermediate results), and error handling and retry mechanisms. Unlike pure prompt engineering, the Harness determines the agent's behavioral boundaries and robustness in uncertain environments. For example, when a tool call fails, the Harness can decide whether to retry, degrade, or ask the user for clarification. Therefore, optimizing the Harness for a vertical domain (e.g., limiting concurrent tool calls, adding domain-specific validation steps) can significantly improve the agent's reliability and efficiency—which is also why customizing it is harder than customizing context.

3. Model: The Deepest Layer of Customization

Adjustments at the model layer are the most difficult, but they clearly change behavior the most. This includes fine-tuning base models or adopting domain-specific pre-trained models. Although costly, model customization can deliver significant performance gains for scenarios that require deep expertise.

Domain-specific pre-trained models refer to models obtained by continued pre-training on top of a general base model using large amounts of domain corpora (e.g., legal documents, medical records, financial reports). Unlike fine-tuning, continued pre-training not only adjusts task-related output layers but also updates all or most of the model's internal parameters, enabling the model to learn the language patterns, terminology distributions, and reasoning logic within the domain. This approach is costly and requires high-quality domain data and significant compute, but it brings deeper domain adaptation and lower inference costs (because lengthy prompts are no longer needed). For vertical agents, when context and Harness optimization hit a ceiling, domain-specific pre-trained models often become the key means to break through performance bottlenecks.

Most Engineers Are Still Stuck at the First Two Layers

From what I've observed, most agent engineers are still tuning context and the execution harness, and usually in that order. This makes sense: context is the easiest to start with, the harness comes next, and model customization has the highest barrier. But it also means that if you can establish an advantage at the model layer first, you may gain a stronger competitive moat.

Regular Iteration: Revisit Every 3–6 Months

Regardless of which layer you're tuning, given the rapid evolution of base models, it's advisable to revisit your context, harness, and model every 3–6 months. Upgrades to base models may render some old customizations redundant; promptly removing parts that are no longer needed (i.e., "cruft") keeps the agent lean and efficient. This habit of regular iteration is key to maintaining the competitiveness of vertical agents.

Conclusion

Differentiation for vertical agents isn't achieved overnight; it requires continuous investment across context, harness, and model, along with a mechanism for regular iteration. For founders, starting with context for rapid validation and gradually going deeper into the harness and model is a viable path.

One-line Summary

Vertical agents can differentiate across three layers—context, harness, and model—and require regular iteration.

Paragraph Summary

Based on a conversation with Harry Stebbings, the article proposes a three-layer differentiation strategy for vertical agents: context customization (prompts, skills, tools) is the easiest way to build a domain moat; the execution harness (planning, tool calls, error handling, memory management) can change the agent's fundamental behavior and is more difficult; model customization (fine-tuning or domain pre-training) is the hardest but has the most significant effect. The author observes that most engineers are still stuck at the first two layers, so establishing an advantage at the model layer first can yield a stronger competitive moat. Because base models evolve rapidly, it's advisable to revisit customizations at each layer every 3–6 months and remove outdated parts (cruft) to keep the agent lean and efficient. The conclusion notes that starting with context for validation and gradually going deeper into the harness and model is a viable path for founders.