83-Page Report Dissects 11 Coding Agents: Three Findings, One Constant Verified

A Rare Source-Level Breakdown

Wavestone AI Lab has released an 83-page research report conducting a source-level breakdown of 11 production-grade coding agents on the market. The study covers 11 systems: Claude Code, Codex CLI, Gemini CLI, Mistral Vibe, OpenHands, Aider, Mini-SWE-Agent, Hermes, Pi, OpenCode, and OpenClaw^Ad, plus a meta-framework control group called Omnigent.

The research examines approximately 4 million lines of Python, TypeScript, and Rust code, dissecting each system by subsystem and overlaying a 90-day longitudinal diff analysis. This represents the most systematic public review of coding agent internal architecture to date.

One sentence in the paper deserves repeated consideration: "Checklist-class assertions (tool counts, feature matrices, version pins) decay on a weekly basis, while structural assertions (loop taxonomy, subsystem anatomy, absence findings) have so far proven durable." This sentence itself explains the report's value—it focuses not on quickly expiring feature lists, but on architectural-level patterns.

Three Core Findings

Double Absence: No General-Purpose Framework, No Embedded Retrieval

The report's most striking finding is "double absence"—none of the 11 systems use a general-purpose agentic framework, and none embed code retrieval as a core mechanism in the context loop.

Systems actually shipping products chose the "boring" route: writing their own schedulers, managing their own context. This finding suggests that general-purpose frameworks may not be suitable for the coding agent niche—not because they aren't good enough, but because production-grade coding agents have very specific and non-standardized context management needs.

What are called "agentic frameworks" typically refer to libraries like LangChain, AutoGen, and CrewAI that provide general-purpose task orchestration, tool calling, and memory management. Their advantage is out-of-the-box usability; the cost is stacked abstraction layers, elongated debugging chains, and difficulty optimizing for a single scenario. Coding agent workloads have distinct characteristics: context windows are extremely precious, tool calls are highly structured, and failure recovery paths require precise control. In this scenario, general framework abstractions become a burden—developers need to know where every token is spent and why each scheduling step occurs. This is why top-tier systems would rather hand-write a few dozen lines of a while loop than introduce a framework with hundreds of dependencies. This phenomenon is not uncommon in software engineering history: performance-sensitive scenarios ultimately tend to break away from general abstractions and return to metal-adjacent implementations.

Skills Mechanisms Are Winning

9 out of 11 systems have adopted skills/plugin systems, a higher adoption rate than MCP (8 out of 11). One system even published a skill written by the Agent itself. Supply chain governance is becoming a new topic.

This data point is worth noting: Skills mechanism adoption exceeds MCP, indicating that in coding agent scenarios, lightweight skill extension methods are more popular than standardized protocols.

Skills and MCP represent two different extension philosophies. MCP (Model Context Protocol) is a standardized protocol proposed by Anthropic that specifies how models communicate with external tools and data sources. Its strength lies in cross-system interoperability—the same MCP server can theoretically be used by any client supporting the protocol. Skills are more lightweight, typically packaged collections of prompts, scripts, and resources loaded into context on demand, without requiring persistent processes or protocol handshakes. In coding agent scenarios, Skills' advantages are: low startup cost, easy version management and review, and distribution in plain text form. The cost is the lack of unified standards—skill formats across different systems are incompatible. The adoption rate difference (9/11 vs 8/11) is not large, but the direction is noteworthy: developers seem to prefer "good enough" extension methods rather than waiting for a complete ecosystem to mature.

Architectural Convergence Through Imitation

90-day diffs show architecture is converging: later entrants are copying the patterns of forerunners. This is not independent evolution, but conscious imitation. For the entire track, this means the window for architectural differentiation may be narrowing.

Architectural convergence is a recurring pattern in technology history. When a clear dominant design emerges in a field, the focus of competition shifts from architectural exploration to implementation quality, ecosystem, and distribution capability. However, note that similarity at the diff level does not equal similarity in capability: copying a loop structure is easy, but copying the accumulated prompt engineering, tool implementation details, and boundary conditions of failure handling is extremely difficult. The forerunner's moat may shift from "knowing what architecture to use" to "knowing why every parameter in the architecture has its current value."

Hands-On Verification: The Death of a Constant

The report cites a Claude Code constant: AUTOCOMPACT_BUFFER_TOKENS = 13,000. The paper's own table includes a warning: the shipping binary version may already differ.

The author conducted actual verification—pulling the 2.1.206 version binary referenced in the paper from npm, then comparing it with the currently installed 2.1.296 version:

grep -aow 'AUTOCOMPACT_BUFFER_TOKENS' <binary>   # 0 hits
grep -aow 'POST_COMPACT_TOKEN_BUDGET'  <binary>   # 0 hits

Two versions, two constants, all zero hits. But the binary contains a renamed constant family: autoCompactWindow, compact_boundary, PreCompact/PostCompact hooks. The old names have decayed, but the compaction mechanism they described continues under new names.

This verification precisely confirms what the paper itself wrote—checklist-class assertions expire quickly. The paper foresaw this problem, and actual verification reproduced the lesson.

Why Would a Consulting Firm Publish This Research?

The author raises a question most interpretations overlook: why would a consulting firm like Wavestone invest resources in this research?

By observing public information, a picture can be assembled:

• Product layer: Maintaining a public agentic-coding skills repository
• Client layer: Homepage showcasing Sanofi × Claude Code case study
• Recruitment layer: Recruiting Agentic engineers and consultants in multiple countries
• Financial layer: Quarterly revenue down 2%, adjusted annual targets, and announced acquisition of an AI/data team
• Partnership layer: Conversational AI platform partnership, Microsoft Copilot specialization certification

This constitutes a typical consulting client acquisition funnel: research builds credibility → "maturity assessment/roadmap" becomes a standardized product → custom projects at the top. The paper's role is to shorten the time required to be trusted.

The author's judgment: AI won't kill consulting, but will change its form. The bottom of the pyramid—research, slides, first drafts, the three cheapest AI tasks—will thin out. What remains is the signature of judgment and accountability: models cannot be held accountable, cannot be fired, cannot join the board. In an environment of information abundance, "having a specific person responsible for this" becomes more valuable.

Caveats Before Citing

Readers of this report should note several points:

Benchmark scores not independently verified. Evaluation numbers appearing in the paper come from system vendors themselves and cannot be rerun locally.

Scale and methodology numbers come from the paper itself. "Approximately 4 million lines," "90-day diff," "29 patterns," "13 cross-domain observations"—these numbers have not been independently verified.

No public code or data artifacts. For a work selling source code research as its value proposition, this is a reproducibility gap. Structural findings don't depend on line-by-line reproduction, but detailed counts do.

Snapshot discipline. Everything cited carries a date. A paper whose cited constant was renamed within months is itself the best argument for this rule.

"Snapshot discipline" refers to the practice that when studying rapidly iterating software systems, all observational conclusions must carry explicit timestamps and version numbers. Because coding agent binaries update on a weekly basis, a constant name or function structure that is correct today may not exist next month. The standard citation method is to simultaneously note the system name, version number, capture date, and data source (such as npm package hash or git commit), otherwise conclusions cannot be reproduced or refuted by others. This discipline applies not only to citing external research but also to internal team technical decision documents—writing "current Claude Code behavior is such-and-such" into a design document without noting the version is almost certain to become obsolete in short order.