Why CodeAct Code-First Agents Haven't Won Yet: A Deep Dive into the Paradigm's Dilemma

A More Elegant Paradigm That Hasn't Won

In 2024, Wang et al. published the CodeAct paper proposing a seemingly simple yet quite disruptive idea: instead of having LLMs output JSON-formatted tool calls, why not let them directly generate executable code as their actions? Under this paradigm, tools are no longer individually invoked APIs but functions you can call directly in code.

CodeAct's core innovation lies in redefining the "action space" of LLM Agents. In the traditional ReAct framework, the model's actions are restricted to selecting a tool from a predefined set and filling in parameters — essentially a discrete, constrained decision space. CodeAct expands this action space to encompass the full expressive power of a programming language, allowing the model to perform variable assignments, conditional logic, loop iterations, exception handling, and other complex operations in a single action. Experimental data from the paper shows that on multi-step tasks, CodeAct reduces interaction rounds by approximately 30% compared to traditional JSON tool-calling approaches, while significantly outperforming baselines on tasks requiring data processing and logic composition.

The benefits of this approach are obvious: code naturally supports nested calls, loops, and far more expressive logical structures. More critically, intermediate data doesn't need to pass through the context window — it can remain in the execution environment rather than being repeatedly stuffed into the conversation history as text.

However, two years later, a Reddit developer posed a pointed question: virtually every agent framework actually being used in practice is still "chat-first," ReAct-based, and built on JSON tool calls. Conversation history goes in, tool calls come out, execute, submit results, repeat. Even where CodeAct-style code execution has made inroads (e.g., Microsoft's Agent Framework offering a code-act provider), it's typically bolted onto a conversational agent as an execute_code tool. Bash has become a key tool, but usually just one among many.

Why the ReAct Chat-First Framework Won the Market First

ReAct (Reasoning + Acting), proposed by Yao et al. in 2022, has the model alternate between reasoning (Thought) and acting (Action), observing action results (Observation) to form a closed loop. In practical engineering implementations, ReAct typically relies on structured tool-calling protocols: the model outputs a JSON object containing a tool name and parameters, the framework layer parses it and routes to the corresponding tool executor, and execution results are injected as text into the conversation history for the model's next round of reference. The engineering advantage of this pattern is its determinism, ease of debugging, and monitorability — but the cost is that every step requires a full model inference cycle, even if simple data passing still goes through the serialize-deserialize-reinject-into-context process.

This Reddit developer's analysis is quite apt. After ChatGPT's massive success in late 2022, the industry's natural next step during 2023-2024 was adding reasoning capabilities and tool-calling to models. "Chat-first" was the path of least resistance — it extended the existing conversational interaction pattern with minimal engineering overhaul.

In other words, chat-first won not because it was technically superior, but because it had a head start on the timeline and aligned with the inertia of the entire ecosystem's evolution. This is a classic path dependency problem.

Institutional Friction: How the Code-First Paradigm Lost to "First Mover"

The author's core thesis for why CodeAct hasn't become mainstream is: Institutional Friction. Even though we're only a few years into the LLM wave, this friction is already powerful enough. He lists several progressively deeper reasons:

Model Training Is Biased Toward Chat-Style Tool Calling

Massive RLHF investment has gone into training models to output well-formatted, structured tool calls. RLHF (Reinforcement Learning from Human Feedback) is one of the core techniques for aligning LLM behavior. In tool-calling scenarios, models need to learn when to call tools, select the right tool, and generate strictly formatted parameter JSON. Training this capability involves enormous carefully constructed training data: manually annotated tool-call examples, synthetic multi-step tool-use trajectories, and reward signals targeting format correctness. The investment from companies like OpenAI and Anthropic in this direction is enormous — it's estimated that GPT-4's tool-calling capability alone went through thousands of hours of human annotation and multiple optimization iterations.

This means a structurally superior paradigm can easily lose to a structurally inferior one backed by tens of thousands of hours of fine-tuning. This is the most fundamental point — it's not that the paradigm doesn't work, it's that the model's "muscle memory" has been trained into a different shape. Changing the output paradigm means massive training infrastructure and data pipelines need to be redesigned, and the inertia created by this investment is extremely difficult to reverse.

Communication Protocols Themselves Assume Chat-First Architecture

The request structure of mainstream APIs — messages: [...] plus a tools: [...] schema list — is inherently chat-first. This assumption is encoded directly at the protocol layer, not something the application layer can easily work around. From OpenAI's Chat Completions API to Anthropic's Messages API, the top-level structure of request bodies presupposes a "conversation history + available tools list" paradigm, and model outputs are forced into categories of "assistant message" or "tool_calls" — there simply isn't a semantic slot for "a program that needs to be executed" in this type system.

Design Limitations of the MCP Protocol

This is one of the author's sharpest observations. Model Context Protocol (MCP), released by Anthropic in late 2024, aims to standardize how LLMs interact with external tools and data sources. Its technical architecture is based on JSON-RPC 2.0, defining three core primitives: Tools (callable functions), Resources (readable data sources), and Prompts (reusable prompt templates). MCP's design philosophy is "servers expose capabilities, clients (models) invoke on demand," with each interaction being a complete request-response round.

MCP invokes typed tools one at a time via JSON-RPC, making it essentially a turn-based tool selection protocol. What the code-first paradigm wants is something closer to an "importable module" — more importantly, it needs data handles rather than data payloads, so intermediate results don't materialize as text in the context. A data handle is a reference pointing to a remote object, allowing lazy loading and incremental operations; a data payload requires complete data serialization on every transfer. From this perspective, MCP is fundamentally "chat-shaped" — it naturally favors discrete, turn-based interaction patterns, while the code-first paradigm's need to "continuously access objects within an execution context, maintain state, and perform streaming data processing" lacks native support in MCP's current architecture.

Toolchains and Evaluation Systems Depend on Discrete Calls

All downstream infrastructure — tracing, evals, monitoring — is built around "which tool was called with what parameters." Mainstream Agent observability platforms like LangSmith, Braintrust, and Arize have "a single tool call" as the core unit of their data models: including call time, tool name, input parameters, output results, latency, and cost. Switching to code-first means this entire observability system needs to be rebuilt from scratch — you no longer need "what tool was called" but rather "what code was executed, which line produced side effects, how did runtime state change." This is an entirely different monitoring paradigm, closer to traditional software APM (Application Performance Monitoring) than current Agent trace systems.

Security Sandboxing Is an Unavoidable Challenge

Code-first means you must build a sandbox to run code, carefully decide which functions are allowed, and handle authentication without directly managing credentials. Current industry sandbox solutions include: container-based isolation (e.g., Docker/gVisor), WebAssembly-based lightweight sandboxes (e.g., Wasmer), and VM-based strong isolation (e.g., Firecracker microVM). Each approach has different trade-offs between security, startup latency, and resource overhead.

The author's bet is on a "sandbox + egress proxy" combination. An egress proxy is a network-layer security control mechanism — code inside the sandbox doesn't directly hold API keys or database credentials but makes external requests through a controlled proxy layer, which handles injecting authentication information, enforcing access control policies, and audit logging. This pattern is more secure than directly exposing credentials inside the sandbox but adds architectural complexity and request latency. He also notes that most agents already have a bash tool, and running code written by the agent in real-time is already a reality. Platforms like E2B and Modal now provide out-of-the-box code sandbox services, partially lowering this barrier.

Reasoning Models Expose Cracks in the Old Architecture

The author shared a very specific war story that illustrates the problem well. He recently tried switching to a reasoning model, and everything broke.

Reasoning models (such as OpenAI's o1/o3 series, DeepSeek-R1, etc.) differ fundamentally from traditional chat models in that they contain an extended "thinking" phase. During this phase, the model generates large amounts of intermediate reasoning tokens (usually invisible to users) before outputting a final answer. This architecture poses new challenges for tool-calling pipelines: during reasoning, the model may need to "hypothetically" consider tool calls multiple times, and its internal state management is fundamentally different from traditional single-turn generation. Additionally, reasoning model outputs tend to be longer and more structured, with different requirements for streaming handling logic.

What broke wasn't parsing the reasoning content but receiving 400 errors from the provider — the server strictly validates that tool_calls[].function.arguments must be valid JSON, and a single malformed message "poisons" the entire conversation history, causing every subsequent request to return 400. The root cause is that reasoning models may produce intermediate-state invalid JSON fragments during their internal reasoning process when generating tool-call parameters, and existing API validation logic never anticipated this situation.

What's interesting is: this isn't a bug specific to one model, but a gap across the entire class of reasoning models. It exists precisely because the entire tool-calling/streaming code path was written under the assumption of "chat-first, non-reasoning models." When the new generation of reasoning models appeared, old assumptions started leaking. This perfectly validates the reality of institutional friction.

The Future of Code-First Agents: Unresolved Questions

The author candidly shares that he's been experimenting with a truly code-first framework. Beyond the benefits argued in the paper, what he values most is: it provides enormous freedom to explore UI forms beyond chat threads. He mentions that even Claude Routines are essentially just chat threads with scheduled triggers.

This touches on a deeper issue: when an Agent's actions are code rather than tool calls, the execution process naturally has "program" characteristics — with explicit control flow, the ability to be paused and resumed, and concurrent branches. This means the Agent's interaction interface doesn't have to be limited to conversational message-by-message display but could be presented as workflow visualizations, real-time code execution panels, or even interactive environments similar to Jupyter Notebooks. This degree of UI freedom is something chat-first architecture struggles to provide.

He poses two questions he's genuinely seeking answers to, worthy of the entire community's consideration:

• Has anyone actually gotten a code-first system running in production? What were the benefits?
• Is the judgment that MCP is fundamentally chat-shaped wrong? Have recent updates changed anything?

The Gap Between Technical Superiority and Ecosystem Victory

The value of this discussion lies not in providing conclusions but in clearly dissecting the gap between "technical superiority" and "ecosystem victory." This is not unprecedented in technology history — Betamax vs. VHS, LISP vs. C/C++, Plan 9 vs. Unix are all classic cases of technically more elegant solutions losing to more mature ecosystems. The CodeAct story reminds us: in the rapidly evolving AI field, first-mover advantage, training investment, protocol standards, and observability toolchains together create powerful path lock-in. Even an architecturally more elegant paradigm needs to clear four hurdles — model training, communication protocols, security sandboxing, and developer tooling — before it can truly shake the existing landscape.

Code-first agents may not have won yet, but the author's "(yet)" preserves the suspense — as reasoning models expose cracks in old assumptions, this paradigm battle is far from settled. It's worth noting that many technology paradigms that eventually prevailed (such as containerization over VMs, REST over SOAP) often underwent rapid flipping only after foundational conditions matured to a critical point. For code-first agents, this tipping point may depend on several variables: the standardization of sandbox infrastructure, the degree of models' native optimization for code generation, and whether a sufficiently compelling killer application emerges to demonstrate this paradigm's overwhelming advantage in specific scenarios.

Key Takeaways