Where Does AI Agent Waste Really Come From: Model Calls or the Orchestration Layer?

The Overlooked Question: Not "What Was Called" but "Why Was It Called"

As AI Agents scale into production environments, observability has become an unavoidable topic. Most tools on the market can tell you which APIs an agent called, how many tokens it consumed, and how much it cost. But one Reddit developer, after digging deeper into the problem, raised a sharper question: Why do these calls happen in the first place?

This developer began exploring the space after AUDR was released and built an open-source CLI tool called KORA Doctor, designed specifically to inspect agent execution traces. His core argument: current agent observability is still stuck at the "results layer" and lacks insight into the "motivation layer." Knowing how much you spent is one thing — knowing why that money was spent and whether it could have been saved is an entirely different challenge.

The term observability originates from control theory. In software engineering, it refers to the ability to infer a system's internal state from its external outputs. Traditional application monitoring focuses on predefined metrics and alerts, while observability emphasizes the ability to trace the root cause of unknown failures after the fact — typically relying on three data pillars: logs, metrics, and distributed traces. In the AI Agent context, traces are especially critical: they record the full call chain, latency, and context for every step in a complete agent run, making them the primary means of reconstructing "what the agent was actually doing." Current agent observability tools (such as LangSmith and Langfuse) are reasonably good at capturing trace data — but as this article points out, a significant analytical gap remains between capturing what happened and understanding why it happened.

The Waste Patterns KORA Doctor Targets

The open-source tool currently focuses on several waste patterns that repeatedly appear in real-world agent runs:

• Redundant model calls: The same reasoning is triggered repeatedly with no caching or reuse mechanism
• Missing cache and reuse: Requests that could hit a cache instead go through the full pipeline again
• Deterministic work delegated to an LLM: Tasks that could be handled directly by rules or code are sent to a large model instead
• Overkill model selection: Expensive, high-end models are used for trivially simple tasks
• Excessive planning and orchestration: The planning and orchestration steps themselves consume a disproportionate share of resources

What these patterns share is that they don't always appear in the most visible place — the model call itself. Much of the waste hides deeper in the agent's execution flow.

The Critical Production Feedback: Waste Starts Before the Model Call

What genuinely shifted the author's thinking was a piece of feedback from a production user. This user, running agents in a live production environment, reported that their biggest source of waste wasn't redundant inference — it was re-fetching the same tool data across multiple steps.

The significance of this finding: waste can be generated before a model call ever happens. The orchestration layer produces unnecessary retrievals, tool calls, and retries — and on top of all that redundancy, you pay another layer of inference costs. In other words, a single unnecessary tool call can trigger a cascade of downstream retrieval and reasoning overhead.

This reframes the problem from pure "LLM cost optimization" into a much broader concern: execution waste across the entire agent run.

The orchestration layer is a core component of AI Agent architecture. It coordinates the execution order of subtasks, decides when to call which tool, handles intermediate results, and determines when to terminate loops. Common orchestration frameworks include LangChain, LlamaIndex, and AutoGen. Because orchestration logic typically lives in code rather than in LLM outputs, traditional token-counting tools are nearly blind to its overhead. Repeated tool data fetching is a particularly telling example: when an agent doesn't cache intermediate results across multi-step reasoning, it re-issues the same API requests or database queries at every step. These calls don't consume tokens, but they generate real latency costs, API fees, and downstream inference redundancy. Waste at this layer can only be identified through full execution trace analysis.

The Shift from "Model Cost" to "Execution Cost"

For a long time, discussions of agent cost optimization have defaulted to "reduce token consumption" or "switch to a cheaper model." But this developer's practical experience suggests that framing may be targeting the wrong thing.

A real waste map needs to cover the entire execution chain — at minimum, the following potential sources:

Model Calls

Direct inference costs. The easiest to monitor and the most likely to be treated as the only optimization target.

Tool Calls & Retrieval

The most hidden and expensive category in production user feedback. Repeatedly fetching the same data, redundant retrieval requests — these silently accumulate into enormous overhead.

Retries & Validation

Failed retries and repeated result validation multiply resource consumption without adding substantive value.

Planning Loops

When an agent falls into excessive planning or inefficient orchestration loops, vast amounts of compute are spent "deciding what to do next" rather than actually doing it.

This shift from "model cost" to "execution cost" means that optimizing agents requires a tool capable of reconstructing the full execution trace — not just a token counter.

Implications for Production Developers

The author has made clear that KORA Doctor is being built around "the problems that actually appear in real traces," which is why he's actively seeking real-world cases from production environments. The question he poses is worth every agent-running team asking themselves: Where does your waste primarily come from — model calls, tool calls, retries, retrieval, planning loops, or redundant validation?

For teams deploying agents in production, this discussion surfaces at least two actionable takeaways:

First, don't reduce cost optimization to swapping models or compressing prompts. If the root cause of waste lives in the orchestration layer, even the cheapest model is just paying for a redundant process.

Second, the value of an observability tool lies in root cause analysis, not just accounting. A tool that can answer "why did this call happen" is the one that can actually guide optimization decisions.

As agent architectures grow more complex, the orchestration layer is quietly becoming a hidden bottleneck for both performance and cost. The emergence of open-source tools like KORA Doctor represents a trend worth watching: pushing agent observability down from the results layer into the execution logic layer.

Related projects: AUDR (https://openaudr.dev/) and KORA Doctor (https://github.com/Krako-Labs/kora-doctor).