Multi-Agent Handoff Loss: The Plan Never Reached the Executor

A three-step pipeline built on a local 8B model outperformed a single call on accuracy. But cracking open the logs reveals that the gain did not come from the heavily-hyped planning step — the planner's full notes were truncated the moment they crossed a LangGraph subgraph boundary. The real work was done by the executor and reviewer rereading the original report.

Experiment Setup: A Deliberately Narrow Handoff

This experiment set out to reproduce a specific, often overlooked failure mode in multi-agent systems. The author ran llama3.1:8b locally on Ollama with temperature 0, a fixed seed, and zero API cost, comparing two conditions across 12 synthetic incident-report extraction tasks: Condition A is one call per task, and Condition B is a three-step pipeline of planner, executor, and reviewer, totaling 36 calls.

Each task requires extracting three fields: root_cause, affected_component, and fix_action. An answer is counted correct if it contains any of the required keywords — this mechanical scoring is designed for reproducibility, not human judgment.

The handoff payload boils down to a single line of Python:

focus_hint = plan_notes.split(".")[0][:60] if plan_notes else ""

The author deliberately models the subgraph boundary with plain Python instead of LangGraph's StateGraph: the subgraph's state schema is what actually crosses the parent-child boundary, and any key the parent step produces that the child schema never declared simply doesn't exist on the other side — no exception, no warning. The boundary is designed to be narrow, mirroring what teams actually do under delivery pressure.

The Data Processing Inequality is a classic theorem in information theory: if random variables X → Y → Z form a Markov chain, then mutual information satisfies I(X;Z) ≤ I(Y;Z) ≤ I(X;Y). In plain terms, any processing of data (summarizing, compressing, rewriting) can never conjure up more information about the original source — it can only lose information or leave it unchanged. In a multi-agent pipeline, the "summary" produced by the previous agent is the intermediate variable Y, which has already lost information relative to the original input X; no matter how smart downstream agents are, they cannot recover from Y the details that were dropped from X. This is the mathematical bedrock ensuring that the game of telephone inevitably grows blurrier with every hop — and it explains why having later steps reread the raw material directly is often more reliable than trusting an upstream summary.

LangGraph's StateGraph and subgraph state schemas are key to understanding the "handoff loss" in this article. LangGraph uses StateGraph to explicitly declare a graph's state structure (schema), and nodes exchange data through this shared state dictionary. When a subgraph is embedded in a parent graph, each maintains its own schema, and only keys declared on both sides pass across the boundary. If a parent step produces a key the child schema never declared, that key simply doesn't exist on the subgraph side — no exception is thrown, no warning is logged, and the data vanishes silently. The author deliberately simulates this behavior with a one-line Python string slice, precisely to highlight how hidden this class of boundary failure is in real engineering.

Results: Accuracy Won, Handoff Lost

Put the numbers side by side:

• Field accuracy: single call 30/36 (83.3%), three-step pipeline 33/36 (91.7%)
• Tasks with all three fields correct: 7/12 vs 9/12
• LLM call count: 12 vs 36
• Total tokens: 1,659 vs 7,323 (4.4×)
• Wall-clock time: 72.7s vs 411.1s (5.7×)

But what that comparison table doesn't show is the handoff itself: truncation occurred in all 12 tasks, with planner output ranging from 368 to 584 characters, while the focus_hint that actually reached the executor was capped at 60 characters. On the retry-storm task, the planner's full note said to "check the retry logic, figure out why the system didn't originally implement exponential backoff," but the executor received "Here's a short investigation note: 'Review the system logs" — cut off before "retry storm" or "backoff" ever appeared.

Deterministic runs with a fixed seed and temperature=0 are the basis for reproducibility here, and they also define the boundary of the conclusions. With sampling temperature at 0 and a fixed random seed, the same model produces nearly deterministic, consistent output for the same input, so each condition only needs one run to yield stable results without repeated sampling and averaging. The cost is that this single deterministic run cannot reflect the variance of real inference — a single field flipping would shift roughly 2.8 percentage points in a 12 task × 3 field = 36 unit score. So although the gap between 91.7% and 83.3% is real, it rests on a single sample trajectory and cannot be directly generalized into a statistically meaningful advantage.

Three Flip Cases: Where the Gain Actually Comes From

Three fields went from wrong to right between the two conditions. The author deliberately records the executor's intermediate answer and the reviewer's final answer separately to determine which step truly mattered — and these three cases resolve through different paths.

On the feature-flag-leak task, the planner's hint is pure boilerplate: "Here's a short investigation note: Investigation Note" — all 60 characters are opener, zero task information. Yet before the reviewer even steps in, the executor, reading the full original report, gives the correct answer: "feature-flag targeting rule leak due to shared cache entry reuse."

On the retry-storm task, the hint is half-generic ("Review the system logs"), and the executor's answer is still wrong, the same gap as a single call. Only after the reviewer gets the full report and rereads it does it correct to "downstream outage caused a retry storm." This is the only case in the whole run that clearly required the reviewer.

The third case is an honest complication. On redis-pool, the single call misses both "redis" and "connection pool"; the executor nails it, and this time the 60-character hint happens to contain "Review the Redis connection pool confi" — the planner coincidentally wrote the right keywords before the truncation point. The author admits he can't rule out that the hint played a role here, but stresses this is just a lucky sentence break, not a capability guaranteed by architecture.

Taken together, the planning step earned no reliable contribution this time. Its only possible contribution depends on incidental wording, not on the architecture itself. The executor, independent of the hint and working purely from the full report, already hits 88.9% (32/36) — measured directly from the saved intermediate answers — and the reviewer adds only one more field on top of that, for 33/36.

Why "Rereading the Source" Is Worth More Than "Passing a Summary"

There is a theoretical basis for the author's observation. A Stanford preprint, "Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets," holds the reasoning-token budget constant across three model families and finds that a single agent consistently matches or beats multi-agent setups. The paper's explanation relies on the Data Processing Inequality: once one agent passes a processed version of context to the next, information at that step can only be lost or preserved, never added.

That focus_hint truncation is a concrete, measurable instance of this inequality in action — not an edge case. Anthropic makes the same point about the cost of this game of telephone in its multi-agent systems engineering post: agents typically use about 4× the tokens of a single chat, and a full multi-agent system about 15×, with part of the overhead flowing to processes where output is repeatedly summarized and rewritten. Anthropic's own fix is to have subagents write full output to shared storage rather than pass compressed summaries through conversation.

This experiment adds a data point in the same direction with a mechanism visible in the logs: 91.7% did beat 83.3%, but the gain was bought mainly by having each step reread the original report, not by the planning step in the architecture — and planning is precisely the part that multi-agent writing spends the most ink justifying.

Anthropic's multi-agent engineering practices point to a cost structure aligned with this article: multi-agent systems typically consume about 4× the tokens of a single chat, and full multi-agent architectures can reach 15×. One notable source of overhead is the repeated summarizing and rewriting of outputs between agents — every compression carries information loss and burns extra compute. Anthropic's proposed fix is to have subagents write full output to shared storage (such as a file or database), with downstream agents reading raw content on demand rather than passing compressed summaries through conversational context. This aligns with the observation in this article: making "reread the original report" a routine pipeline action preserves critical information better than relying on a planning summary that has been retold several times.

Is the Planning Step Worth It? First Ask What the Schema Carries

The conclusion this round of experiments supports is narrower than either "multi-agent wins" or "multi-agent is a trap." Both overinterpret a single local run of 12 tasks. What it genuinely supports is just two points: a step that revisits the raw material instead of trusting the previous step's summary can catch real errors — here, three fields' worth — while a planning step whose handoff goes through a too-narrow schema buys nothing reliable; cut it and route straight from report to executor to reviewer, and the summary table would look almost identical.

Before adding a planning step to a pipeline, the question worth asking is not "will more structure help" but "what exactly must survive from this step to the next, and does my state schema actually carry it?" That is a concrete, checkable question, not a feeling. Before adding a reviewer step, the better question is whether the reviewer gets the raw material or the previous step's output — the reviewer's advantage in this run came precisely from rereading the report, not from critiquing the executor's JSON in isolation.

Reproducibility and Honest Gaps

The full script, task set, and per-task raw output are in the public repo github.com/priyansh18/agent-handoff-loss-benchmark, all run locally on Ollama with zero API keys. Two limitations are explicitly flagged: first, each condition runs only one deterministic pass, not a repeated-trial distribution, so a single field flip shifts 2.8 percentage points in a 36-unit score; second, the keyword-based scorer rewards completeness over precision, which partly explains why the other three tasks stayed wrong under both conditions.

The question the author leaves at the end is worth asking anyone who has ever built a planner-executor-reviewer pipeline: have you actually checked what survives the handoff in your planning step, or — like the author almost did — have you simply trusted it?

One-line Summary

A three-step agent pipeline scores higher, but the gain comes from rereading the original report, and the planning step contributes almost nothing due to handoff truncation.

Paragraph Summary

The author uses a local llama3.1:8b on 12 synthetic tasks to compare a single call against a planner-executor-reviewer three-step pipeline. The pipeline's field accuracy is 91.7%, above the single call's 83.3%, but it uses 4.4× the tokens and 5.7× the wall-clock time. The key finding: the planner's 368–584 character notes are truncated to 60 characters at every subgraph state schema boundary, all 12 tasks without exception. Of the three fields that flipped from wrong to right, two were solved by the executor rereading the original report when the hint was sheer boilerplate, and the only fix that depended on the reviewer also came from its second reading. The executor hits 88.9% independent of the hint, so the planning step delivered no reliable gain.