Inside Claude Code's Main Loop: How QueryEngine Drives the Agent with a State Machine

Title

Inside Claude Code's Main Loop: How QueryEngine Drives the Agent with a State Machine

Meta Description

A deep dive into Claude Code's queryLoop main loop: how it drives the agent as a state machine, using seven transition reasons and over a dozen terminal states to unify approvals, hooks, parallel scheduling, and recovery—treating retries as sibling state transitions rather than conventional exception handling.

Article Content

Claude Code's agent main loop is often imagined as a five-line while (has_tool_use) pseudocode snippet. But cracking open the source reveals it's actually an explicit state machine: before each model call it must decide which path to take, and after the call it must decide where to jump next. Retries, auto-compaction, and Stop hook vetoes are all first-class branches of this machine—not after-the-fact patches.

This is the fifth article in the Claude Code Agent Loop series. Based on a source read of the roughly 1,700-line queryLoop in src/query.ts, it places the mechanisms covered in the first four articles—tool declarations, approvals, hooks, parallel scheduling, stop_reason handling, and recovery—into a single skeleton.

First, Correct a Naming Misconception

src/QueryEngine.ts is a file of about 45 KB. Judging by its name alone, it's easy to assume it implements the query engine's main loop.

That's not the case. QueryEngine is an SDK adaptation layer that provides a stable interface for SDK and non-interactive CLI consumers; its submitMessage() is an async generator that exposes loop events to the caller piece by piece. The real main loop is queryLoop in src/query.ts, a function of over 1,700 lines.

This naming is most likely historical baggage: the main loop was already called queryLoop before QueryEngine was introduced for SDK integration. File size is a misleading cue here—the seemingly smaller function actually carries the more complex control flow.

The Shape of the State Machine: Seven Transition Reasons

The high-level structure of the main loop is extremely plain:

while (true) {
    Decide what to do this round based on state.transition.reason
    Execute: call the LLM, process the stream, append messages, run tools...
    Produce a new state.transition, deciding the next round
}

The key is the state.transition.reason union type, which contains seven values:

• next_turn: Normal progression—call the model and process its output.
• collapse_drain_retry: Context is full—perform aggressive context collapsing, then retry.
• reactive_compact_retry: The API returns prompt_too_long—compact the conversation, then retry.
• max_output_tokens_escalate: Output hits the token limit—raise max_tokens, then retry.
• max_output_tokens_recovery: The raised limit is hit again—inject a continuation message, then retry.
• stop_hook_blocking: A Stop hook blocked completion, forcing another round.
• token_budget_continuation: The output token budget mode continues after crossing 500K.

At the start of each round the reason is checked and the corresponding path chosen; at the end of each round, a transition for the next round is produced based on the current work result: normal progression, a recovery branch, or a terminal result. The so-called state machine is not while (has_tool_use), but rather continue as long as the transition is not terminal.

What a Single Iteration Actually Does

One iteration equals one API call plus a batch of tool processing, roughly in this order:

1. Build the request: assemble messages, load tools, prepare the system prompt.
2. Call the LLM: consume the streaming response.
3. Evaluate the result: check whether the content contains tool_use, and handle anomalous stop reasons.
4. Run tools: go through approvals, hooks, and parallel scheduling.
5. Choose the next transition: update state.transition.reason.

Every round goes through this entire chain. The difficulty isn't in any single step, but in carrying the correct state from one round to the next.

stop_reason is a field returned by the LLM API in each response, explaining why the model stopped generating. Common values in the Anthropic Messages API include: end_turn (natural finish), max_tokens (output limit reached), tool_use (requesting a tool call), stop_sequence (custom stop sequence hit), and so on. The main loop doesn't simply forward it—it translates it into the next action decision: tool_use means entering the tool-execution branch; max_tokens triggers max_output_tokens_escalate or max_output_tokens_recovery; prompt_too_long maps to reactive_compact_retry. In other words, stop_reason is the translation layer between model-side signals and loop-side transitions, and understanding it is key to understanding the recovery branches.

Terminal States: Why the Loop Stops

Transitions describe how the loop continues; a separate set of terminal states describes why it stops, including:

• completed: Normal finish—the model no longer produces tool calls.
• max_turns: The maxTurns circuit breaker is hit.
• aborted_tools / aborted_streaming: The user interrupted during tool execution or model streaming, respectively.
• hook_stopped: A post-tool hook blocked progression.
• stop_hook_prevented: A Stop hook blocked completion until retry quota was exhausted.
• blocking_limit, image_error / model_error, prompt_too_long, etc.

There are over a dozen terminal variants, each carrying a specific termination reason, allowing SDK consumers to give appropriate feedback rather than treating every ending as the same generic failure.

Stop hook is a category of callback in the Claude Code hooks system, triggered when the model indicates "this round of work is complete." It gives external scripts a veto: the hook can return a blocking signal, forcing the loop not into a terminal state but into another round, letting the model continue correcting or filling gaps. In contrast, PreToolUse / PostToolUse hooks intercept before and after tool execution, respectively. Together, these three hooks form the policy injection points inside the loop—approval scripts, audit logs, and formatting tools can all intervene through them. A Stop hook veto is not an error but a formal state-transition reason, stop_hook_blocking, sitting at the same level as the retry branches, and therefore has explicit quota control and won't loop indefinitely.

Recovery Is a State Transition Inside the Loop, Not a Nested try/catch

This aligns with the design principle from the third article in the series: turn problems into data or state the system can continue processing, rather than letting exceptions tear down the loop.

Tool failures become error tool_results for the model; potentially recoverable failures in the main loop become the next round's transition.

Naive recovery designs are usually nested:

try:
    call_llm()
except PromptTooLong:
    compact()
    call_llm()  # nested retry
except MaxTokens:
    escalate()
    call_llm()

Claude Code doesn't retry immediately within the current iteration. It records the next action, returns to the top of the loop, and lets the next round dispatch from that state: the current iteration ends with a recoverable failure → state.transition.reason = 'reactive_compact_retry' → the next round starts from the top of the while → seeing that reason, it performs compaction → within that round it calls the LLM → then emits the next transition.

The core insight is: recovery is a first-class state transition, not an appendage of exception handling.

Why It Matters: Recovery Can Chain and Stack

After a successful reactive compact, the next model call may hit max_tokens, and the transition becomes max_output_tokens_escalate; if the raised limit is hit again, it becomes max_output_tokens_recovery, injecting a continuation message. These paths are all sibling branches within the same while loop. With nested try/except, each layer of recovery would require another nested branch or complex re-entry rules, whereas the state machine keeps control flow flat.

Context compaction is a core mechanism in long-conversation scenarios. When the message history approaches the model's context window, a direct call will be rejected by the API because the total token count exceeds the limit, returning prompt_too_long. Compaction works by using an extra model call to summarize the earlier conversation into a shorter message, replacing the original segment, thereby squeezing the total length back within the window. Claude Code distinguishes two compaction paths: collapse_drain_retry is proactive collapsing, done early when context is nearly full; reactive_compact_retry is passive compaction, done only after the API explicitly errors. Both are state transitions rather than exception handling, so a compaction failure itself can become the next round's transition, forming an observable, testable recovery chain.

Why It Matters: Transitions Can Be Asserted in Tests

The source explicitly treats state.transition as a test assertion surface. After completing a unit of work, a test can directly assert state.transition.reason === 'reactive_compact_retry', which is far more stable than grepping an error message out of the output stream.

Hiding Recoverable Errors from SDK Consumers

This design has a deeper intent: while attempting recovery, the loop hides intermediate errors from the caller.

Some SDK consumers (for example, product shells built around agents) treat any API event with an error field as a final failure and immediately show it to the user. Claude Code keeps recoverable errors internal: on receiving prompt_too_long, it doesn't report immediately but first attempts compaction; if both compaction and the next call succeed, then from the SDK caller's perspective, that transient error never happened. Only after recovery completely fails does the loop emit a terminal error.

This is the significance of understanding the loop as a recovery engine rather than a mere error handler: an error is an internal signal that may trigger the next state transition, and not necessarily an external outcome.

Main Agent and Subagents Share queryLoop

Another surprising design: the main agent and subagents reuse the same queryLoop implementation. They each run independent loop instances and do not share the same live state machine; identity is distinguished via toolUseContext.agentId.

The cost is that the loop is littered with checks like if (!toolUseContext.agentId) to guard work that belongs only to the main thread, including memory prefetching, mobile UI summaries, MCP cleanup, and Stop-hook locking.

The benefit is considerable: fix the loop logic once, and the main agent and all subagents benefit at once. Subagent context isolation, worktrees, and sandboxing happen outside the loop in AgentTool.tsx; the loop itself is shared code with only a few identity-dependent branches.

Subagent refers to an independent execution unit spawned by the main agent via AgentTool, used to isolate a subtask into a clean context and drive it forward in parallel. Its key difference from the main agent isn't the loop logic but context isolation: a subagent has its own message history, tool set, and working directory (which can be attached to a worktree or sandbox), so its errors, long context, and tool output don't pollute the main conversation. The loop entry point determines the current identity via toolUseContext.agentId—having an agentId means it's a subagent, and it skips memory prefetching, UI summaries, MCP cleanup, and other work meaningful only to the main thread. This "shared loop code + identity branching" design leaves isolation complexity outside the loop, avoiding a second lifecycle maintained just for subagents.

The Main Loop in Full

Placing the first four articles into a state-machine view, a single iteration can be drawn like this:

─── Iteration start ───────────────────────────────
Dispatch from state.transition.reason:

  next_turn                    → enter normal flow
  reactive_compact_retry       → compact first
  collapse_drain_retry         → collapse context first
  max_output_tokens_escalate   → raise max_tokens first
  max_output_tokens_recovery   → inject a [continue] message first
  stop_hook_blocking           → enter normal flow due to completion being vetoed
  token_budget_continuation    → continue the output budget mode

─── Main flow ─────────────────────────────────
Call the LLM and consume the stream
      ↓
Append the message to the conversation history
      ↓
Does the content contain tool_use?
      ├─ No → check whether stop_reason needs recovery
      │       ├─ Yes → create a recovery transition for the next round
      │       └─ No → run the Stop hook
      │               ├─ Blocked → transition = stop_hook_blocking; continue
      │               └─ Allowed → return { reason: 'completed' }; exit
      └─ Yes → execute tools: approval → PreToolUse hook
              → dispatch in batches by isConcurrencySafe
              → append tool_result and maintain the pairing invariant
              → PostToolUse hook
              (any tool failure becomes an is_error result and never escapes)

─── Iteration end ───────────────────────────────
Update state.transition.reason with this round's result
      ↓
Return to the top of the while

Approvals, hooks, scheduling, completion checks, and recovery are not independent features stitched around the loop; they are a phase or transition within the same state-machine skeleton.

Key Takeaways

• QueryEngine.ts is the SDK adaptation layer; the main loop is queryLoop in src/query.ts.
• The main loop is a state machine with seven transition reasons and over a dozen terminal results.
• Recovery paths are sibling transitions rather than nested retries, so they chain and stack cleanly and are easy to assert in tests.
• Recoverable failures are hidden from SDK callers until recovery is exhausted.
• The loop is a recovery engine: an error can be the signal for the next state transition, rather than an externally visible failure.
• The main agent and subagents reuse the same queryLoop code, running independently and branching on agentId when needed.

The next article will focus on the "call the LLM" step: how server-sent events flow out of the Anthropic API, how event fragments are assembled into a complete message, and how the UI incrementally renders that message.

One-line Summary

Claude Code's agent main loop is an explicit state machine; recovery is a sibling state transition rather than a nested retry.

Paragraph Summary

This is the fifth article in the Claude Code Agent Loop series. Based on a source read of the roughly 1,700-line queryLoop in src/query.ts, it points out that the main loop is not simple while (has_tool_use) pseudocode but an explicit state machine. QueryEngine.ts is merely the SDK adaptation layer; the real main loop lives in query.ts. The loop has seven transition reasons (next_turn, various retries, stop_hook_blocking, token_budget_continuation) and over a dozen terminal results. The core design: recovery is modeled as a sibling state transition rather than a nested try/catch, so it can chain and stack and is easy to assert in tests; recoverable errors are hidden from SDK consumers until recovery is exhausted, making the loop feel more like a recovery engine. Additionally, the main agent and subagents reuse the same queryLoop, distinguishing identity via agentId.