Distilling Finite-State Machines from Agent Trajectories: Is Behavior Driven by the Model or the Framework?

Who Really Determines Agent Behavior: The Model or the Framework?

When evaluating an AI Agent's performance, we instinctively credit — or blame — the underlying large language model (LLM). But a new study poses a sharp question: how much of your Agent's behavior actually comes from the model itself, versus the "harness" you've wrapped around it?

A harness refers to the execution framework built around the LLM — prompt engineering, tool-calling logic, state management, memory mechanisms, loop control, and the broader engineering scaffold. The study's central finding is surprising: an Agent's behavioral topology is shaped more by the deployment framework than by the underlying LLM.

This suggests we may need to recalibrate where we focus optimization efforts. Rather than repeatedly swapping or fine-tuning foundation models, it may be more productive to take a closer look at the surrounding engineering design.

Core Methodology: Compressing Trajectories into a State Machine

From Trajectories to Finite-State Machines (FSMs)

The key technical contribution is "collapsing" an entire corpus of Agent execution trajectories into a compact Finite-State Machine (FSM). The idea draws from automata theory — each Agent execution trace is fundamentally a sequence of state transitions, and across many trajectories, large amounts of repeated and shared behavioral patterns emerge.

Using automata induction techniques, the researchers distilled thousands of trajectories into a machine with very few states. Experiments across twelve public datasets show that the induced state machines contain only 7 to 43 states. In other words, what appears to be complex and variable Agent behavior can be described by a remarkably small number of underlying states.

Fidelity and Stability Metrics

The quality metrics for these state machines are equally noteworthy:

• Replay fitness reaches 0.997: When replayed on held-out data, the FSM nearly perfectly reproduces real trajectories.
• Topological structure is nearly identical across data splits: Regardless of how the dataset is partitioned, the induced state machine topology remains highly consistent — indicating it captures stable structural patterns rather than overfitted noise.
• Construction takes only milliseconds: The entire induction process is extremely efficient, imposing almost no computational overhead.

Taken together, these three points form a compelling chain of evidence: Agent behavior does have a stable, compressible, and reproducible underlying structure.

Practical Value: Prediction, Monitoring, and Early Stopping

This finite-state machine is not merely a theoretical analysis tool — it demonstrates clear engineering value across multiple real-world tasks.

Next-Step Prediction: Outperforming Existing Methods

The researchers used FSM states as context for a next-step prediction task. The results outperformed Agent Workflow Memory (AWM) on every dataset with ground-truth matches. This suggests that using the current "state" as an explicit signal is more effective at predicting an Agent's next action than relying on workflow memory alone.

Failure Prediction: AUROC Up to 0.94

The team designed per-state behavioral features for each state and used them to predict whether a run would fail. On held-out sets, failure prediction achieved an AUROC as high as 0.94 — meaning that state-level behavioral features alone can distinguish successful from failed runs with high accuracy.

Online Monitoring and Early Stopping

Perhaps the most deployment-ready capability is online monitoring. The researchers built an online monitor that can rank failing runs ahead of successful ones from partial traces alone, enabling early stopping before a task completes.

For cost-sensitive Agent deployments, this is enormously valuable: rather than letting a doomed run exhaust its full token and tool-call budget, the system can identify and abort it early. This directly impacts cost control and response efficiency in production environments.

Why the Framework, Not the Model, Drives Behavior

The Inference Behind Structural Stability

The researchers' core argument is: behavioral topology is shaped more by the deployment framework than by the underlying LLM.

The logical foundation rests on the stability of the state machine. If Agent behavior were primarily driven by the LLM's stochastic sampling and internal reasoning, the induced state machines should be more divergent, harder to compress, and inconsistent across data splits. But the experiments show the opposite — very few states, very high fidelity, and consistent topology across splits — all pointing to a system governed by external constraints.

In other words, the harness — through tool interfaces, prompt templates, loop conditions, and other mechanisms — effectively lays down clear "tracks" for the LLM to follow. The LLM makes local decisions at each node, but the overall behavioral skeleton is predetermined by the engineering framework.

Practical Implications for Agent Engineering

This perspective has several direct implications for current Agent development practice:

1. Shift the debugging focus: When an Agent fails repeatedly, auditing the framework logic (tool design, state transitions, termination conditions) may be more effective than switching models.
2. A new path to interpretability: Compressing a black-box Agent into a visualizable finite-state machine offers a structured handle for understanding and auditing Agent behavior.
3. Cross-model transferability: Since behavioral structure is primarily determined by the framework, optimization insights accumulated at the framework level can theoretically be transferred and reused across different underlying models.

A Structural Map for Agent Behavior

The significance of this work lies in offering a methodology for transforming chaotic Agent trajectories into clear, structured representations. The finite-state machine serves simultaneously as an analysis tool, a monitoring engine, and a mirror that illuminates the division of labor between model and framework.

As Agent systems grow more complex and more deeply embedded in production environments, interpretable, predictable, and controllable analytical frameworks like this will become increasingly important. And the deepest reminder this work offers may be: beyond the pursuit of ever more powerful foundation models, we may be severely underestimating the importance of getting the "harness" engineering right.