Harness Router: A Tool Routing Decision Layer Built for AI Agents

AI coding assistants and agent systems are becoming increasingly capable, but one long-standing pain point has never been properly addressed: as the number of tools grows, how can an LLM accurately select the right one? A developer shared their open-source project Harness Router on Reddit, attempting to answer this question by introducing a dedicated "decision layer."

The Problem: Why LLMs Make Wrong Tool Choices

In a typical agent architecture, the model "freely picks" from a large pool of available tools. This works fine when the toolset is small — but once the number of tools grows, especially when tools have similar names or overlapping functionality, the model's selections become unreliable.

The author highlights a particularly representative example: tools like get_x, fetch_x, and read_x, whose semantics are highly similar. For humans, the differences may be obvious, but for an LLM, this ambiguity directly leads to incorrect calls. On top of that, stuffing all tool descriptions into the context introduces two hidden costs: higher token consumption and wasted inference compute on "obviously straightforward routing decisions."

The Solution: Insert a Routing Layer Before Execution

The core idea behind Harness Router is straightforward: don't let the LLM blindly choose from a large pool of tools — instead, insert a decision layer before actual execution.

This approach essentially decouples "tool selection" from the model's free-form reasoning, handing it off to a more controlled and interpretable routing component. The design is framework-agnostic, meaning it isn't tied to any specific agent framework and can theoretically integrate with a variety of agent harnesses.

The latest version adds support for Codex hooks, breaking the overall flow into several clear phases:

• SessionStart: Discovers all available tools once at the beginning of a session
• PreToolUse: Inspects every tool call before it happens

The decision logic in the PreToolUse phase is particularly noteworthy, reflecting the author's thinking around "safe fallback":

• Same selection → Allow (pass through directly)
• A better alternative exists → Trigger re-planning
• Router itself fails → Apply a fail-open policy — do not block execution

The fail-open design is critical. It ensures that even if the routing layer encounters a problem, the entire agent won't get stuck — it simply falls back to the original execution path. This is a pragmatic trade-off for production stability.

Codex hooks are a lifecycle hook mechanism provided by OpenAI Codex that allows developers to inject custom logic at specific points in the agent's execution flow, without modifying the core agent code. This is similar to middleware or event hooks in web frameworks — you can attach your own processing logic "before a request comes in" or "before a tool is called."

Harness Router uses this mechanism to transparently intercept tool calls: developers don't need to change the agent's main logic; the routing layer silently steps in as a "side-channel check." The SessionStart hook handles tool registration and discovery during session initialization — essentially a warm-up phase — while the PreToolUse hook performs routing decisions before each tool call fires, determining whether to allow it, swap in a better tool, or trigger a deeper MCTS search. This hook-based design makes the routing layer nearly transparent to the upstream agent logic, lowering the cost of integration.

Advanced Capability: Handling Complex Routing with MCTS

For harder routing decisions, Harness Router introduces Monte Carlo Tree Search (MCTS), capable of running thousands of local simulations to evaluate different routing paths.

This elevates the project beyond simple rule-matching. MCTS is commonly used in game-playing and decision-search scenarios. Applying it to tool routing means the system can evaluate which path is more likely to achieve the goal when faced with multiple plausible candidate tools — through large-scale simulation rather than a one-shot model judgment.

The complete execution flow can be summarized as:

Codex → PreToolUse → shortlist → Harness Router → route / MCTS → execute

The model first produces a shortlist of candidate tools, then Harness Router decides whether to route directly or invoke MCTS for deep search, and only then does execution proceed. Two usage modes are supported: it can be called explicitly as a skill/MCP tool, or it can transparently hook in — giving developers considerable flexibility.

Monte Carlo Tree Search (MCTS) is a heuristic search algorithm based on random sampling, best known for its outstanding performance in Go AI systems such as AlphaGo. The core idea is: starting from the current state, use a large number of random simulations (rollouts) to estimate the expected payoff of different decision paths, dynamically balancing "exploring new paths" against "exploiting known good ones" — the exploration-exploitation trade-off described by the UCB formula.

The analogy for applying MCTS to tool routing is: each candidate tool is like a "move choice," and each simulation is like playing out the task assuming that tool was chosen. Through thousands of simulations, the system can quantitatively compare the probability of success for "choosing fetch_x vs. get_x" in the current task context, rather than relying on the model's one-shot intuition. This approach provides more robust decision-making when tool semantics heavily overlap and single-pass reliability is insufficient — though it does introduce additional local compute overhead. It's therefore best suited for complex scenarios with low routing confidence, not every single call.

Four Specific Costs It Aims to Reduce

The author spells out the project's goals clearly, targeting four categories of overhead:

1. Wrong tool calls — Reduce the probability of the model selecting the wrong tool
2. Redundant tool descriptions in context — No need to feed all tool documentation to the model
3. Token usage — A leaner context directly translates to lower costs
4. Inference wasted on obvious routing decisions — Hand simple decisions off to the routing layer

These four points cover the major efficiency bottlenecks in current agent tool-calling architectures — the direction is right on target.

An Open Question: How to Measure Routing Quality

At the end of the post, the author raises a question worth the entire community thinking about: how do you measure tool routing quality beyond simple "success rate"?

The author says they're especially interested in benchmarking against tool sets deliberately designed to be confusing — like the get_x / fetch_x / read_x example mentioned earlier. These are precisely the scenarios where conventional tool selection becomes visibly ambiguous, making them the ideal proving ground for a routing layer.

The question itself is a good reminder: simply checking "did the call succeed" may not be sufficient to evaluate a routing system. Whether the optimal tool was selected, whether unnecessary retries were avoided, how many tokens were saved, how much latency was added — all of these dimensions arguably belong in a comprehensive evaluation framework.

Summary

Harness Router represents a direction that's gaining momentum in agent engineering: extracting critical decisions like tool selection from the LLM's black-box judgment and handing them to a controllable, pluggable, fail-safe independent layer. The integration of Codex hooks, the robust fail-open design, and the introduction of MCTS all reflect serious consideration for production readiness.

The project is open-source on GitHub (Protocol-Lattice/harness-router) and is worth checking out for developers building agent systems who are frustrated by chaotic tool-calling behavior. Of course, how well it actually performs still awaits validation through more benchmarks on real-world, complex tool sets.