SchemaShield-Lite: Fixing LLM Agent Interface Drift Without Retraining

Deployed LLM agents often fail when tool interface text changes—subtle modifications to tool descriptions, parameter schemas, or candidate lists, even when the underlying semantics remain unchanged, can cause function calls to fail. This "interface drift" has become a practical reliability challenge for tool-augmented language models. A new study proposes SchemaShield-Lite, a lightweight, training-free repair layer that restores robustness without retraining the base model.

Interface Drift: The Overlooked Reliability Gap

Tool-augmented language model agents rely on three types of interface information at inference time: tool description text, parameter schemas, and candidate tool lists. These interfaces often drift after deployment due to version iterations, configuration adjustments, or environment changes, even though the underlying tool functionality may remain completely unchanged.

For example, a tool name changes from get_weather to fetch_weather, a description changes from "get weather" to "query weather conditions," or functionally similar distractor tools are mixed into the candidate list—these changes are trivial to human developers but can cause the model to call the wrong tool or generate parameters that don't match the new schema.

The study defines this failure mode as "interface drift" and points out its pervasiveness: there is a gap between the interface forms the model saw during training and those it actually encounters at deployment, while the cost and timeline of retraining or fine-tuning are often unacceptable.

Function calling is the core mechanism by which large language models interact with external tools. The model generates structured call instructions (usually in JSON format) containing the tool name and parameters based on user requests and tool interface information. This capability depends on the model's learning of interface text patterns during training, so when the interface text changes, the model may fail to generalize correctly.

Parameter schema refers to the normative definition of tool parameters, typically in JSON Schema format, describing parameter names, types, whether they are required, value ranges, and other constraints. The model needs to generate compliant parameters according to the schema, and changes to the schema (such as a parameter name changing from city to location) will directly result in invalid generated calls.

The candidate tool list is the set of tools the model can choose from at inference time. The number and similarity of tools in the list affect the difficulty of the model's selection, and distractor tools (functionally similar but differently named) significantly increase the probability of misselection.

SchemaShield-Lite: A Three-Layer Defense for Training-Free Repair

The SchemaShield-Lite proposed by the research team is an inference-time repair layer that does not modify the base model weights but intercepts and repairs function call errors through three mechanisms:

1. Schema Validation: Checks whether the parameters generated by the model comply with the current tool's schema definition, intercepting format errors in advance.
2. Candidate-set Retry: When the model calls a non-existent tool name, it retries using the current candidate tool list to guide the model toward the correct tool.
3. One-shot Constrained Repair: For calls that fail validation, the model is asked to regenerate a repaired call once under constraints.

These three mechanisms combine into a lightweight wrapper layer that requires no access to the model's internal state, no labeled data, and no additional training at deployment.

A training-free repair layer refers to a lightweight component that intervenes at the inference stage without updating model parameters. Unlike fine-tuning or retraining, it corrects model outputs through external logic (such as validation, retry, and constrained generation), with low deployment cost and no impact on the base model.

Constrained repair is a controlled regeneration strategy: after the model's first call fails, the current tool's schema and candidate list are used as constraints to guide the model to regenerate the call within a limited scope. One-shot means only one repair attempt is made, avoiding infinite loops.

Schema validation is typically performed after the model generates a call and before actual execution, checking whether the parameters comply with the JSON Schema definition (such as types, required fields, and enum values). Validation failure can trigger a retry or repair, thereby intercepting format errors.

Experimental Data: Accuracy Rebounds Substantially

The study evaluated SchemaShield-Lite on the Qwen3.5-9B model using two function calling benchmarks: DICE-300 and BFCL-200.

On DICE-300, accuracy improved from 0.717 to 0.813, recovering 19 of the 22 failures caused by drift; on BFCL-200, accuracy improved from 0.580 to 0.805, recovering 33 of the 45 failures. More importantly, no negative effects from the repair were observed—the repair layer did not introduce new errors.

Ablation experiments further reveal that simple retry strategies cannot explain these gains, indicating that candidate-set retry needs to work in concert with schema validation and constrained repair.

Distractor Tools Are the Main Source of Harm

The study breaks down different types of interface drift and finds that candidate-set drift—especially the addition of distractor tools—accounts for most of the performance loss. In comparison, merely modifying tool descriptions or parameter schemas is less harmful.

This finding has direct engineering implications: when deploying agents, maintaining the purity of the candidate tool list is more critical than optimizing the text descriptions of individual tools. When a system contains many functionally similar tools, the model is more easily misled, and SchemaShield-Lite's candidate-set retry mechanism targets exactly this pain point.

Practical Implications: Validation and Repair as a Deployment Layer

SchemaShield-Lite is positioned not as a replacement for model capabilities but as a robustness layer at deployment. For LLM agent systems already in production, this means the interface drift problem can be mitigated through a lightweight middleware without interrupting service or retraining.

The study's conclusion establishes the "validation plus repair" paradigm as a practical robustness solution for tool-using agents. As agents move from demos to production environments, frequent interface changes will become the norm, and the value of such training-free repair layers will become even more prominent.

Future work may need to explore more complex drift scenarios, error propagation in multi-tool collaborative calls, and the compatibility of the repair layer with different base models. But the current results already show that retraining is not the only way out when facing interface drift.

One-line Summary