Compliance-Bounded Self-Evolution of LLM Agents: Auditable Admission Gating in Credit Pipelines

When Self-Evolution Meets Regulatory Scrutiny

Self-improving large language model (LLM) agents could, in theory, automatically adjust credit scoring pipelines as regulations change. But a fundamental tension exists: if an agent rewrites itself directly, it destroys the very audit trail that regulators need — a named change, a recorded test, an explicit approval.

This arXiv paper (arXiv:2610.10629v1) makes a compelling argument: an agent's self-evolution is only auditable when constrained within a runtime harness — meaning the three mutable surfaces of instruction text, tool-call logic, and primitive composition — while model weights remain fixed. Every adaptation becomes a diff with a stated reason and accompanying tests, rather than an opaque weight mutation.

The Dual-Loop Engine and Admission Gate

The paper's core engineering contribution is a dual-loop engine built on this constraint, paired with a single admission gate. Before any change is deployed, the gate writes a hash-chained record, forming a tamper-evident provenance trail.

Notably, the researchers evaluate this gate in a simulation environment using mock agents and a seeded-search proposer rather than real language models. This design choice strips out LLM nondeterminism, allowing the gate's effectiveness to be measured cleanly — a pragmatic experimental decision.

Quantified Gate Performance

Experiments covered three types of supervisory re-interpretation, three severity levels, and 10 random seeds per condition. The results are compelling:

• Out of 7,449 candidate changes, the gated loop admitted only 144;
• Not one of those 144 changes degraded error rates on held-out historical data;
• The gate restored false-positive rates to oracle-level performance while adding no missed flags across all low- and medium-severity scenarios.

The dual-loop engine draws from the inner/outer loop separation principle in control theory. The inner loop handles concrete credit-scoring tasks under the current rule framework; the outer loop periodically reviews inner-loop performance and proposes adjustments to the harness layer. Separating "execution" from "evolution" prevents the two from contaminating each other. Hash-chained records are a lightweight application of blockchain-style audit logging: each change record includes a hash of the previous one, so any tampering with history breaks the chain and is immediately detectable. This provides mathematical non-repudiation for regulatory review, rather than relying solely on access controls or manual signatures. The seeded-search proposer is a deterministic change-generation mechanism: by fixing the random seed, researchers can reproduce the exact same sequence of candidate changes across experimental conditions, cleanly separating gate performance from the stochastic noise of LLM sampling and ensuring strict reproducibility in controlled experiments.

The Danger of Unconstrained Systems

To highlight the gate's value, the researchers ran a control experiment: replacing the admission gate with a naive check common in unconstrained systems — namely, the signal of "fewer visible errors in recent trajectories."

The results were sobering. The same loop admitted 309 harmful changes, and in 49 out of 90 runs, the missed-flag rate remained above 10%. False positives appeared to drop on the surface — but only because the screening standard had quietly been relaxed. This exposes a trap that's easy to fall into in real deployments: using "fewer errors" as a self-optimization signal induces the system toward an outcome that looks better but is actually more dangerous.

Re-Interpretations Must Be Encoded as Rules

The paper also yields a subtle but important finding: when evaluated against pre-shift labels, the gate rejected every single candidate change. This means a supervisory re-interpretation must be explicitly encoded as a rule that re-labels history — it cannot be left for the agent to infer implicitly. This establishes a clear operational boundary for change management in compliance scenarios.

On remediation capability: parameter drift and scope drift can be locally repaired; structural drift can only be resolved through primitive replacement. The paper honestly acknowledges a limitation: at the highest structural severity level, the gate's fixed tolerance blocked correct replacements in half the seeds. In other words, an overly conservative gate can also reject the right evolutionary direction — an inherent tension between safety and adaptability.

The paper distinguishes three types of drift, and understanding this taxonomy is essential for grasping the gate's applicability boundaries. Parameter drift refers to feature weights or thresholds shifting over time — for example, a change in the coefficient for income's effect on default probability — which can be locally repaired by adjusting numerical parameters in the instruction text. Scope drift refers to input data distributions exceeding the original training or calibration range, such as the emergence of new borrower populations, typically addressed by extending the coverage logic of tool calls. Structural drift represents the most fundamental change: the original primitive composition logic is no longer applicable and requires replacing underlying computational primitives — effectively an architectural reconstruction of the pipeline. It is at this level that a fixed-tolerance gate produces false rejections, unable to distinguish "destructive changes" from "necessary architectural replacements," revealing the deepest tension between hard-coded rules and dynamic adaptation.

Mapping to Regulatory Frameworks

This work is not purely theoretical — it explicitly maps its mechanisms to EU AI Act provisions governing high-risk credit scoring. The researchers also flag a notable regulatory gap: the April 2026 U.S. model risk guidance excludes agentic AI from its scope.

This regulatory misalignment means that existing U.S. model risk management frameworks may not provide adequate coverage for deploying self-evolving agents in high-stakes domains like credit. The paper's "harness as the only mutable surface" approach offers a technical path to filling that gap: by confining evolution to an auditable layer, regulators can still see the trio of named change, test, and approval.

The EU AI Act, which formally took effect in 2024, explicitly classifies credit scoring as a high-risk AI system, requiring deployers to maintain technical documentation, implement human oversight, and ensure explainability throughout the system lifecycle. The core compliance pressure centers on change traceability — any modification to a high-risk system must be reassessed for whether it triggers re-declaration obligations. The harness-layer diff mechanism proposed in the paper aligns closely with this requirement: each diff naturally corresponds to a "named change," the accompanying test results constitute incremental updates to technical documentation, and the admission gate's approval records satisfy human oversight audit requirements. By contrast, the U.S. SR 11-7 Model Risk Management Guidance (issued in 2011; the 2026 draft update still does not cover agentic AI) primarily targets validation and change management for static models. Its "development–validation–deployment" three-stage process assumes that models will not autonomously modify their own logic after deployment — a premise that fundamentally breaks down in the context of self-evolving agents.

Implications for the Industry

The value of this research lies not in any specific performance number, but in how it reframes the conditions under which self-evolving agents can be deployed. Three core insights stand out:

First, the binary of fixed weights / mutable harness ensures that every change an agent makes leaves an auditable trace — a principle with broad applicability across any regulated vertical.

Second, the measurement signal for self-optimization must be carefully guarded. Simply chasing "fewer visible errors" will systematically relax safety screening.

Third, there is a real tradeoff between compliance constraints and adaptive capacity. Fixed-tolerance gates will fail under extreme structural change, pointing to the need for more adaptive admission strategies in future work.

For teams exploring agentic AI deployment, this "compliance-bounded self-evolution" framework offers a reference paradigm for striking a balance between innovation and auditability.