Taming Non-Deterministic AI: Practical Lessons in Hardening Agentic Systems

Title

Taming Non-Deterministic AI: Practical Lessons in Hardening Agentic Systems

Meta Description

The Synthadoc team shares practical experience hardening Agentic AI systems: starting from five typical failure modes (models rewriting pages, retaining wrong duplicates, tool-call leakage, etc.), they propose a two-layer architecture of hardening and harness engineering, emphasizing code-level control rather than prompts to tame non-deterministic AI.

Article Content

When building Agentic AI systems, the most frustrating issues are often not the dramatic hallucination errors, but the model's tendency to "do better" on its own after "understanding the task." The Synthadoc team repeatedly encountered this class of problems while developing their content health maintenance workflow. Their experience reveals a key principle: To tame a non-deterministic AI machine, what you need is not cleverer prompts, but an engineered constraint architecture.

Problem Scenario: When Wiki Pages Start to "Proliferate"

Synthadoc is an open-source AI-native wiki engine whose job is to synthesize structured Markdown wiki pages from raw documents such as PDFs, web pages, and transcripts via an LLM. Over time, as the same topic gets ingested repeatedly from multiple sources, problems follow.

A page about Alan Turing might accumulate ten ## Bletchley Park and Cryptanalysis headings and eight ## Early Life and Education headings. In isolation the page looks coherent, but structurally it has fragmented: duplicated content, contradictory emphasis, and sections that appear thin because a heading was added before body text was written.

The content health scanner is responsible for detecting these issues. It deliberately uses pure Python heuristics and does not rely on an LLM—deterministic regular expressions identify structural problems such as duplicate sections, empty sections, thin sections, and unclosed code blocks. This part is reliable. The next step—using an LLM to fix the problems—is where the story gets really interesting.

Agentic AI is fundamentally different from traditional one-shot question-and-answer AI calls. In the traditional mode, the user asks and the model answers; the interaction is one-off and stateless. An Agentic system, by contrast, gives the model the ability to autonomously plan, call tools, read external resources, and iterate across multiple rounds to advance a task. Synthadoc's content health workflow is exactly this paradigm: the LLM not only generates text, but also reads pages, understands structural issues, produces modification suggestions, and ultimately affects persisted state. This autonomy brings a leap in capability, but also a new failure surface—the model is no longer an "answerer" but an "actor," and each of its outputs may directly rewrite real data. Therefore, the design focus of Agentic systems shifts from "how to ask better" to "how to limit what the model can affect," which is precisely the origin of the "boundaries" and "harness" repeatedly emphasized in this article.

Five Failure Modes: How Models "Mean Well but Do Harm"

The team originally designed a clear five-step workflow: load the page and issue list, send it to the LLM for processing, show a before/after comparison, wait for user approval, and take a snapshot before writing. Each step has its purpose. In actual operation, however, the results exposed gaps in hardening across multiple dimensions.

Failure One: The Model Rewrote the Entire Page

The first problem was immediate. A page about the open-source movement had twelve structural issues—three duplicate sections and nine thin sections. But the returned "fixed" content had no structural similarity to the original. The original heading ## Richard Stallman and GNU (1983) disappeared, and so did ## The Cathedral and the Bazaar (1999). In their place were entirely new headings like ## Historical Framework and Origins.

The content was factually reasonable, but it was something the user never requested and could not safely approve—all traceability between the original and the proposed version had been lost. The model had effectively rewritten a wiki page using its own pretrained knowledge, treating the page we provided as a rough reference. The word "fix" in the prompt was interpreted by the model as permission to rewrite.

Failure Two: Retaining the Wrong Duplicate

On a computing history page, ## Moore's Law appeared three times. Two were short summaries of about forty words; the third was a detailed paragraph with four footnote citations. The initial prompt said to "keep the first occurrence." The model followed faithfully, keeping the forty-word summary and discarding the version with citations.

Technically correct, practically wrong. The rule was precise but lacked consideration of the true goal.

Failure Three: Tool-Call Leakage from CLI Providers

Synthadoc supports multiple LLM backends, one of which is a CLI-based agent tool (such as opencode or Claude Code). When the content health workflow was routed through that provider, the response text contained shell command output and traces of internal tool calls. The model bypassed our carefully constructed context, read the actual file on disk directly, and its internal tool calls leaked into the response text that was about to be written to the wiki.

Failure Four: Planning Notes Treated as Final Content

In some runs, proposed content was mixed with lines like ### LISP (1958) - actually LISP is more third gen but... and ### Introduction paragraph (existing). These were the model's internal planning notes—uncleaned draft scaffolding. A user quickly reviewing a diff in a modal would very likely miss them.

Failure Five: "Correcting" a Date and Renaming a Heading

After tightening the prompt, one run still returned a subtle violation. The original heading was ## The Cathedral and the Bazaar (1999), and the proposed content changed it to ## The Cathedral and the Bazaar, with the year removed. The model "knew" that the article was actually published in 1997, so it silently "corrected" what it considered a user error. This was an editorial judgment we had not authorized.

LLM hallucination in technical contexts usually refers to a model generating plausible but factually incorrect content. But in Agentic scenarios, the destructiveness of hallucination is amplified: the model is not just "saying the wrong thing," but "doing the wrong thing." Among the failures observed in this article, the model substitutes its own pretrained knowledge for the real data provided by the user—for example, silently "correcting" the publication year of The Cathedral and the Bazaar or deleting parentheses around a year it deemed outdated. The root of such behavior is that language models are trained to minimize prediction error and produce the most likely next token, not to faithfully execute instructions. The verb "fix" in the model's probability space naturally leans toward "rewrite more accurately," and the model cannot hard-stop at a "fact check" point the way a traditional program can. Understanding this is the premise for understanding all the hardening strategies in this article: you cannot expect the model to constrain itself; you can only place external controls at the boundaries of its inputs and outputs.

Failure Mode Map: Drift and Hallucination

These failures fall precisely into two categories. Category A—Drift: the model understands the task but applies the wrong judgment. Category B—Hallucination: the model replaces the task with its own knowledge.

The distinction is critical. Drift means the correct process is making the wrong decision—"keep the first occurrence" rather than "keep the most complete version." Hallucination means the correct process is replaced entirely—generating a page from weights rather than fixing the page we handed it. The mitigation measures for the two are completely different.

Non-determinism is the key difference between LLM outputs and ordinary software functions. Given the same input and prompt, a model may produce different results because of sampling temperature, provider implementation differences, context length, internal reasoning loops, and other factors, and even within a single session its behavior may vary from one call to the next. The Synthadoc team uses "compound non-determinism" to describe the dilemma with CLI providers: an API provider is a stateless text generation endpoint whose behavior is relatively predictable; a CLI agent tool (such as Claude Code) is itself an Agentic system with planning, tool calls, and multi-step reasoning, so its output passes through another layer of autonomous decision-making. When two layers of non-determinism stack, even carefully constructed prompts may be washed out by internal reasoning loops. This explains why the system needs a separate "six-issue cap" for the CLI path—not because model capability is insufficient, but because the control chain is too long and the accumulated marginal unpredictability exceeds the manageable threshold.

Two-Layer Hardening: Hardening and Harness Engineering

The fixes fall into two layers. Hardening targets what the model produces: stricter prompts, output parsing, provider-specific safeguards. Harness engineering targets what the model can affect: human gates, rollback mechanisms, scope exclusions. Hardening reduces the probability of bad output; the harness limits the consequences of output that slips through. Neither is sufficient alone.

Hardening Measures

1. Artifact extraction as an input boundary. Introduce a _extract_md_from_response() preprocessing function that strips everything before the first # H1 heading in the response. Tool-call output from CLI providers always appears before the actual content. Using the first H1 as a clean boundary restores the Markdown content without introducing noise. This is boundary enforcement at the output interface: treating the model's raw response as untrusted input to the parser.

2. Draft detection as an output gate. Add an _is_draft() regex to catch linguistic markers of incomplete thinking: TODO, FIXME, Expand to, Hmm,, Wait,, I'll, Let me, etc. If there is a match, the proposed content is flagged, the page is automatically skipped, and a clear explanation is given in the UI. This is a deterministic check that the model cannot disable by rephrasing.

3. Surgical prompt constraints. Replace "fix the structural issues listed below" with an explicit hierarchy of constraints. Absolute rules include: preserve exactly every existing section heading (including date years in parentheses), preserve exactly the order of sections, do not add new top-level or second-level sections, do not rename, merge, split, or reorder any section. The only allowed changes are: for duplicate sections, keep the most complete version and place it as-is; for thin/empty sections, add factual narrative under the existing heading.

Changing "fix the issues" to "allowed changes" is deliberate. When permissions are enumerated rather than prohibitions, the model cannot rationalize its own behavior.

4. Issue count threshold. Even after tightening the prompt, pages with more than six issues still produced hallucinated new sections in CLI provider runs. The CLI provider is itself an agent tool with its own reasoning loop, and compound non-determinism (our model + its internal model) can overwhelm long issue lists. The team set a hard skip at six issues for the CLI path, while API providers use a separate code path without this limit.

Harness Engineering

5. System page exclusion. Thin sections on pages like index and dashboard were correctly identified by the scanner, but they are navigation pages rather than content pages. Use the existing SYSTEM_PAGE_SLUGS constant to exclude them from the repair loop, avoiding duplicate definitions and magic strings.

6. Human gate as a structural requirement. Every proposed fix must go through before/after review. This is not a feature but an architectural guarantee: no automated process can damage wiki content without explicit consent. The confirmation card shows the page slug, issue summary, the first 2000 characters of the original content, the first 2000 characters of the proposed content, and approve/skip buttons plus a five-minute timeout auto-reject. Scrollability of code blocks is a must-solve UX issue—a gate that exists in theory but is unusable in practice is not a gate.

7. Snapshots and rollback as a safety net. Before writing any approved fix, snapshot_if_changed() takes a versioned copy of the original content. If an approved result turns out to be problematic later, the user can request a rollback by name. This is not recovery from a system failure, but a first-class workflow feature that acknowledges the model may produce subtle errors not caught during review.

Defensive programming is the engineering root of the hardening philosophy in this article. In traditional software, this principle means trusting no external input—user input, network responses, and third-party API return values should all be validated before use. Synthadoc migrates this to LLM outputs: the model's response is treated as "untrusted input" and filtered through parsers, regex gates, and structural validation before entering the write path. _extract_md_from_response() stripping everything before the first H1 and _is_draft() detecting draft language are both typical input sanitization techniques. At a deeper level, "enumerating what is allowed rather than what is forbidden" corresponds to the Principle of Least Privilege in security engineering: rather than telling the model what it must not do (a prohibition list can be infinitely long, and the model will rationalize exceptions), tell it only what it can do (narrowing the action space to something enumerable). This approach shifts the security model of Agentic systems from "trust but verify" to "distrust by default, release layer by layer."

Core Principle: Architecture, Not Text

All fixes follow a consistent pattern: every control—whether hardening the model output or constraining the harness around it—is implemented in code, not in text.

"Do not hallucinate" in a system prompt is only behavioral advice. The model will consider it, weigh it against other pressures, and sometimes override it. This is not a defect; it is how language models work. There is no hard stop in the token prediction process for "this will be a hallucination."

The control matrix the team built is as follows: the hardening layer includes _extract_md_from_response() (runs on the raw response bytes), the _is_draft() regex (deterministic matching, outside the model loop), the allowed-changes prompt (narrowing the action space), and an issue-count cap (a hardcoded gate; the model never even sees the page); the harness layer includes system page exclusion (constant slug lookup, not in the prompt), before/after human review (no write path can bypass it), and snapshot/rollback (written before applying a fix).

These controls form layers. A fix that evades the prompt constraints is caught by the draft detector or the human gate. A fix that passes the human gate can be rolled back. No single layer is sufficient, but stacked layers are.

Broader Relevance Beyond Wiki Maintenance

Updating a wiki page incorrectly is a recoverable mistake. But the observed failure modes—the model substituting its own knowledge for the task, internal tool calls leaking into output, silently "correcting" data it considers wrong—are not domain-specific.

The same patterns appear in: Agentic coding tools refactoring code the user never asked to touch; documentation automation systems "improving" legal clauses beyond the approved scope; data pipelines where LLMs normalize values using world knowledge rather than the provided schema; and financial report generators extrapolating numbers that do not exist in the source data.

In each case, the consequences of unhardened Agentic behavior scale with the risk of the domain. A hallucinated section in a personal wiki is recoverable; a hallucinated number in a financial disclosure is not.

The argument for hardening is not that models are bad. It is that models are non-deterministic, and production systems need deterministic guarantees at the boundary where AI outputs meet persistent state.

Actionable Practical Checklist

If you are building an Agentic system that reads and modifies persistent data, consider the following minimum requirements:

Boundary enforcement. Treat model output as untrusted input at every handoff. Parse it, validate it, and strip anything that does not belong before it reaches the next step.

Deterministic output gates. Add validation logic that runs after the model response and is unaffected by model behavior. Draft detection, schema validation, output format checks. These are deterministic and cannot be bypassed by the model.

Enumerate permissions rather than prohibitions. Tell the model in the prompt what it is allowed to do, not just what to avoid. A bounded set of allowed actions is easier to reason about than an open-ended task plus an exception list.

Acknowledge provider heterogeneity. Not all "model" configurations behave the same. CLI-based agent providers and stateless API completion endpoints have fundamentally different behavioral properties. Design the hardening layer accordingly, or constrain which providers can touch which operations.

Human gates as a structural requirement. For any workflow that modifies persistent state, the human review step is not UX decoration but a control. Make it usable. If the review UI cannot show the full diff, the gate is not operating as intended.

Reversibility as a first-class design constraint. Snapshots and rollback should be designed in from the start, not bolted on after the first production incident. Assume that models will sometimes produce subtle erroneous results that pass review, and design recovery paths accordingly.

Conclusion

Synthadoc's content health maintenance workflow went through multiple iterations before it behaved predictably. Each iteration targeted a specific failure mode rather than rewording the prompt. In the end, the team trusted the workflow—not because the model was perfect, but because the system was designed to catch and contain errors.

Non-deterministic machines can be tamed. But taming comes from engineering, not prompts.

One-line Summary

The reliable operation of Agentic AI depends on an engineered constraint architecture rather than cleverer prompts, applying dual controls of hardening and harness to the two classes of failures: drift and hallucination.

Paragraph Summary

Using the content health maintenance workflow of the open-source wiki engine Synthadoc as a case study, this article reveals the core failure modes of Agentic AI systems and the strategies for addressing them. The team's original five-step workflow exposed five types of problems in practice: the model rewriting the entire page, retaining the wrong duplicate, tool-call leakage from CLI providers, planning notes mixed into final content, and unauthorized edits such as silently changing dates. The author divides these failures precisely into two categories—drift (the model understands the task but misjudges) and hallucination (the model substitutes its own knowledge for the task)—and notes that the two require different mitigations. The fixes are divided into two layers: hardening, which controls what the model produces, including artifact extraction, draft detection, enumerated prompt constraints, and issue-count thresholds; and harness engineering, which controls what the model can affect, including system page exclusion, human gates, and snapshot rollback. The core argument is that all controls must be implemented in code rather than prompts, because prompts are only behavioral advice and the model sometimes overrides them. This pattern generalizes to all Agentic scenarios including coding tools, documentation automation, data pipelines, and financial reporting, where higher risk increases the need for deterministic boundary guarantees.