Refusing to Cite Unverifiable Law: How AILegal Uses Engineering to Eliminate Legal Hallucinations

The most dangerous failure mode for legal AI isn't refusing to answer—it's confidently citing a statute that looks plausible but doesn't actually exist. When building a legal assistant for people who can't afford lawyers, the AILegal team made "never state a legal fact not verified against an official source" their single iron rule, and designed an entire executable engineering architecture around it. This article breaks down their technical approach, measured results, and unsolved problems.

Starting from a real failure: plausible-looking citations are the deadliest

The early version used the usual approach: carefully crafted prompts, stuffing the corpus into the context window, and instructing the model to "answer only based on these sources." On the surface this method seemed fine, but manual review exposed two types of errors.

The first was fabricated statute numbers—the assistant cited "18 U.S.C. § 301," a provision that simply doesn't exist in that form. The second was misattribution—it cited Section 157 of the UK's Consumer Credit Act 1974, which does exist, but has nothing to do with the user's question.

The author pinpointed the key point: neither answer "looked" wrong. A plausible-looking citation is far more dangerous than an obvious refusal, because users lack the ability to tell the difference. This is also the core risk that distinguishes legal AI from general Q&A products—the cost of a wrong answer could be missing a statute of limitations.

Making verification a data structure, not a prompt

AILegal's core design philosophy: don't rely on prompts to constrain model behavior—turn every legal norm into a structured record.

What a Norm record contains

Each norm is stored as a Norm object with fields including a unique ID, statute code (e.g., "15 U.S.C. § 9058(c)"), jurisdiction, a human-reviewed plain-language summary, official source URL, hash of the verified version, verification timestamp, and most critically, the verified boolean flag.

The corpus currently holds 53 norms: 11 for Russia, 26 for the US, 9 for the UK, and 7 for Canada. Not a large scale, but every entry is traceable and re-checkable.

Scheduled tasks and hash verification

The system runs a scheduled task every three days, fetching each official source, requiring that the numbers and terms in the summary actually appear in the fetched text, and storing the hash of the verified version. If the hash changes, the norm is flagged for manual re-verification.

There's a deliberate design choice here: the verified flag is never set by a machine. The script can only prove that "the text really says what we claim it says," while "can this norm be used to respond to a deadline" must be judged by a human. The author explicitly notes that conflating these two judgments is precisely the root cause of legal AI starting to fabricate law out of thin air.

Hash verification is the key mechanism here. A hash function (such as SHA-256) maps text of any length to a fixed-length fingerprint, and any tiny change to the text produces a completely different hash value. AILegal stores the hash of the official source page at verification time, then re-fetches three days later and computes a new hash; if the two differ, the source text has been modified (e.g., statutory amendment, page update), and the system immediately flags the norm for review, preventing it from being used to generate answers. This mechanism solves the timeliness problem of legal information: statutes get amended, while model training data is static. Hash comparison provides a low-cost, automatable means of change detection, but as the author emphasizes, it can only prove that "the text changed," not judge "whether the change affects the legal conclusion"—the latter still requires human intervention.

The "gate" mechanism at answer time

When generating answers, the rules are deliberately designed to be "boring": if a norm hasn't been verified, the assistant won't produce a deadline or provision number derived from it—instead it explicitly states it cannot confirm, and asks follow-up questions about the missing factual information.

The team also measured the effectiveness of a "soft" classifier designed to catch paraphrased expressions via stem similarity. Of 31 test paraphrases, the exact-match stage resolved 12; the soft-match stage added hints for 14 more, of which 8 were correct—an accuracy of about 57%. That number is far from sufficient to support directly answering legal questions—one out of every two hints points to the wrong norm. Therefore the soft-match stage only asks clarifying questions and never outputs conclusions.

The comparison between the soft-match classifier and exact matching deserves elaboration. Exact matching requires that keywords in the user query match entries in the norm summary exactly (or after simple stemming), yielding low recall but high precision. Soft matching catches paraphrases via stem similarity (e.g., grouping "вычисли" and "вычислять" under the same stem), expanding recall at the cost of introducing many false positives—in testing, 57% of hints pointed to the wrong norm. AILegal's engineering trade-off: the soft-match stage is used only to generate clarifying questions ("Did you mean...?"), never to output legal conclusions directly. This "tiered confidence" design uses the high-precision path for final answers and the low-precision path only for interactive guidance, achieving a safe balance between recall and precision.

Unsolved problems: synonyms and semantic representation

The biggest weakness is synonym recognition. In Russian, "вычисли" and "посчитай," "списали" and "удержали"—different words, same meaning—and without a synonym source, similarity scoring can't tell them apart.

The team tried local embedding models (nomic-embed-text, qwen3-embedding) on standard paraphrases, and the results landed at top-3/top-4 with cosine gaps of only 0.001–0.03—noise level. The author's conclusion is worth noting: the real solution isn't a better model, but building a semantic representation for each norm—human-written and human-reviewed everyday search phrasings. That's legal work, not prompt fine-tuning.

This also reveals a broader industry judgment: when a task demands high precision, marginal gains in model capability often matter less than high-quality domain data structures.

An embedding model maps text to high-dimensional vectors, where semantically similar texts are closer in vector space—typically measured by cosine similarity. The nomic-embed-text and qwen3-embedding models AILegal tried are locally runnable open-source embedding models that require no external API calls, suiting the privacy requirements of legal data. However, testing showed that cosine gaps between standard paraphrases were only 0.001–0.03, and the gaps between texts with different meanings were in the same range—meaning the model cannot reliably distinguish "synonymous paraphrase" from "unrelated expression," and similarity scoring degrades to noise. The author concludes: for a field like law that demands extremely high precision, general-purpose embedding models lack the semantic discrimination needed, and a "semantic representation" must be manually built for each norm (i.e., enumerating the various everyday phrasings that correspond to that norm)—this is essentially domain knowledge engineering, not a model selection problem.

Why "boring failures" are worth pursuing

If you ask a legal assistant "when is the deadline?" and it answers confidently, you can't tell whether that's a verified answer or a hallucination. The gate mechanism makes the failure mode predictable: "I cannot confirm this from the official text." When the cost of a wrong answer is missing a court deadline, that predictability is exactly the quality the product most needs.

The team openly invites users to try to break the system: pick any norm, compare it against its linked official source, and try to get the assistant to say something not present in the source text. This adversarial verification approach is worth borrowing for AI products in other high-risk domains. Note that this product does not constitute legal advice; legal content is supported in Russian and English, and the interface supports more languages.

One-line Summary

AILegal uses structured Norm records and human-verification gates to stop legal AI from confidently fabricating statutes.

Paragraph Summary

The AILegal team designed an engineering architecture built on the iron rule "never state a legal fact not verified against an official source," targeting legal AI's most dangerous failure mode—confidently citing nonexistent or irrelevant statutes. The core is storing each legal norm as a structured Norm record with the official source URL, version hash, and a human-set verified flag that machines may never set themselves. The system fetches official sources every three days and checks for hash changes; the corpus currently covers 53 norms across the US, UK, Canada, and Russia. Answers go through a gate mechanism: unverified norms yield no conclusions, and the soft-match classifier, at only 57% precision, is used solely to ask clarifying questions. The biggest current weakness is synonym recognition—local embedding models produce cosine gaps of just 0.001–0.03, which is noise level—and the solution lies in manually building semantic representations for norms rather than swapping models.