Teaching the Enterprise Knowledge Graph to Say "I Don't Know": A Citation-First GraphRAG Practice

Every engineer has experienced this moment: you're about to modify a service but aren't sure which downstream systems will be affected. The answer might be buried in a Confluence page from last year, in some PR description, or in a casual remark in meeting notes—but nobody can tell you exactly where.

Dropping in an LLM to answer questions? The most dangerous thing about this approach is how it fails: silently. Ask a question that isn't answered in the documentation, and you'll get a fluent, plausible, but completely unfounded paragraph. For scenarios like change impact analysis, a confident wrong answer is worse than no answer—it could directly cause an online incident to be approved.

ZEHN Core has set one iron rule for itself: every answer must cite the sentence it came from; if it can't, answer "I don't know." This project is built on FalkorDB, was entered in the FalkorDB Graph Hacks hackathon, and the code is open source.

Why a Graph, Why FalkorDB

"If I change X, what breaks?" is fundamentally a graph problem—it requires walking the chain of "X is used by Y, Y feeds Z." Pure vector search only finds passages that mention X; it can't trace dependencies across documents.

The team used FalkorDB and its GraphRAG SDK to ingest a corpus of Confluence pages, GitHub PRs, meeting notes, and emails. The key design decision was to make provenance information part of the graph's shape itself, rather than metadata attached after the fact:

(Document)-[:PART_OF]->(Chunk)<-[:MENTIONED_IN]-(Entity)
(Entity)-[:RELATES {rel_type, fact, spans}]->(Entity)

Every relationship carries a predicate, an extracted "fact," and the character spans in the source text. Getting from "origin-mapper affects indexer" back to the sentence itself takes just a one-hop Cypher query.

On top of this there are two more layers of protection: retrieval is executed through a fixed, whitelisted set of named queries, up to three hops, with a traversal policy deciding which predicates can be walked; and FalkorDB's multi-named-graph feature lets extraction experiments and pilots run in isolated graphs without disturbing the baseline graph.

GraphRAG is a variant of retrieval-augmented generation (RAG): traditional RAG retrieves similar text fragments from a vector store and stitches them directly into the prompt, whereas GraphRAG first extracts entities and relationships from the corpus to form a knowledge graph, then traverses the graph's edges during retrieval—enabling it to answer questions requiring multi-hop reasoning, such as "A depends on B, and B depends on C." The cost is that the extraction stage introduces noise: relationships judged by the LLM may not actually be supported by the source text, so GraphRAG systems typically need an additional evidence verification layer—which is exactly why this article repeatedly emphasizes provenance.

FalkorDB is a low-latency graph database implemented on sparse matrices (GraphBLAS), supporting the property graph model and the openCypher query language. One of its differentiating features is multi-named graphs: multiple isolated graphs can coexist within the same instance, each with its own schema and data, with queries explicitly specifying the graph name. This lets the team spin up a separate graph for extraction-strategy experiments as a control without affecting the production baseline graph. Its GraphRAG SDK encapsulates the extraction and query pipeline from unstructured documents to entity-relationship graphs.

Three Layers of Mechanisms to Prevent Hallucination—Only One Is a Prompt

The author candidly points out that simply writing "do not use external knowledge" in the prompt is necessary but far from sufficient. The real defenses lie in the other two layers:

Citation verification. Every evidence ID the model writes is compared against the IDs actually returned by the tools. If even one citation is fabricated, the entire answer is rejected. A real citation cannot vouch for a fabricated one.

Forced abstention. No valid citation, no answer—no matter how confident the prose.

To verify this mechanism, the test suite deliberately feeds the agent loop a misbehaving model: one that fabricates citations and answers without evidence, then asserts that the answer is indeed withheld.

Citation verification is fundamentally different from the common practice of "asking the model to cite its sources": the latter is only a request at the prompt level, and the model may still fabricate a syntactically valid but nonexistent ID; the former is programmatic post-hoc validation—comparing each evidence identifier in the model's output against the result set actually returned by the tool call, and if any ID is not in the returned set, the entire answer is invalidated. This is an all-or-nothing policy: a single fabricated citation means the answer's chain of reasoning is deemed untrustworthy, because the model has already shown a tendency to stitch together nonexistent evidence, and its other "real" citations cannot vouch for it.

Forced abstention means the system proactively outputs "I don't know" when valid evidence is lacking, rather than letting the model improvise. In engineering terms, it requires an explicit "evidence sufficiency" gate in the answer-generation path, and that gate must be decided by code rather than by the model. Together, the two mean the system's default state is refusal, and answering is an exception that must be unlocked by evidence—the exact opposite of the default in most RAG systems, which is "answer by default, apologize occasionally."

Scoring the Graph's Own Assertions

The existence of a path in the graph doesn't mean the document actually said so. LLM extraction is noisy: the model sees "the release pipeline will emit a v2 index" and eagerly writes an AFFECTS edge, even though that sentence describes a plan, not a dependency.

So for every returned path, the system compares each hop against its own source sentence and labels it:

Label	Meaning
Source-supported	Every predicate on the path is supported by its sentence
Inferred assumption	The path exists, but at least one hop is not asserted by its sentence
Insufficient evidence	Negated, hypothetical, or entity binding failed

In the UI, inferred hops are drawn as dashed edges with warning markers. When asked about the Redwood API, the graph found twelve paths—eleven of which were labeled inferred assumption. A system trying to look impressive would present twelve confident dependencies; ZEHN Core chooses to grade its own homework.

The same principle applies to ambiguity. "Redwood Inference" is simultaneously a team, a service, and a customer in the corpus. Rather than merging them into a tidier answer, the system shows all three identities and then stops.

Entity binding (entity resolution) is the most error-prone step in knowledge graph extraction: the extractor must decide which entity node a mention of "Redwood" in the text actually refers to. When the corpus contains same-named objects with different referents (a team, a service, and a customer all called Redwood Inference), a binding error attaches relationship edges to the wrong node, distorting the entire reasoning path. This article mentions about one-third of relationship binding errors, mainly stemming from this kind of same-name ambiguity and failed coreference resolution. Common mitigations include: attaching types and contextual fingerprints to entities, requiring extraction to output character spans pointing to the source text so humans can review, and refusing to merge when multiple candidates share the same name—displaying them side by side instead.

Five Lessons from Practice

1. The model's "facts" are not quotations. Graph extraction stores a natural-language fact field on every edge. The team's empirical finding: of 186 facts, 0 were verbatim from the source text. They read like quotations, which is exactly what makes them dangerous. They stopped rendering fact as a citation and instead built a citation-first path that stores the original sentences.

2. Audit your edges, not just your answers. Manual audits found that roughly one-third of sampled relationships were bound to the wrong entity target. This number wasn't hidden—it's presented as a visible limitation alongside every investigation.

3. Know the physical shape of your schema. In this graph, every edge is physically RELATES, and the semantic type lives in r.rel_type. Querying :DEPENDS_ON returns empty—and empty looks exactly like "no dependencies." This taught the team its most important engineering rule: failure must never look like an empty result. A database error and "nothing depends on it" are two different facts, and the UI renders them differently.

4. Don't turn on what you haven't validated. The team evaluated making CALLS edges traversable. It produced interesting chains but couldn't be backed by evidence, so it stays off in production—and the currently active policy is displayed live in the UI. If someone enables it, the badge shows evaluation mode.

5. Say what you haven't built yet. The corpus is a 30-document subset covering four source types; there's no ticket source and no staleness detection. The UI's history page says "history is not persisted in this build" rather than showing fake data rows. A control that does nothing when clicked is worse than no control at all.

Results and Takeaways

ZEHN Core ships as a FastAPI backend plus a React frontend, with seven screens: a change impact radar with a tabbed evidence inspector, an agent that streams tool calls live, an interactive knowledge browser, and an observability console that reports API health and graph health as separate facts. The project is backed by 959 backend tests and 63 frontend tests, including tests that assert "cannot cite a source that doesn't exist."

The author's closing takeaway has nothing to do with graphs or models: trustworthiness is a visible product feature—a dashed edge, a refusal, a limitation listed up front. Users can't verify fluency, but they can verify a sentence.

This approach is valuable for any team building enterprise RAG systems: citation-first isn't a constraint—it's a trust design that users can perceive.