SourceCheck: Replace Copying with Citations to Build Verifiable and Trustworthy LLM Output

Why Do We Need SourceCheck?

As large language models (LLMs) become ubiquitous, more and more people are accustomed to saying "I asked the AI, and the result was..." But from the very first day LLMs entered the public eye, the hallucination problem, knowledge drift, and the contamination of model outputs by low-quality web content have been continuously eroding the trustworthiness of LLM outputs.

LLM "hallucination" refers to the model generating content that seems plausible but is actually inaccurate or entirely fabricated. This phenomenon stems from the fundamental nature of LLMs: they are probability-based text prediction systems that learn language patterns by training on massive corpora, rather than truly "understanding" or "memorizing" facts. When a model encounters questions insufficiently covered in its training data, it tends to fill the output with statistically "plausible-sounding" content rather than acknowledging uncertainty. Research shows that even top-tier models like GPT-4 can still have hallucination rates of 15%–30% in specific domains.

SourceCheck is an open-source tool project built precisely to address this pain point. It continues the core philosophy from ReviewDeck (a code review project), extending "how to make LLMs reliably split code" into a more universal proposition — how to build verifiable LLM output.

Its core idea can be summarized in one sentence: Replace copying with citations. Instead of having the LLM use large blocks of output tokens to restate sources and original text, a new protocol format enables the model to output minimal metadata describing "which paragraph from which document was cited," while a deterministic program verifies the legitimacy of the citation. This design philosophy closely mirrors academic citation standards — papers don't copy entire referenced works but instead provide precise "author, year, page number" locators, leaving readers to consult and verify the original text themselves. Transplanting this mature trust mechanism from the human knowledge system to the LLM output layer is SourceCheck's most fundamental paradigm innovation.

How Does SourceCheck's Citation Format Work?

Let's look at a practical demo: suppose we have a description about "Orbit 2.4," which contains specific numbers like "version 2.4" and "37%," as well as technical concepts like "VE Plugin API." All of this information comes from specific documents.

SourceCheck works as follows:

• Green annotations (verification passed): Facts that passed the check — clicking on them reveals the source. For example, "Orbit 2.4" traces back to the explicit description of "2.4.0" in the ChangeLog document, with the LLM assessing 90% relevance between the two.
• Red annotations (hallucination detected): Content identified as hallucination. For example, if the original text states "continues to use YAML, no format change," but the output claims "YAML has been changed to TOML," the system flags it as a hallucination.
• Multi-source corroboration: The number "37%" might come from two places — the ChangeLog showing latency dropping from 820ms to 517ms (the Agent calculates this as approximately 37% and assigns 80% confidence), and the Benchmark results explicitly stating "reduced by 37%."

The key point is that whether a fact passes or fails verification, the judgment basis can be traced back through citations. The metadata only contains line numbers and start/end string descriptions, without restating large chunks of the original text — in LLM pricing models, output tokens typically cost more than input tokens, so restating original text in bulk not only wastes money but also introduces new distortions due to the model's tendency to paraphrase. The deterministic program is responsible for checking whether the referenced range actually exists in the source document.

The Four Core Modules of SourceCheck Explained

Protocol Design: A Unified Reference Positioning System

SourceCheck defines a custom protocol using Reference as the core concept to simultaneously locate two types of content:

1. Claim — A specific assertion in the LLM output that needs verification
2. Source — The original document fragment that supports or refutes the claim

Both use the unified Reference format for positioning, which provides exceptional consistency and extensibility.

Compared to ReviewDeck, which only supported line number positioning (sufficient for code scenarios), SourceCheck introduces an important upgrade: in addition to vertical line number positioning, it adds horizontal string boundary capability. This is because in general text scenarios, it's often necessary to locate content across lines or find more precise fragments within the same line.

But what if the string doesn't match, or the same string appears multiple times on a given line? This is exactly the problem the deterministic program is designed to solve. A "deterministic program" specifically refers to a traditional algorithmic program that doesn't rely on probabilistic models and produces completely predictable results with every execution. In contrast to the stochastic output of LLMs, deterministic programs provide clear "pass/fail" verdicts for operations like string matching and line number verification, with no gray areas. This division of labor — "LLMs handle semantic understanding, deterministic programs handle factual verification" — is an important design pattern in current AI engineering.

The system validates these edge cases, and when validation fails, it uses Skills to guide the Agent to re-output a more precise and unique Reference. This "output → verify → feedback → re-output" loop ensures continuous improvement in citation accuracy.