Ratified

Answers to questions like revision lists and counts are not in any text but in the graph structure, which constitutes a reason to use a graph database, though strictly speaking it is not a failure of retrieval.

Ratified's hard rule is that every answer must end with a link a stranger can open.

Ratified's answers come from graph traversal rather than model memory; no model writes Cypher, and a deterministic router picks one handwritten query for each question; if the traversal returns nothing, the card plainly says "the graph doesn't know."

Ratified contrasts with mainstream RAG approaches: RAG hands retrieved text chunks to the model to freely compose an answer, whereas Ratified replaces generative inference with structured graph paths, putting verifiability first.

The team scraped 3,078 emails from the TLS list (1,324 of them containing full text), 633 threads and 403 participants, with each email linked to its author and thread.

The team loaded 113 drafts and their 839 revisions from the IETF datatracker, chained them with SUPERSEDED, and connected them to the 66 RFCs they eventually evolved into.

ALIGNS_SECTION edges map each section of each revision to the RFC section it became, matching on heading text and recording whether the text is still changing (less than 98% similarity to the final published version).

For §4.2.1, the Cypher query returns the chain: email→thread→draft -24→-25→-26→-27→-28→RFC 8446→§4.2.1.

The Ratified project aims to verify whether a graph database can automatically link email threads and ensure every step is verifiable, choosing a public mailing list as its testbed.

The fix rule: an email counts only if it was written during the period when the revision it points to was still current (before the next revision or the RFC's publication month).

After the fix, §4.2.1 yields the answer of 6 emails, 5 people, March 2018, all written during the period when that clause still differed from the final version.

In the "who knows what" ranking, the draft editor ranks first on both metrics (PageRank and Betweenness), and a chair ranks second.

Building a full-text index on section headings allows natural-language questions to be resolved to sections, but indexing section body text actually made results worse, because broad overview sections overwhelm precise headings, so in the end only headings were indexed.

Each answer card has a panel listing the Cypher actually executed by FalkorDB, the parameters, the row count, and the database's self-reported run_time_ms; for the §4.2.1 question, for example, there are 4 queries totaling under 1 millisecond on a laptop.

When asked about content that was never ingested (such as IESG votes or meeting minutes), Ratified returns a gray card saying "the graph doesn't know," which the authors consider the feature they trust most.

On 13 questions, the graph approach (Ratified) answered 13/13 correctly; a BM25 plus bge-small retriever on the same text hit exactly 1/10, placed within the top ten 3/10, and three counting questions could not be scored.

The authors acknowledge the comparison favors them; the fair version: on the three questions where the evidence actually exists in text visible to the retriever, the graph scores 3/3 and retrieval 1/3.

First-version flaw: a name, "RFC Errata System," was listed as someone who had "discussed" §4.2.10 before its publication, when in fact the errata report was filed years later.