VIDRAFT Open-Sources Six Projects: From GraphRAG to Quantum Error Correction
VIDRAFT open-sources six projects focused on AI output verifiability, covering GraphRAG, metacognition evaluation, local inference, and quantum benchmarks.
VIDRAFT has moved the open-source portion of its core toolchain from Hugging Face to the GitHub organization final-bench, releasing six projects at once around the theme of "making AI system outputs verifiable." ONGRID adds zero-token confidence evaluation to GraphRAG; FINAL Bench evaluates models' ability to discover and correct their own errors; Open Quantum Challenge harness publishes quantum benchmark scoring code; POCKET provides quantized models that run on laptop CPUs; ZTC judges answer trustworthiness in a single forward pass; ALL Bench Leaderboard aggregates public benchmark results. These interlocking projects provide infrastructure for building reliable AI pipelines.
VIDRAFT has moved the open-source portion of its core toolchain from Hugging Face to GitHub (github.com/final-bench), releasing six projects at once. These projects address some of the thorniest problems in current AI engineering: the trustworthiness of retrieval augmentation, model self-correction capability, localized deployment, and the transparency of quantum computing benchmarks. For developers, these aren't scattered code snippets but a set of interlocking infrastructure components.
ONGRID: A GraphRAG That "Questions" Itself
Mainstream GraphRAG tools do two things: retrieval and generation. ONGRID adds a third—judging whether the answer is trustworthy.
Usage is straightforward: after importing documents, ONGRID builds a three-dimensional ontology graph composed of typed entities and relationships. When a question is asked, the LLM answers based on facts in the graph, after which a confidence evaluator checks answer quality without generating any tokens. This "zero-token review" design means verification costs are nearly negligible—a quite practical trade-off for teams needing to deploy RAG systems at scale.
Project URL: final-bench/ongrid
GraphRAG is a variant of retrieval-augmented generation (RAG). Traditional RAG typically retrieves text fragments based on vector similarity, whereas GraphRAG first builds a knowledge graph (entities and relationships), then performs retrieval and reasoning based on the graph structure. This structure captures multi-hop relationships between entities, making it suitable for question answering that requires logical coherence. ONGRID further introduces a "three-dimensional ontology graph," which applies typing constraints to entities and relationships, enabling more precise matching during retrieval using ontology hierarchies.
Zero-token review means the evaluator generates no natural language tokens, instead directly outputting a confidence score or binary judgment. This is typically implemented via a lightweight classification head or regression head, avoiding the high latency and cost of autoregressive generation, making the verification step add almost no overhead.
FINAL Bench: Can Models Recognize Their Own Mistakes
Traditional accuracy benchmarks only care whether the final answer is correct. FINAL Bench asks a different question: can a model discover that it's wrong, admit the error, and correct it?
This benchmark covers a functional metacognition evaluation of 100 tasks across 15 domains, spanning from mathematics and medicine to ethics and economics. The paper has been published on arXiv, and the dataset is hosted on Hugging Face. Decomposing "metacognition" into a measurable task set is the most valuable part of this project—it turns "whether a model knows what it doesn't know" from a philosophical discussion into a scorable metric.
Project URL: final-bench/leaderboard
Metacognition in cognitive science refers to "cognition about one's own cognitive processes." In the AI context, it often refers to a model's ability to assess its own knowledge boundaries, identify uncertainty, and discover and correct errors. FINAL Bench operationalizes metacognition into measurable tasks: the model must not only provide an answer but also judge whether it might be wrong and correct itself when prompted. This evaluation goes beyond traditional accuracy and is closer to the reliability requirements of real-world deployment.
Open Quantum Challenge harness: Laying the Scoring Criteria Bare
The credibility of a leaderboard depends on whether the scoring code is public, so VIDRAFT directly released the scoring tool.
This harness includes reproducible scoring logic and frozen test problems, scoring quantum simulation tasks and surface code decoders, with results presented as logical error rates along with 95% Wilson confidence intervals. Anyone can reproduce numbers consistent with the leaderboard, while contestants' individual methods remain confidential. This "public scoring, private methods" structure is a reasonable approach to maintaining benchmark credibility in fields with extremely high barriers to entry like quantum computing.
Project URL: final-bench/quantum-bench-harness
Surface code is one of the most mainstream topological encoding schemes in quantum error correction. It encodes logical qubits into physical qubits on a two-dimensional grid and uses local measurements to detect errors. The decoder is an algorithm that infers the most likely error pattern from measurement syndromes, and its performance directly determines the logical error rate. Wilson confidence interval is an interval estimation method for binomial proportions that is more robust than the normal approximation when sample sizes are small or proportions are close to 0/1, and is commonly used to assess the statistical uncertainty of error rates.
POCKET: Good Models That Run on Laptops
POCKET is a series of quantized models that can run locally on laptop CPUs—no GPU or network connection required.
The repository itself is a model index plus runtime recipes, with GGUF weights hosted on Hugging Face. According to VIDRAFT, cumulative downloads of POCKET builds have exceeded two million—a number that shows local inference demand is real and substantial. For privacy-sensitive scenarios or offline environments, the value of this kind of toolchain is plain to see.
Project URL: final-bench/pocket
GGUF is a model file format designed specifically for local inference, introduced by the llama.cpp community. It supports quantizing model weights to 2–8 bits and includes metadata and tensor layouts for efficient loading and execution on CPUs. GGUF weights are typically hosted on Hugging Face, and users can load them directly through runtimes such as llama.cpp without a GPU or network connection—ideal for privacy-sensitive or offline scenarios.
ZTC: Judging Answer Trustworthiness in a Single Forward Pass
ZTC stands for Zero-Token Confidence. Its input is a question-answer pair generated by any model, and its output is whether the answer is trustworthy. The entire process completes in a single forward pass without generating any text.
It offers three sizes—4B, 9B, and 27B—progressing from CPU-friendly to highest quality. The repository includes inference code, a model index, and methodology documentation. The combination of ZTC and ONGRID is worth noting: the former performs general answer verification, while the latter enforces graph-based factual constraints. Together they can form a low-cost, high-trust verification pipeline.
Project URL: final-bench/ztc
ALL Bench Leaderboard: A Single Entry Point for Side-by-Side Comparison
Rather than hunting through scattered tables, it's better to centralize. ALL Bench Leaderboard aggregates public benchmark results, supports side-by-side model comparison, and is open-sourced under the Apache-2.0 license.
Project URL: final-bench/ALL-Bench-Leaderboard
Where to Start
VIDRAFT offers entry-point suggestions organized by scenario:
- Building a retrieval system with verification: start with ONGRID and ZTC
- Evaluating model capabilities: FINAL Bench and ALL Bench Leaderboard
- Running models offline: POCKET
- Quantum error correction research: Open Quantum Challenge harness
All projects welcome issues and PRs; if something doesn't run, you can report it directly in the repository.
Looking at the overall layout, VIDRAFT's six projects revolve around the same thread: making the outputs of AI systems verifiable. Whether it's RAG confidence judgment, model self-correction evaluation, or public scoring of quantum benchmarks, they all point to the keyword "trust." For developers looking to build reliable AI pipelines, this combination offers a starting point worth exploring.
Related articles
macOS Is No Longer UNIX: Apple Didn't Renew Certification, But the Technology Is Unchanged
Apple did not renew macOS's UNIX certification, and the Open Group has removed Apple's entry. This article analyzes the technical reality behind the expiration, Apple's possible intentions, and the practical impact on developers.
Building Stateful Multi-Agent Systems with Azure Cosmos DB
Based on the official Azure Cosmos DB tutorial, this article explains how to use the Agent Memory Toolkit to add shared persistent memory to a multi-agent travel planner, covering short-term, long-term, episodic, and semantic memory plus cross-session state continuity.
Production AI Agent Operations: A Three-Step Framework for Observing, Evaluating, and Optimizing
Azure Cosmos DB team proposes a three-step framework for production AI Agent operations: observe trace tracking, evaluate with LLM as a judge and regression testing, optimize cost across RU, retrieval, memory, and LLM layers.