A 68% Failure Rate by Design: Inside OpenAmer's Self-Verifying Outcome Ledger

Most AI agent projects showcase their successes — polished demos, hand-picked case studies, dashboards full of green metrics. OpenAmer takes the opposite approach: it publicly records every single run, including the majority that fail.

OpenAmer is a self-verifying AI agent running on a CPU-only Windows laptop. It maintains a public outcome ledger at memory/si/outcome_ledger.jsonl. Every run is logged as a single line containing a timestamp, the task, the objective, the agent that executed it, and a passed: true/false verdict.

What's surprising is this: the ledger doesn't hide failures — and failures vastly outnumber successes.

The Numbers: A 31.9% Pass Rate

According to data read directly from the ledger by the project's maintainers:

Metric	Value
Total recorded outcomes	1,701
Passed	543
Failed	1,158
Pass rate	31.9%

A nearly 70% failure rate would be virtually unthinkable in conventional product marketing. But the project's position is clear: this pass rate isn't a flaw to be papered over — it's the entire point of the design. A system that only records successes can never tell you why things go wrong. And "why" is the only basis for making the next run better.

This kind of candor is itself a differentiator. In an industry saturated with carefully packaged capability claims, a project willing to put its failure rate on the label becomes far easier to trust in everything else it says.

Every Failure Comes with a Reason

The key feature of the OpenAmer ledger isn't just that failures are recorded — it's that each failure includes a structured why field, not merely a passed: false flag. Failure causes are categorized into human-readable entries such as:

• Task timeout
• Tool returned an unexpected data structure
• The path chosen by the agent failed to reach the objective
• A dependency was missing at execution time

This design makes failures analyzable. A pile of unexplained red entries is just noise. Failures with attached causes are signals that can be consumed and fed back into the system.

Ledger-First Writes: A Mechanism Against Duplicate Side Effects

One detail worth noting for technical readers is the write order. Ledger entries are written before side effects occur, and side effects are only committed once the ledger row has been successfully persisted. This ordering creates an anti-duplication guarantee:

An effect may land without a corresponding receipt — but a receipt can never exist for an effect that never happened. In other words, "effect landed, receipt missing" is itself a recorded terminal state, not a silently ignored gap. This mirrors the classic engineering tradeoff between "at-least-once" and "exactly-once" semantics in distributed systems, reflecting a deliberate and rigorous approach to reliability.

Background: "At-least-once" and "exactly-once" are foundational concepts in distributed systems describing message or operation delivery guarantees. At-least-once ensures an operation will eventually execute but may produce duplicates on retry; exactly-once adds deduplication on top of that, at higher implementation cost — typically requiring idempotency design or two-phase commit. OpenAmer opts for the more conservative at-least-once approach: it tolerates the detectable inconsistency of "effect landed but receipt missing" rather than allowing the harder-to-trace "receipt exists but effect never executed" ghost state. The practical upside: operators debugging issues only need to scan the ledger for orphaned entries with no corresponding side effect, rather than hunting for unknown execution artifacts in the wild.

Why Publish the Failures

The project offers three reasons, ranked by importance:

First, self-measurement without failure records is just optimism. A dashboard that only shows green is decoration. The ledger's value comes precisely from the fact that its red entries are real and countable.

Second, the failure distribution is the training signal. The self-improvement loop reads from this very ledger, and failures dominate it. Improving the pass rate means tackling that 68% — not congratulating yourself on the 32% that worked. It's a straightforward logic that's easy to overlook: the room for improvement always lives in the failures.

Third, anyone can verify it. The ledger file sits in the memory directory of the repository. You can count the lines yourself. There are no capability claims here that can't be verified with a single command.

This "verifiability-first" posture is especially rare at a time when AI product capability inflation is the norm.

What It Isn't

The project's language is deliberately restrained. OpenAmer doesn't claim to be the best agent in the world, or even to be particularly good. What it makes is a narrow, specific, verifiable claim:

This agent can accurately tell you how often it fails, and why — in writing, in a file you can read right now.

That's a far lower bar than "world's best" — but it's a bar the project can actually clear. This kind of self-awareness is itself a mark of engineering maturity: rather than promising what it can't deliver, it commits to one thing it can do and that anyone can audit.

Open Source and Reproducible

OpenAmer is hosted at github.com/openamer/openamer under the Apache-2.0 license. It runs on CPU only, is Windows-native, and includes self-improvement capabilities. The ledger file is at memory/si/outcome_ledger.jsonl.

One caveat: the numbers above change as the agent continues to run — the first entry is dated 2026-09-28 and the last 2026-10-11. The project maintainers themselves emphasize: check the file directly before citing any of these figures. This reminder about the shelf life of its own data is yet another expression of the project's core philosophy: everything should be grounded in verifiable fact.

Background: Apache-2.0 is a permissive open-source license that allows free use, modification, and distribution of code — including integration into commercial products — with the only requirement being that the original copyright notice and license text are preserved. Unlike the GPL family of licenses, Apache-2.0 does not require derivative works to be released under the same terms, meaning downstream developers can embed the code in closed-source projects without triggering license propagation. For developers looking to build their own agents on top of OpenAmer or reproduce its ledger mechanism, Apache-2.0 means virtually no legal barriers to adoption.

Summary

OpenAmer offers a concrete example of what AI agent transparency can look like: instead of manufacturing an illusion of capability through cherry-picked successes, it writes every run — failures included — into a ledger that anyone can inspect. A 31.9% pass rate isn't pretty, but it's real. Failure records with root causes aren't flattering, but they're the starting point for improvement. For developers who care about AI agent reliability and observability, this design philosophy is worth studying.