COGEXT: A Local, Offline AI Agent Commitment Extraction Tool — No API Key Required

How Should AI Agent "Commitments" Be Tracked?

AI agents are increasingly embedded in real-world workflows, and they make all kinds of commitments — things like "I'll send the report to Sarah by Friday EOD." The problem is that whether these commitments are actually fulfilled often lacks a reliable verification mechanism. Developer yaminbinyoosuf shared his ongoing project COGEXT on Reddit, aiming to build an accountability layer for AI agents.

The core idea is straightforward: agents make actionable commitments, yet existing systems struggle to track whether those commitments are honored. COGEXT's goal is to take these promises scattered throughout natural language and make them structured and trackable.

A Local Solution You Can Run with a Single pip install

The author's latest release is a fully local, offline commitment extractor called cogext-primitive. Its biggest selling point is what it doesn't need: no API key, no cloud, no sign-up. The usage is refreshingly simple:

pip install cogext-primitive

from cogext_primitive import extract_commitments

commitments = extract_commitments(
    "I'll send the report to Sarah by Friday EOD."
)

for c in commitments:
    print(f"{c.action} {c.object} to {c.recipient} by {c.deadline}")

This code extracts the key elements of a commitment from a single natural-language sentence: the action, object, recipient, and deadline. The entire process runs locally with no dependency on external services — a pragmatic choice for developers who care about data privacy or need offline deployment.

State Machine–Driven Commitment Tracking

Extraction is just the first step. COGEXT also tracks the full lifecycle of each commitment through a state machine: open → due → overdue → fulfilled. This design means commitments aren't just identified — they're continuously monitored. That's the accountability layer in action.

A State Machine is a classic model in computer science for describing how a system transitions between different states. It consists of a finite set of states, events that trigger transitions between them, and actions executed during each transition. In COGEXT's design, every extracted commitment is an independent state machine instance: it starts as open, automatically transitions to due as the deadline approaches, moves to overdue if not completed after the deadline passes, and jumps to fulfilled once a completion signal is detected. The advantage of this design is that transition logic is transparent and auditable. External systems can easily subscribe to specific state-change events (e.g., triggering an alert when a commitment enters overdue), and the model supports sharing commitment state across agents in multi-agent collaboration scenarios, providing a unified visibility layer across the entire workflow.

Deliberate Design Trade-offs

What this tool intentionally skips is worth paying attention to. The author explicitly states that the extractor will pass on three categories:

• Compound promises within a single sentence
• Pronoun-shaped objects
• Anything vague

This isn't a capability gap — it's a deliberate design philosophy. The author describes the extractor as "precision-tuned" rather than "recall-tuned." In other words, it would rather extract less and guarantee that every extracted commitment is clear and verifiable, rather than pollute results with ambiguous language.

For a system built around accountability, this trade-off makes sense: incorrectly tracking a commitment that was never actually made is more costly than missing a vague one.

Precision and Recall are two core metrics for evaluating extraction and classification systems in information retrieval and machine learning. Precision measures "how many of the extracted results are actually correct," while recall measures "how many of the results that should have been extracted were actually found." The two are typically in tension: improving precision means being more conservative, which can miss edge cases; improving recall means capturing more aggressively, which can introduce noise. COGEXT explicitly chooses precision over recall — it would rather have false negatives than false positives. For an accountability system, the consequence of a false positive — holding an agent responsible for a commitment it never actually made — directly undermines user trust in the system, so this trade-off has a clear logical basis at the product level.

The Data Behind the Design

To support this design decision, the author also published an audit of 120 agent outputs, available on their research page. This analysis serves as the empirical foundation for the precision-first strategy — by observing the actual shape of commitments in real agent outputs, it informs what should be extracted and what should be skipped.

The project is open source under the MIT license and completely free:

• GitHub: github.com/yaminbinyoosuf/cogext-primitive
• PyPI: pip install cogext-primitive

Summary and Takeaways

COGEXT addresses a genuinely emerging need: as AI agents take on more autonomous tasks, making them "say what they mean and do what they say" is becoming part of reliability engineering. Structuring commitments from free text and tracking their fulfillment with a state machine is a lightweight but targeted approach.

The author is also soliciting feedback from the community, looking to understand what "commitment patterns" developers are actually observing from their own agents. For teams building multi-agent systems or needing to audit AI outputs, this local, zero-dependency tool is worth trying out — it doesn't solve everything, but it solves one specific problem in a clear and controllable way.