Tired of Babysitting AI Coding Agents? This Open-Source Tool Makes Them Self-Check

AI is getting better and better at writing code, but the "ship it as soon as it's written" habit hasn't changed one bit. Did it run the tests? Did it actually fix the bug? Did it check that file? One developer got fed up with this repeated acceptance loop, so he built an open-source configuration set that makes agents pass their own checks before they can say "done."

Why "Done" Is Becoming Less and Less Trustworthy

The original poster put it plainly: models have gotten absurdly good at writing code, but he still spends a huge amount of time confirming whether the agent actually did what it said. He named three common failure modes one by one — replying "Done!" when the tests were never run; replying "Fixed!" when the same bug comes right back; replying "Everything looks good!" while conveniently missing the one file that was never checked.

The pain point here isn't model capability — it's that the cost of acceptance has shifted onto the human. Every time you switch to a new project, you have to explain the same coding standards, testing requirements, and workflows all over again. The author calls this repetitive labor "babysitting," and the metaphor is apt: the agent does the work, the human is responsible for catching what falls through, and catching things often ends up being more exhausting than doing the work.

What Is Agent Config: An Open-Source Configuration Set That "Lays Down the Rules" for Agents

The author distilled these needs into a project — Agent Config, positioned as "your AI engineering system," open-sourced under the MIT license with no paid tier. It's not a new coding model, but a collection of skills, rules, workflows, and quality checks layered on top of Claude Code and several other coding tools.

Based on the capabilities the author lists, the design philosophy breaks down into a few parts. Look before you touch: the agent analyzes the codebase before changing code, avoiding blind edits. End-to-end workflow: planning, implementation, testing, and review are chained into a single line, rather than letting the agent jump straight to writing code. Multi-perspective review: it approaches from four angles — bugs, security, testing, and code quality — rather than a vague "take a look." Memory and project context: prior decisions are stored so you don't have to rediscover them every time. On-demand loading: skills and rules load based on the current task, rather than stuffing everything into the context all at once. It also supports multiple coding tools, preventing the whole configuration set from being locked into a single vendor.

What the author values most is the production validator. It can be used standalone, specifically to test whether the agent's "done" actually holds up. This gets at the core value of the whole approach: partially taking the act of "acceptance" out of human hands and turning it into a fixed step in the agent's delivery pipeline.

Claude Code is a command-line coding agent tool from Anthropic that can read codebases, execute commands, and modify files directly in the terminal, interacting with developers in natural language. Unlike traditional code completion plugins, Claude Code is an "agentic" tool: it autonomously plans steps, calls toolchains, and completes multi-file changes. These tools have a very high capability ceiling, but precisely because of that, their "completion claims" need external verification — an agent can claim a task is done without actually running tests or checking all relevant files. A configuration layer like Agent Config works on top of Claude Code's rule and skill injection mechanisms, constraining agent behavior through preset workflows and quality gates. Understanding this makes it clear why the author emphasizes that it's "a layer wrapped around the tool," rather than a replacement for the model itself.

On-demand loading (lazy loading) is a key design pattern in agent engineering. Coding agents have limited context windows, and if you stuff all coding standards, testing requirements, and project conventions into the prompt at once, it crowds out genuinely useful code and task information, causing the model's attention to scatter and costs to rise. The idea behind on-demand loading is: split rules and skills into independent modules, and dynamically inject the relevant parts based on the current task type (e.g., "write tests," "security review," "refactor"). This reduces the token cost of each call and also reduces interference between rules. When the author mentions that "skills and rules load based on the current task, rather than stuffing everything into the context all at once," this is exactly that pattern in action. It's also a direct means of countering context bloat, consistent with the author's wariness of complexity.

More Agents Doesn't Mean Better Results: The Author's Own Restraint

What's worth noting is the author's wariness of complexity. He explicitly writes that he's working hard to keep this thing from becoming a "token-burning monster" — more agents, more rules, and more context don't automatically lead to better results. The project has already grown beyond his initial vision, and he admits he's still figuring out where added complexity is genuinely useful and where it's just noise.

This kind of self-doubt is rare in open-source project introductions, but it's a useful reminder for users: context bloat itself is a cost. Loading skills on demand and injecting rules by task type is essentially fighting the urge to pile in every rule. If readers plan to build something similar themselves, this lesson may be more worth learning from than the feature list.

Context bloat refers to the phenomenon where an agent's prompt or conversation history accumulates too much irrelevant information, leading to degraded model performance and higher inference costs. In coding agent scenarios, common triggers include: stuffing entire codebase summaries into the context, stacking multiple layers of rules, retaining lengthy conversation histories, and introducing multiple sub-agents each carrying their own independent context. Context bloat doesn't just burn money — it also causes the model to "get distracted" on key decisions, with important rules drowned out by noise. Common industry strategies include: on-demand loading, summarizing and compressing history, externalizing long-term memory into searchable storage, and limiting the number and scope of sub-agents. The author admits the project has grown beyond his initial vision and is working hard to avoid becoming a "token-burning monster," which shows he recognizes that complexity doesn't yield linear returns — every added layer of rules needs to prove its marginal value.

The Trust Question: Do You Accept the Agent's Work, or Does the Agent Accept Its Own?

The original post ends by throwing two questions back to the community: Do you unconditionally trust an agent when it says "task complete," or have you already built your own checking system? And if you're already using a similar configuration, what actually works?

There are no standard answers to these questions, but they mark a watershed moment for AI coding tools today. When a model's ability to write code is no longer the bottleneck, the engineering focus shifts to how to verify an agent's output. Agent Config's answer is to build the checking process into the agent's workflow, making "self-checking" a default action rather than an after-the-fact补救 (remedy). Whether this direction can be generalized depends on how ruthless the validator itself is — a validator that only says "looks good" is barely different from no validator at all.

For developers heavily using Claude Code or similar tools, this project offers a reference starting point: rather than giving verbal reminders every time, write the standards as checkpoints the agent must pass through. As for how strict the checkpoints should be and how many rules to configure, even the author is still experimenting — which is probably just the norm for AI coding workflows at this stage.

GitHub: https://github.com/event4u-app/agent-config