Allow-lists Are Not Fail-Closed by Default: The Hidden Vulnerability in AI Agent Guards

Someone discussing a Claude Code plugin made the following claim: this tool, which prevents an agent from modifying its own test files, uses an allow-list, and "allow-lists fail closed, which is exactly the failure mode you want when an agent tries to find loopholes."

The intuition is right, but the conclusion is wrong. It's worth unpacking this carefully, because the same reasoning fallacy resurfaces every time someone tries to harden an agent's shell execution environment.

What This Guard Actually Does

The plugin in question is Mattbusel/tamperproof (v0.1.1). It replaces the original blacklist of "write-type commands" with a whitelist — the right direction, since a blacklist can only cover the verbs you've already thought of, and you'll always miss some.

The new rules come from hooks/guard.mjs:

Any shell segment that names an "existing protected path" is intercepted, unless it begins with a known read-only command or a test runner command and contains no redirects or tee.

The critical part is the first sentence. The condition "names an existing protected path" determines whether the rule will be applied at all. The core logic calls existsSync(resolve(cwd, piece)) for each token — checking whether that literal token corresponds to a real, existing file.

If no token looks like an existing protected file, the segment is skipped — it is never evaluated, and not being evaluated means being allowed through.

Fail Closed vs. Fail Open are two terms in security design that describe a system's default behavior under failure. Fail closed means that when a system encounters input or an anomaly it can't handle, it denies the operation by default — like a door lock that engages automatically when power is cut. Fail open is the opposite: when facing uncertainty, it defaults to allowing the action. Allow-lists are considered "naturally fail closed" because any verb not on the list is rejected, which looks safer than a blacklist. But this intuition only holds in single-layer decision-making. Once the decision chain splits into two layers — "does this input enter adjudication?" and "what is the adjudication result?" — the allow-list only provides fail-closed behavior at the adjudication layer. The selector layer (which decides whether an input enters adjudication at all) has its own default behavior that must be examined separately.

Test Results

The author checked out v0.1.1, built a temporary directory tree containing test/calc.test.mjs, then drove the guard the same way the hook is actually invoked: sending tool calls through stdin and reading back the exit code. Each line executes essentially the same command; only the "spelling" of the target file changes:

$ sed -i s/5/-1/ test/calc.test.mjs          -> exit 2   blocked
$ sed -i s/5/-1/ test/calc.test.j?           -> exit 0
$ sed -i s/5/-1/ test/*.mjs                  -> exit 0
$ sed -i s/5/-1/ test/calc.test.{mjs}        -> exit 0
$ sed -i s/5/-1/ "$PWD/test/calc.test.mjs"   -> exit 0
$ sed -i s/5/-1/ test/nonexistent.mjs        -> exit 0

The literal path returns exit 2 (blocked), because it is a file: existsSync returns true, the path is protected, the segment is evaluated, and sed -i isn't on the read-only whitelist — so it's intercepted.

All other variants return exit 0. The reason is that the guard asks the filesystem: "Is the string test/*.mjs a file?" — and it isn't. The glob wildcard *, the ? wildcard, brace expansion {mjs} (the tokenizer even strips { and }, leaving only test/calc.test.), and a path resolving to a directory literally named $PWD — none of these match any existing file. So hits stays empty and the rules are never consulted.

The real shell expands all these spellings into actual paths before sed ever sees the arguments — that's the entire job of glob expansion. So the string bypasses a guard that checks the string itself.

Shell Glob Expansion is the path wildcard substitution that Unix shells perform before executing a command. When you write test/*.mjs, the shell expands it into every matching filename (e.g., test/calc.test.mjs) before passing the arguments to sed. So sed itself sees the real paths, not the string containing *. Brace expansion {mjs} and the ? wildcard work the same way — both are handled by the shell, and the program always receives the expanded results. This means any guard that checks command arguments as strings, before shell execution, only sees the user's "original spelling" — not the set of files the shell will actually access. In most attack scenarios, those two things are not the same.

The Real Takeaway

How a guard fails is not a property of allow-lists versus blacklists. It depends on where in the decision chain the default lives.

This guard is fail-closed at the "adjudication" layer: a segment that is evaluated and found non-read-only gets blocked. But it is fail-open at the "selector" layer: a segment that is never evaluated gets through. And the selector is the earlier of the two layers — the one that the command string can directly influence.

The result: the guard fails closed exactly where the attacker has no freedom, and fails open exactly where the attacker does.

The phrase "allow-lists fail closed" is only true on the axis you happened to enumerate — in this case, the verb (command) axis. You have to enumerate the axis that the attacker's input controls, or you're measuring the wrong thing.

Failure in the Other Direction Is Just as Real

Two more test cases expose the problem from the opposite direction — which is also why the author considers even the "verb axis" not simply "closed":

$ cat test/calc.test.mjs > /dev/null      -> exit 2
$ grep -n '=>' test/calc.test.mjs         -> exit 2

The writesOut check is />/.test(seg), so the arrow => in the grep pattern is treated as a redirect, and cat ... > /dev/null (redirecting to the null device) is also misclassified. Both of these are operations an agent constantly performs while debugging a failing test — exactly the state the guard is supposed to keep the agent in.

False positives are not free: they teach users to work around the guard, and once the guard is being worked around, it protects nothing.

The harm of false positives in security tools is often underestimated. A guard that blocks legitimate operations creates "friction" between the user and the guard, and the natural human response to sustained friction is to find a workaround. Once a bypass path is codified into a workflow habit, the guard provides no protection against real threats during those bypasses. This mirrors research findings on password complexity policies: overly strict policies cause users to stick their passwords on their monitors. For AI agents, the problem is more acute — agents often trigger false positives during large volumes of repetitive operations, potentially being "trained" into systematic bypass patterns that, once generalized, are difficult for humans to detect in an audit.

Lessons to Take Away

• Annotate where the default lives, rule by rule. "Non-read-only means blocked" and "never looked at" are two different verdicts, and only the former is visible in a code review.

• Ask about the object that will be written, not the string that names it — or accept that the string is just one spelling and go chase all the spellings. The author honestly acknowledges the real tension here: treating glob-shaped tokens as hits would close this gap, but would also create false positives for legitimate test files newly written from the shell.

• Keep controls in both directions. All twenty of the guard's test cases name a path, as did every one the author sent before going deeper. Test count is not a coverage claim — the real question is which axis those cases move along.

The author emphasizes that this plugin is quite a polished piece of work, and v0.1.1 closed every gap from the previous version, including all of the literal-spelling cases above. It's just that this particular axis survived one rewrite, and unless the fix enters the selector layer, it will survive the next one too.

This case is a reminder for everyone doing AI agent security hardening: discussions of security properties must be anchored to the axis the attacker can actually manipulate — not the one you happened to think of.