AdversaryGate
A merge gate tool for code merge scenarios involving AI coding agents. By running test suites against the baseline and the patch, it issues one of three verdicts, MERGE, BLOCK or INCONCLUSIVE, to assess the test adequacy of a patch, and supports verification mechanisms such as mutation testing and annotation canaries.
Timeline (last 90 days)
The merge gate tool AdversaryGate, released by developer Sanflow10, shipped six patch versions (3.0.0 to 3.0.6) in four days.
Almost all of AdversaryGate's vulnerabilities stem from the same pattern: fail-open by omission, that is, when input falls outside the measurement range, the gate silently skips it without marking it as INCONCLUSIVE.
AdversaryGate's adjudication logic outputs three results: MERGE (exit code 0), BLOCK (exit code 1), INCONCLUSIVE (exit code 2).
AdversaryGate's design principle is: a measurement that no one has actually performed cannot be used to pass the gate.
AG-046 vulnerability: a test that hashes changed files would fail on any edit, killing all mutants and making mutation strength compute as 1.0; 3.0.0 fixed it by introducing a comment canary.
Vulnerability fixed in 3.0.1: claims comparing ASTs or bytecode could still pass via the comment canary; subsequently the canary added no-op assignments to change the syntax tree and bytecode, but this was a partial fix, and claims that read only a smaller fragment of the syntax tree could still pass.
Vulnerability fixed in 3.0.1: after a .py file is marked as binary in .gitattributes, git diff outputs 'Binary files ... differ', the changed lines disappear from the coverage diff, and the patch goes straight into MERGE; after the fix, such diffs are rejected.
Deleting one line of the README once made coverage become 1.0; after the fix, patches with no code lines are marked as unmeasured for coverage, and cannot obtain MERGE on the basis of coverage.
3.0.4 fixed two problems: same-line branches (such as return 999 if x == 0 else x + 1) generating branch mutants whose survivors trigger INCONCLUSIVE; and the fact that previously only the first 12 changed lines were mutated, whereas now every changed line is broken at least once.
AG-047 vulnerability (fixed in 3.0.6): the gate judged what was a test based solely on path, so modules named like tests but imported by the code under test were excluded from coverage and mutation; after the fix such modules are treated as source code, but modules dynamically imported via importlib or __import__ still cannot be detected.
4 more timeline events
All Facts (14)
Almost all of AdversaryGate's vulnerabilities stem from the same pattern: fail-open by omission, that is, when input falls outside the measurement range, the gate silently skips it without marking it as INCONCLUSIVE.
50%UnverifiedVulnerability fixed in 3.0.1: claims comparing ASTs or bytecode could still pass via the comment canary; subsequently the canary added no-op assignments to change the syntax tree and bytecode, but this was a partial fix, and claims that read only a smaller fragment of the syntax tree could still pass.
50%UnverifiedAdversaryGate's adjudication logic outputs three results: MERGE (exit code 0), BLOCK (exit code 1), INCONCLUSIVE (exit code 2).
50%UnverifiedAdversaryGate's design principle is: a measurement that no one has actually performed cannot be used to pass the gate.
50%UnverifiedAdversaryGate's known limitations include: measurement supports only Python and pytest, and other languages return INCONCLUSIVE rather than a speculative MERGE; it cannot prevent an Agent running as the same OS user from accessing the gate's temporary directory; and the canary fix is partial.
50%UnverifiedEvery vulnerability of AdversaryGate is documented in the public ERRORS_AND_INCONSISTENCIES.md, including reproduction methods and fixes.
50%UnverifiedThe problems where honest patches were misjudged as INCONCLUSIVE include: the '\ No newline at end of file' marker causing line number offsets, the MCP's 900-second timeout being shorter than the full test suite runtime, and the default memory limit on Android (Termux) causing the process to be aborted; these problems were fixed in 3.0.5.
50%UnverifiedVersions 3.0.2 and 3.0.3 were released with an API token, the MCP package stayed at 3.0.0, and the core files had no attestation; the author removed all versions before 3.0.4 from PyPI and GitHub, and the release process is now signed tag -> Trusted Publishing -> both packages carry PyPI attestation.
50%UnverifiedDeleting one line of the README once made coverage become 1.0; after the fix, patches with no code lines are marked as unmeasured for coverage, and cannot obtain MERGE on the basis of coverage.
50%UnverifiedAG-046 vulnerability: a test that hashes changed files would fail on any edit, killing all mutants and making mutation strength compute as 1.0; 3.0.0 fixed it by introducing a comment canary.
50%UnverifiedVulnerability fixed in 3.0.1: after a .py file is marked as binary in .gitattributes, git diff outputs 'Binary files ... differ', the changed lines disappear from the coverage diff, and the patch goes straight into MERGE; after the fix, such diffs are rejected.
50%UnverifiedThe merge gate tool AdversaryGate, released by developer Sanflow10, shipped six patch versions (3.0.0 to 3.0.6) in four days.
50%Unverified3.0.4 fixed two problems: same-line branches (such as return 999 if x == 0 else x + 1) generating branch mutants whose survivors trigger INCONCLUSIVE; and the fact that previously only the first 12 changed lines were mutated, whereas now every changed line is broken at least once.
50%UnverifiedAG-047 vulnerability (fixed in 3.0.6): the gate judged what was a test based solely on path, so modules named like tests but imported by the code under test were excluded from coverage and mutation; after the fix such modules are treated as source code, but modules dynamically imported via importlib or __import__ still cannot be detected.
50%