pr-witness
A code review assistance tool that runs crossrun—executing old tests against new code—to detect whether AI agents or developers have concealed bugs by modifying test files. It exposes structural blind spots in CI systems.
Timeline (last 90 days)
pr-witness is a tool developed by developer talhayme that runs three combinations: base tests on base code, head tests on head code, and base tests on head code (crossrun).
By running old tests on new code, crossrun can expose tests that have been quietly modified to accommodate a bug.
The development methodology of pr-witness is to first establish an annotated corpus and evaluation framework in phase zero, and not write any detectors before that.
The pr-witness corpus has 17 cases, of which 9 are cheating and 8 are honest changes that look like cheating.
pr-witness improved precision from 0.67 in the first version to 0.80 and then to 0.89 through the crossrun mechanism, while achieving zero false negatives.
The third layer of pr-witness uses deterministic templates (in English and Russian, without a model) to read PR descriptions in order to verify whether the description is consistent with the actual run results.
All Facts (6)
pr-witness is a tool developed by developer talhayme that runs three combinations: base tests on base code, head tests on head code, and base tests on head code (crossrun).
50%UnverifiedBy running old tests on new code, crossrun can expose tests that have been quietly modified to accommodate a bug.
50%UnverifiedThe development methodology of pr-witness is to first establish an annotated corpus and evaluation framework in phase zero, and not write any detectors before that.
50%UnverifiedThe pr-witness corpus has 17 cases, of which 9 are cheating and 8 are honest changes that look like cheating.
50%Unverifiedpr-witness improved precision from 0.67 in the first version to 0.80 and then to 0.89 through the crossrun mechanism, while achieving zero false negatives.
50%UnverifiedThe third layer of pr-witness uses deterministic templates (in English and Russian, without a model) to read PR descriptions in order to verify whether the description is consistent with the actual run results.
50%