Frozen Record
A governance mechanism for CI test-skip decisions that requires any skipped test to be accompanied by structured evidence independently verifiable by reviewers. The evidence must include fields such as test_id, fixture hash, run divergence, and oracle file. Any missing or tampered field triggers a rejection, preventing AI-generated code patches from concealing genuine regressions.
Timeline (last 90 days)
The core idea of the frozen record mechanism is that skipping a test must be accompanied by independently verifiable evidence, and this check defaults to failure (fail closed)
A skip request is rejected if any field is missing, if the results of two runs are consistent, or if the basis for the determination has been modified by the same patch
A valid frozen record has six mandatory fields: test_id, fixture_path, fixture_sha256, disagreement, oracle_path, and oracle_blob — none can be omitted
The frozen record object should be stored in .freeze/record.json so that CI and reviewers open the same file, rather than being buried in Pull Request comments
Two consistent results (both passing or both failing) cannot serve as valid evidence for skipping a test; only a 'disagreement' — a change in result with the fixture bytes unchanged — constitutes the minimum valid observation
The mechanism deliberately errs on the side of letting rare flaky tests stay red rather than isolating a stable bug after a lucky re-run
Tests that call the network, the clock, or shared services are outside the scope of the frozen record mechanism, because file hashes cannot explain why they produce disagreements
The mechanism follows a strictly ordered five-step process that stops at the first rejection, and no skip marker may be added before 'accepted' is printed in step 5
Two samples are insufficient to estimate the flakiness rate; only the fact that the two results differed once is detected; the parent blob rule also does not evaluate assertion strength
The original text explicitly states that the commands, checker scripts, and sample property files therein were not actually executed, and the hash values in the text are all placeholders.
All Facts (10)
The core idea of the frozen record mechanism is that skipping a test must be accompanied by independently verifiable evidence, and this check defaults to failure (fail closed)
50%UnverifiedA skip request is rejected if any field is missing, if the results of two runs are consistent, or if the basis for the determination has been modified by the same patch
50%UnverifiedA valid frozen record has six mandatory fields: test_id, fixture_path, fixture_sha256, disagreement, oracle_path, and oracle_blob — none can be omitted
50%UnverifiedThe frozen record object should be stored in .freeze/record.json so that CI and reviewers open the same file, rather than being buried in Pull Request comments
50%UnverifiedTwo consistent results (both passing or both failing) cannot serve as valid evidence for skipping a test; only a 'disagreement' — a change in result with the fixture bytes unchanged — constitutes the minimum valid observation
50%UnverifiedThe mechanism deliberately errs on the side of letting rare flaky tests stay red rather than isolating a stable bug after a lucky re-run
50%UnverifiedTests that call the network, the clock, or shared services are outside the scope of the frozen record mechanism, because file hashes cannot explain why they produce disagreements
50%UnverifiedThe mechanism follows a strictly ordered five-step process that stops at the first rejection, and no skip marker may be added before 'accepted' is printed in step 5
50%UnverifiedTwo samples are insufficient to estimate the flakiness rate; only the fact that the two results differed once is detected; the parent blob rule also does not evaluate assertion strength
50%UnverifiedThe original text explicitly states that the commands, checker scripts, and sample property files therein were not actually executed, and the hash values in the text are all placeholders.
50%