golden set
A set of test cases locked and made immutable before an experiment. Each case contains a stable identifier, an exact prompt, and a deterministic checker, and file hashes are used to guarantee version consistency.
Timeline (last 90 days)
A golden set is only meaningful if it is immutable during experiments, which is done by writing the cases to disk, hashing the files and recording the hash in every report.
A golden set refers to a collection of test cases that have been manually confirmed and carry authoritative labels; it is the "ruler" of the evaluation pipeline, used to measure the absolute quality of the system under test and to calibrate reviewer models.
All Facts (2)
A golden set is only meaningful if it is immutable during experiments, which is done by writing the cases to disk, hashing the files and recording the hash in every report.
50%UnverifiedA golden set refers to a collection of test cases that have been manually confirmed and carry authoritative labels; it is the "ruler" of the evaluation pipeline, used to measure the absolute quality of the system under test and to calibrate reviewer models.
50%