Benchmarkconstraint satisfaction benchmark
Constraint-Satisfaction Benchmark
A type of benchmark for evaluating model capabilities that requires a model to satisfy multiple hard constraints simultaneously, such as format, prohibited words and logical conditions. It is typically used to test the precise control and planning abilities of language models.
Timeline (last 90 days)
Oct 20
A constraint satisfaction benchmark gives multiple hard rules, and the model must satisfy all constraints simultaneously to pass; it breaks capability down into discrete, automatically verifiable conditions, reducing evaluator subjective bias, but the sample size per question is small and coverage is limited, making it better suited as a qualitative signal than as a basis for ranking.
Unverified50%