BBQ
A bias benchmark dataset designed for question-answering scenarios. It evaluates model performance on social bias dimensions by constructing multiple-choice questions set in ambiguous contexts, covering social group attributes including race, gender, and age.
Timeline (last 90 days)
Elastic by default applies int8 quantization to float32 vectors to reduce memory usage, BBQ further compresses vectors to nearly one bit, and disk BBQ nearly eliminates the memory overhead of vector search.
Mainstream bias benchmarks currently include datasets such as WinoBias, StereoSet, and BBQ, which measure a model's stereotyping tendencies along dimensions such as gender and race by constructing contrastive sentence pairs or multiple-choice questions.
A common limitation shared by mainstream bias benchmarks is that they measure a model's explicit choices in specific controlled contexts, rather than implicit biases that emerge in open-ended generation, long-text reasoning, or complex role-playing scenarios.
BBQ (Bias Benchmark for QA) detects models' bias tendencies across nine social dimensions through ambiguous and disambiguating question-answer pairs.
All Facts (4)
Elastic by default applies int8 quantization to float32 vectors to reduce memory usage, BBQ further compresses vectors to nearly one bit, and disk BBQ nearly eliminates the memory overhead of vector search.
50%UnverifiedMainstream bias benchmarks currently include datasets such as WinoBias, StereoSet, and BBQ, which measure a model's stereotyping tendencies along dimensions such as gender and race by constructing contrastive sentence pairs or multiple-choice questions.
50%UnverifiedA common limitation shared by mainstream bias benchmarks is that they measure a model's explicit choices in specific controlled contexts, rather than implicit biases that emerge in open-ended generation, long-text reasoning, or complex role-playing scenarios.
50%UnverifiedBBQ (Bias Benchmark for QA) detects models' bias tendencies across nine social dimensions through ambiguous and disambiguating question-answer pairs.
50%