BBH
A benchmark composed of 23 tasks selected from BIG-Bench on which mainstream models perform significantly below random baseline. It is designed to evaluate capabilities that are difficult to solve through data memorization, including logical reasoning and symbolic manipulation.
Timeline (last 90 days)
On a real LLM (DeepSeek), the researchers evaluated 50 samples each on the GSM8K, MATH, and BBH benchmarks, and GraphEvoResearcher achieved accuracies of 98%, 92%, and 82% respectively, the highest among all baselines.
GSM8K, MATH, and BBH are three commonly used reasoning benchmarks with a clear difficulty gradient: GSM8K consists of elementary school math word problems, MATH covers competition-level math problems, and BBH contains 23 task types requiring multi-step symbolic manipulation or strategic reasoning.
在全部 38 项基准中，Aplomb 1 在 8 项上拿到 5.3B 以内模型最高分，包括 MMLU-Pro、GPQA Diamond 和 BBH
All Facts (3)
On a real LLM (DeepSeek), the researchers evaluated 50 samples each on the GSM8K, MATH, and BBH benchmarks, and GraphEvoResearcher achieved accuracies of 98%, 92%, and 82% respectively, the highest among all baselines.
50%UnverifiedGSM8K, MATH, and BBH are three commonly used reasoning benchmarks with a clear difficulty gradient: GSM8K consists of elementary school math word problems, MATH covers competition-level math problems, and BBH contains 23 task types requiring multi-step symbolic manipulation or strategic reasoning.
50%Unverified在全部 38 项基准中，Aplomb 1 在 8 项上拿到 5.3B 以内模型最高分，包括 MMLU-Pro、GPQA Diamond 和 BBH
50%