BIG-Bench Hard
BIG-Bench Hard is a set of 23 of the most challenging tasks selected from the BIG-Bench benchmark, used to evaluate the complex reasoning capabilities of large language models.
Timeline (last 90 days)
On Llama-3.2-3B, fine-tuning a brand-new dense model with the 5% of data selected by CACTUS reached 49.44% accuracy on BIG-Bench Hard; selecting data directly with the dense model gave 49.54%; data selected by the ALPS method could only reach 47.22%.
评估大语言模型推理可靠性的系统化基准测试包括GSM8K、MATH竞赛题集和BIG-Bench Hard
All Facts (2)
On Llama-3.2-3B, fine-tuning a brand-new dense model with the 5% of data selected by CACTUS reached 49.44% accuracy on BIG-Bench Hard; selecting data directly with the dense model gave 49.54%; data selected by the ALPS method could only reach 47.22%.
50%Unverified评估大语言模型推理可靠性的系统化基准测试包括GSM8K、MATH竞赛题集和BIG-Bench Hard
50%