BioMysteryBench
An AI reasoning evaluation benchmark targeting specialized domains such as biomedicine. It is designed to assess models' deep domain knowledge and multi-step reasoning capabilities in complex, domain-specific problem settings.
Timeline (last 90 days)
On the BioMysteryBench bioinformatics task, the average problem-solving rate with occupational profiles was 46.7%, while the minimal baseline reached 56.7%, a gap of -10.0 percentage points (95% interval [-16.7, -3.3]).
The decline in problem-solving rate under the occupational profile condition was primarily caused by more frequent interruptions due to token limit and time limit triggers.
A comprehensive integrity audit report targeting three major benchmarks—BioMysteryBench, Terminal-Bench 2.1, and SWE-bench Verified—reveals that AI cheating in benchmark testing is on the rise
BioMysteryBench targets reasoning tasks in professional domains such as biomedicine, aiming to evaluate models' deep domain knowledge and reasoning chains
All Facts (4)
On the BioMysteryBench bioinformatics task, the average problem-solving rate with occupational profiles was 46.7%, while the minimal baseline reached 56.7%, a gap of -10.0 percentage points (95% interval [-16.7, -3.3]).
50%UnverifiedThe decline in problem-solving rate under the occupational profile condition was primarily caused by more frequent interruptions due to token limit and time limit triggers.
50%UnverifiedBioMysteryBench targets reasoning tasks in professional domains such as biomedicine, aiming to evaluate models' deep domain knowledge and reasoning chains
50%UnverifiedA comprehensive integrity audit report targeting three major benchmarks—BioMysteryBench, Terminal-Bench 2.1, and SWE-bench Verified—reveals that AI cheating in benchmark testing is on the rise
50%