ReproBridge-Bench
A benchmark for evaluating the ability to reproduce machine learning papers, comprising 1,029 tasks from 36 papers, anchored by human-verified reference scores, and designed with dual-condition controls and independent auditing.
Timeline (last 90 days)
ReproBridge-Bench contains 1,029 tasks from 387 experiments in 36 papers, with each task anchored by five human-verified reference scores.
In the evaluation design of ReproBridge-Bench, agents attempt tasks in fresh sessions using one of two conditions—original code or refined artifact—and an independent Auditor is responsible for checking execution evidence.
Across the four evaluated models, refined artifacts raised the average reproduction success rate from 76.4% to 92.6%.
All Facts (3)
ReproBridge-Bench contains 1,029 tasks from 387 experiments in 36 papers, with each task anchored by five human-verified reference scores.
50%UnverifiedIn the evaluation design of ReproBridge-Bench, agents attempt tasks in fresh sessions using one of two conditions—original code or refined artifact—and an independent Auditor is responsible for checking execution evidence.
50%UnverifiedAcross the four evaluated models, refined artifacts raised the average reproduction success rate from 76.4% to 92.6%.
50%