ProgramBench
A benchmark for evaluating the ability of programming agents to build complete programs from scratch. It supports program generation task evaluation across system-level programming languages including C, C++, Go, and Rust.
Timeline (last 90 days)
Among the 14 benchmarks Meta itself published, Muse Wonder won 5, mainly in long-horizon coding, legal agent tasks, and Meta's internal memory tests; it lost 9, including ProgramBench (Opus leads 70.5 to 50.0).
After fine-tuning using trajectories generated from building environments with Program-Env, the performance of the Qwen3.8-27B model improved from 34.87% to 42.84%.
Program-Agent pushed the base model and the supervised fine-tuned model to 55.71% and 56.25%, respectively.
The effectiveness of ProgramGym has been validated on the ProgramBench benchmark.
All Facts (4)
Among the 14 benchmarks Meta itself published, Muse Wonder won 5, mainly in long-horizon coding, legal agent tasks, and Meta's internal memory tests; it lost 9, including ProgramBench (Opus leads 70.5 to 50.0).
50%UnverifiedAfter fine-tuning using trajectories generated from building environments with Program-Env, the performance of the Qwen3.8-27B model improved from 34.87% to 42.84%.
50%UnverifiedProgram-Agent pushed the base model and the supervised fine-tuned model to 55.71% and 56.25%, respectively.
50%UnverifiedThe effectiveness of ProgramGym has been validated on the ProgramBench benchmark.
50%