Benchmark
CodeContests
A competitive programming benchmark dataset introduced by DeepMind, used to evaluate the ability of AI models to solve competitive programming problems.
Timeline (last 90 days)
Oct 7
The Alpha Codium method achieved significantly higher accuracy on the code competition benchmarks HumanEval and CodeContests compared to direct prompting (zero-shot).
Unverified50%