LiveCodeBench
LiveCodeBench is a comprehensive coding evaluation benchmark for large language models that continuously collects problems from major competitive programming platforms such as LeetCode, AtCoder, and Codeforces to prevent models from overfitting to static test sets. Beyond code generation, the benchmark covers multiple programming task dimensions including code repair and test case generation, providing a dynamic and thorough assessment of model programming capabilities.
Core Facts
Timeline (last 90 days)
The IQ3S tier scores 92.93 on GPQA Diamond (vs. BF16 baseline 91.92), 86.86 on LiveCodeBench V6 (vs. 87.43), with a three-task average of 93.26 vs. 93.12, and a file size less than one quarter of BF16 (approximately 22%).
Code capability benchmarks widely recognized in academia and industry include HumanEval, SWE-bench, and LiveCodeBench, but these benchmarks do not yet cover comprehensive tasks such as 3D game generation.
Version 2.0 shows declines in knowledge-based capabilities (MMLU-Pro 72.5 vs. base 78.5) and competition-level coding ability (LiveCodeBench 51 vs. 56).
SWE-bench and LiveCodeBench are relatively well-recognized industry benchmarks for evaluating code large language models
LiveCodeBench adopts a temporal cutoff strategy, using only competition problems released after a model's training cutoff date to prevent data leakage.
New-generation benchmarks such as SWE-bench and LiveCodeBench attempt to evaluate model capabilities in real-world scenarios, but still cannot fully cover engineering complexity.
LiveCodeBench continuously scrapes new problems from competitive programming platforms such as LeetCode, Codeforces, and AtCoder to prevent data contamination.
LiveCodeBench continuously crawls new problems from competitive programming platforms such as LeetCode, Codeforces, and AtCoder to prevent models from gaming the benchmark, and its evaluation covers four dimensions: code generation, self-repair, code execution reasoning, and test output prediction.
Qwen2.5-Max-0902 topped the overall leaderboard on the LiveCodeBench programming rankings with a score of 1691.
Qwen3 27B's performance on coding and agent benchmarks is: SWE-bench Pro 61.7, Terminal-Bench 2.1 73.0, and LiveCodeBench V6 90.3.
3 more timeline events
All Facts (14)
Mainstream benchmarks for evaluating programming large models include HumanEval, MBPP, SWE-bench, and LiveCodeBench
80%VerifiedQwen3 27B scored 90.3 on LiveCodeBench V6, slightly surpassing Opus 4.6 Max's score of 88.8.
65%UnverifiedLiveCodeBench continuously crawls new problems from competitive programming platforms such as LeetCode, Codeforces, and AtCoder to prevent models from gaming the benchmark, and its evaluation covers four dimensions: code generation, self-repair, code execution reasoning, and test output prediction.
60%UnverifiedLiveCodeBench uses continuously updated new problems from competitive programming platforms to avoid training data leakage.
60%UnverifiedSWE-bench evaluates a model's ability to resolve real GitHub issues, while LiveCodeBench uses competitive programming problems to prevent data contamination.
60%UnverifiedDynamic benchmarks such as LiveCodeBench and EvoEval reduce data contamination by continuously updating their problem sets or generating variant problems in real time.
60%UnverifiedThe IQ3S tier scores 92.93 on GPQA Diamond (vs. BF16 baseline 91.92), 86.86 on LiveCodeBench V6 (vs. 87.43), with a three-task average of 93.26 vs. 93.12, and a file size less than one quarter of BF16 (approximately 22%).
50%UnverifiedCode capability benchmarks widely recognized in academia and industry include HumanEval, SWE-bench, and LiveCodeBench, but these benchmarks do not yet cover comprehensive tasks such as 3D game generation.
50%UnverifiedVersion 2.0 shows declines in knowledge-based capabilities (MMLU-Pro 72.5 vs. base 78.5) and competition-level coding ability (LiveCodeBench 51 vs. 56).
50%UnverifiedSWE-bench and LiveCodeBench are relatively well-recognized industry benchmarks for evaluating code large language models
50%UnverifiedLiveCodeBench adopts a temporal cutoff strategy, using only competition problems released after a model's training cutoff date to prevent data leakage.
50%UnverifiedNew-generation benchmarks such as SWE-bench and LiveCodeBench attempt to evaluate model capabilities in real-world scenarios, but still cannot fully cover engineering complexity.
50%UnverifiedLiveCodeBench continuously scrapes new problems from competitive programming platforms such as LeetCode, Codeforces, and AtCoder to prevent data contamination.
50%UnverifiedQwen3 27B's performance on coding and agent benchmarks is: SWE-bench Pro 61.7, Terminal-Bench 2.1 73.0, and LiveCodeBench V6 90.3.
50%