Unverified50% confidenceFactExact time
MMLU考察跨学科知识理解、HumanEval测试代码生成能力、SWE-bench专门针对真实GitHub Issue修复能力
1
Sources
50%
Confidence
Long-term
Relevance
7/19/2026
First Seen
Sources
Related Claims
UnverifiedSWE-bench评估模型解决真实GitHub Issue的能力，被认为比HumanEval更贴近Agent实际工作场景75% similarUnverifiedSWE-bench requires models to resolve real software defects on GitHub and is the authoritative benchmark for evaluating agent-level programming capabilities.74% similarUnverifiedSWE-bench requires models to fix issues in real GitHub repositories, while HumanEval/MBPP are function-level code generation benchmarks.74% similarUnverifiedSWE-bench evaluates a model's ability to resolve real GitHub issues, while LiveCodeBench uses competitive programming problems to prevent data contamination.73% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/558246API
curl https://kongchang.com/api/v1/knowledge/claims/558246MCP
get_claim(id=558246)