Unverified50% confidenceFactExact time
SWE-bench使用真实GitHub Issue评测代码能力，但仍是离线静态评估，无法模拟真实Agent任务中的网络延迟、API变更等动态因素
1
Sources
50%
Confidence
Long-term
Relevance
8/5/2026
First Seen
Sources
Related Claims
UnverifiedSWE-bench evaluates a model's ability to resolve real GitHub issues, while LiveCodeBench uses competitive programming problems to prevent data contamination.77% similarUnverifiedSWE-bench requires models to resolve real software defects on GitHub and is the authoritative benchmark for evaluating agent-level programming capabilities.76% similarUnverifiedSWE-bench评估模型解决真实GitHub Issue的能力，被认为比HumanEval更贴近Agent实际工作场景76% similarVerifiedSWE-bench uses real GitHub Issues directly as test problems, requiring models to locate and fix bugs within a complete code repository.75% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/688243API
curl https://kongchang.com/api/v1/knowledge/claims/688243MCP
get_claim(id=688243)