Unverified50% confidenceFactExact time
Long-horizon coding benchmarks (such as SWE-bench) evaluate a model's ability to handle repository-level tasks, including reading codebases, locating bugs, making cross-file modifications, and passing tests.
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn coding capability benchmarks such as SWE-bench, competing systems typically need to autonomously complete the full workflow of reading the codebase, understanding the issue, locating files, modifying code, and running tests.79% similarUnverifiedThe actionable method for validating a coding model is to select a real repair task with tests, fix the repository version and tool permissions, have candidate models complete the same task, and compare first-pass rate, patch readability, per-run cost, and number of human takeovers.74% similarUnverifiedSWE-bench相比HumanEval、MBPP等传统基准更贴近实际开发场景，考验模型对大型代码库的理解、跨文件依赖追踪和缺陷定位能力73% similarUnverified基准测试通常在受控环境中运行，不涉及长对话中的上下文管理、用户偏好记忆、与现有代码库风格一致性等维度73% similarUnverifiedSWE-bench involves high-level capabilities such as cross-file dependency analysis, version compatibility judgment, and regression test passing, making it more reflective of real engineering scenarios than function-writing benchmarks such as HumanEval72% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990172API
curl https://kongchang.com/api/v1/knowledge/claims/990172MCP
get_claim(id=990172)