Unverified50% confidenceFactExact time
SWE-bench requires agents to submit patches that pass a test suite without human intervention
1
Sources
50%
Confidence
Long-term
Relevance
10/9/2026
First Seen
Sources
Related Entities
Related Claims
Unverified评估智能体编程能力的主流基准包括SWE-bench、SWE-bench Verified和HumanEval72% similarVerifiedSWE-bench Verified is one of the authoritative industry leaderboards for AI programming ability, with human annotators filtering out samples with ambiguous descriptions or unstable tests.72% similarUnverifiedSWE-bench involves high-level capabilities such as cross-file dependency analysis, version compatibility judgment, and regression test passing, making it more reflective of real engineering scenarios than function-writing benchmarks such as HumanEval72% similarUnverifiedRecommendations for tool evaluation: use deterministic testing as much as possible for what can be deterministically tested, and then use LLM to evaluate touchpoints with agents.71% similarUnverifiedAgent evaluation benchmark suites include GAIA, AgentBench, and SWE-bench.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/990200API
curl https://kongchang.com/api/v1/knowledge/claims/990200MCP
get_claim(id=990200)