Expired50% confidenceFactExact time
Mainstream benchmarks like SWE-Bench Pro heavily rely on issues and commits from public repositories, creating data contamination risk for models trained on public GitHub code.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
7/2/2026
First Seen
Valid until: 9/30/2026(expired)
Sources
DeepSWE Benchmark Reveals the Truth: GPT 5.5 Leads Opus 4.7 by a Wide Margin
bilibili7k的每日搬运5/29/2026
Related Claims
UnverifiedSWE-bench evaluates a model's ability to resolve real GitHub issues, while LiveCodeBench uses competitive programming problems to prevent data contamination.71% similarUnverified数据污染问题源于大模型在包含 GitHub、Stack Overflow、LeetCode 等平台公开代码题解的语料上进行预训练70% similarUnverifiedSWE-bench (Software Engineering Benchmark) is currently the core benchmark for measuring agent coding capability, requiring the model to start from real GitHub Issues, autonomously locate problems in the codebase, and submit fix patches.68% similarUnverifiedGitHub Copilot生成代码中常见的安全问题包括SQL注入、跨站脚本攻击（XSS）和不安全的随机数生成67% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/46950API
curl https://kongchang.com/api/v1/knowledge/claims/46950MCP
get_claim(id=46950)