Unverified50% confidenceFactExact time
Half of SWE-Race's tasks are private for cross-validation, and the scores of three models on the public set and private set are consistent.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/6/2026
First Seen
Valid until: 1/4/2027
Sources
Related Entities
Related Claims
Unverified实验采用8:2比例划分数据，20%作为内部测试集，训练集内部使用5折交叉验证，并移除重复记录以避免数据泄漏60% similarUnverifiedThe test case generation solution is divided into five progressive levels: a single agent plus prompt achieves only approximately 30% coverage, a workflow approach can achieve approximately 80% coverage, and an exploratory learning approach can achieve over 90% coverage.57% similarUnverifiedAbout half of the tasks are easy for all models, with scores close to 100%; on the other half of difficult tasks, the three models scored 50%, 45%, and 23% respectively.56% similarUnverifiedThe gap between selection and generation exceeds 50 percentage points, revealing that reliability is far lower than benchmarks suggest when users need the model to directly write out kinship terms.56% similarUnverified2024 年的多项基准测试表明，经过严格安全对齐的模型在代码生成任务上的拒绝率比基座模型高出 15-30%，在创意写作任务上的自我审查率高达 40%54% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/980434API
curl https://kongchang.com/api/v1/knowledge/claims/980434MCP
get_claim(id=980434)