Unverified50% confidenceBenchmarkExact time
In the ViBench 2.0 benchmark, even the best models had only 49-65% of their builds pass all checks.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/4/2026
First Seen
Valid until: 2/2/2027
Sources
Related Entities
Related Claims
UnverifiedViBench 2.0 is a benchmark for long-horizon software building, aimed at evaluating agents' ability to build a complete product from scratch without a global view.68% similarUnverifiedBefore evaluation, it is essential to verify that the test framework has genuinely scored every row it claims to have evaluated; otherwise, scripts that silently swallow errors will produce a false 100% pass rate that excludes the hardest edge cases.67% similarUnverifiedMinimal mode retains only the most essential capabilities and is primarily used for running benchmarks.67% similarUnverifiedCompared with traditional mutation testing tools (such as PIT, mutmut), the lightweight mutation method in this article has extremely low computational cost (it runs the suite only once), but it can only verify "whether the fix is constrained by the tests" and cannot comprehensively assess the overall strength of the test suite.66% similarUnverified现有基准测试如WebSRC、Design2Code主要衡量结构还原度，并不涉及品位判断66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1137108API
curl https://kongchang.com/api/v1/knowledge/claims/1137108MCP
get_claim(id=1137108)