Unverified50% confidenceBenchmarkExact time
In a test of 48 synthesized programming requests, under the default four-option question Laya matched the team's annotation on only 21 of them, with a maximum confidence of only 0.26.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
10/15/2026
First Seen
Valid until: 1/13/2027
Sources
Related Entities
Related Claims
UnverifiedThe limitations of this test include: limited task scale (50 tasks, 3 trials per prompt), style versions written by a single author, length confounding between the florid and normal versions, testing only English and local models below 14B, and the perplexity check covering only three smaller Qwen models and using full-precision weights while actual inference used 4-bit quantization.69% similarUnverified46个模型中有41个完整跑完全部28道题，3个因服务错误未完成，GPT-5.6 Terra因配额限制止步24/28，GPT-6 Astra只跑了1道题68% similarUnverifiedThe test results showed that 27 out of 30 tickets were routed to the correct team.68% similarUnverifiedIn a PR review comparison test, Opus found 23 valid issues (including 8 nits), Muse found 16, and Sol found only 9.67% similarUnverifiedIn a small-scale test published by the PartsPlan developer, all 30 chart values were answered correctly.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1037626API
curl https://kongchang.com/api/v1/knowledge/claims/1037626MCP
get_claim(id=1037626)