Unverified50% confidenceFactExact time
A single test run produces at least three different numbers: collected, selected, executed (call phase), while patches usually report only the last one.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
Unverified2.5-24 took first place in all three instruction-following tests.67% similarUnverifiedThe final v3 version adds a salvage mechanism: it retains the best result from 3 attempts, and if the model produces at least 14 valid squares, it fills in the remaining squares from the starter list.64% similarUnverifiedThe limitations of this test include: limited task scale (50 tasks, 3 trials per prompt), style versions written by a single author, length confounding between the florid and normal versions, testing only English and local models below 14B, and the perplexity check covering only three smaller Qwen models and using full-precision weights while actual inference used 4-bit quantization.62% similarUnverifiedSingle-turn tests use strict exact-parameter scoring weighted by severity (L4=8 points decreasing to L0=0 points).60% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1049681API
curl https://kongchang.com/api/v1/knowledge/claims/1049681MCP
get_claim(id=1049681)