Unverified50% confidenceDecision RuleExact time
The rule for judging a skill as clearly helpful is: on at least one task the two score intervals (mean plus or minus one standard deviation) do not overlap and the gap is no less than 0.05, while no task clearly moves in the opposite direction; a clear decline on any one task outweighs improvements on other tasks.
1
Sources
50%
Confidence
Long-term
Relevance
10/19/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThose relaxed verification rules that can reduce the number of verification rounds are usually accompanied by a marked drop in task accuracy.76% similarUnverifiedThere is a significant evaluation-application gap between benchmark scores and actual production environment performance, as standard evaluations are typically single-turn, short-context tasks with standard answers.75% similarUnverifiedA single average score conceals the composition of the score, so the set should be split by difficulty into clear, adversarial and ambiguous, and reported separately; when clear tasks are the majority and easy to pass, they average away the failures on adversarial and ambiguous tasks.74% similarUnverifiedWhen interpreting benchmarks that use binary thresholds, one should distinguish between the two signals of threshold pass rate and dimension scores, to avoid misreading threshold luck as a capability gap.72% similarUnverifiedInference regret measures the gap between the global optimum value and the function value of the point the algorithm finally returns.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1065272API
curl https://kongchang.com/api/v1/knowledge/claims/1065272MCP
get_claim(id=1065272)