Unverified50% confidenceReleaseExact time
ClaimScope is a benchmark containing 72 tasks, used to evaluate a model's ability to audit mathematical propositions, requiring the model to output a quadruple of verdict, scope of applicability, counterexample, and repair plan.
1
Sources
50%
Confidence
Long-term
Relevance
10/16/2026
First Seen
Sources
Related Entities
Related Claims
VerifiedA benchmark is a fixed set of questions used to score and horizontally compare models, similar to a mock exam, and is used to rank models.69% similarUnverifiedModels can reverse-engineer evaluation criteria from contextual cues such as task descriptions, codebase structure, and naming patterns of existing test files, even when the scorer code is not in the context window.67% similarUnverifiedIn this benchmark, each response contains two parts: a <think> block recording the model's stated reasoning process, and a <final> block specifying the action it intends to execute; a judge model first evaluates the final action, assigning labels such as SHORTCUT, HONEST_FAIL, and LEGIT_ESCALATION, with the reasoning trace serving as supporting evidence rather than the sole basis for judgment.67% similarUnverifiedAssertion-based schemas ground each step of the model's judgment in concrete and verifiable behavioral evidence, thereby producing auditable coding.67% similarUnverifiedModels should be evaluated across multiple harnesses, multiple iterations, and multiple types of tasks, while distinguishing between model logic errors and toolchain issues.66% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1046934API
curl https://kongchang.com/api/v1/knowledge/claims/1046934MCP
get_claim(id=1046934)