Unverified50% confidenceFactExact time
MedPhysBench's error-correction task shows the model a colleague's full worksheet or only the final result, with half correct and half containing a single implanted error, thereby testing the model's ability to find errors.
1
Sources
50%
Confidence
Long-term
Relevance
11/4/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe evaluation metric proposed by Fail-First focuses on whether the system continues to repeat the same type of error after the environment has proven an action wrong, making it more diagnostically valuable than task success rate.72% similarUnverifiedThe researchers open-sourced MedPhysBench's code, per-problem results, and targeted test responses.72% similarUnverifiedTerminal Bench tests what a model can do rather than what it knows, measuring the model's complete closed-loop capability of planning, executing, observing feedback, and correcting errors72% similarUnverifiedEach sample in FastBench comes with a manually annotated evidence interval, and the evaluation not only judges whether the answer is correct but also can locate whether the model attended to the correct time segment.71% similarUnverifiedArtificially planted errors are a common evaluation method for testing whether AI has the ability to detect problems.71% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1134257API
curl https://kongchang.com/api/v1/knowledge/claims/1134257MCP
get_claim(id=1134257)