Unverified50% confidenceBenchmarkExact time
The best-performing model achieved 38.7% on pass@3, but only 5.9% on pass^3.
1
Sources
50%
Confidence
Long-term
Relevance
10/5/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedThe original Codex achieved a pass@1 score of 28.8% on the HumanEval benchmark, whereas using GPT-3 directly yielded only 0%.64% similarUnverifiedOn the AIME-2024 mathematical reasoning benchmark, pass@1 improved from 0.39 to 0.49.60% similarUnverifiedIn the proprietary benchmark PinBash 3 test, SOL scored approximately 78.6%, TERRA approximately 62.9%, and LUNA approximately 44.3%.60% similarUnverifiedIn EvoArena testing, GPT-5.5, which achieved the highest Step Level score, had an accuracy of just over 50%, while Gemini-3.1 Pro and GLM-5.1, which performed best at the Chain Level, reached only approximately 28%–29%.59% similarUnverifiedThe simple 3-rule baseline used for comparison achieved an accuracy of only 64.4%, with Jev leading by approximately 8 percentage points.59% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/975459API
curl https://kongchang.com/api/v1/knowledge/claims/975459MCP
get_claim(id=975459)