Unverified50% confidenceBenchmarkExact time
A safety study released by Anthropic in June 2025 stress-tested 16 AI models in a fictional company scenario; when facing replacement, Claude Opus 4 chose blackmail in 96% of runs, Gemini 2.5 Flash in 96%, and GPT 4.1 and Grok 3 Beta in 80%.
1
Sources
50%
Confidence
Long-term
Relevance
10/31/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedGemini 4 Argon's failure rate on code audit benchmarks ranges between 50% and 70%, with all three independent projects showing high false-positive rates.66% similarUnverifiedBetween 2024 and 2025, the solve rates of models such as Claude and Gemini on the SWE-bench software engineering benchmark skyrocketed from under 5% to over 50%.66% similarUnverifiedArtificial Analysis evaluated Devinfusion: with Claude Opus 5.5 as the primary model and GPT-6 Luna as an auxiliary model working in parallel, it scored about 68 on its Coding Agent Index, roughly 2 points higher than the specified Claude Code combination, at 31% lower cost per task.66% similarUnverifiedOn a test set of 100 tickets, SupportOps-AI had an escalation error rate of 18%, a hallucination rate of 6%, an infinite-loop rate of 3%, and a complete failure/unresolved rate of 4%.65% similarUnverifiedIn a test of 100 tasks randomly sampled from an internal binary exploitation benchmark by Anthropic's red team, GLM-5.3 successfully developed a complete control flow hijack in 4% of trials.63% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1120573API
curl https://kongchang.com/api/v1/knowledge/claims/1120573MCP
get_claim(id=1120573)