Unverified50% confidenceBenchmarkExact time
Controlled experiments show that after providing models with complete requirements, the F2P pass rates of Seed Evolving and GPT-6 Astra increased by about 16 percentage points, but the full task success rate did not change at all.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
11/17/2026
First Seen
Valid until: 2/15/2027
Sources
Related Entities
Related Claims
UnverifiedAmong the 12 model × task combinations, only gpt-5.4-mini showed a statistically significant accuracy improvement on the logical reasoning task: jumping from 15% to 97.5%, with p=0.00056 after Holm correction.68% similarUnverified6-bit quantization did not lose accuracy in the experiments; instead it was +1.8 and +0.9 percentage points higher than fp16 (the study did not explain this phenomenon).66% similarUnverifiedOn performance model code construction tasks, the average pass rate of all model configurations other than GPT-5.6 Sol was below 15%, with extremely large fluctuations across multiple runs.66% similarUnverifiedThe per-task cost of GPT-6 Astra is lower than that of its predecessor, because a more capable model completes the same task in fewer steps and with fewer tokens.65% similarUnverifiedThe transfer experiments in the study show that fine-tuning on the same dataset yields performance gains ranging from +6.8 to +23.1 percentage points.65% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1179748API
curl https://kongchang.com/api/v1/knowledge/claims/1179748MCP
get_claim(id=1179748)