Unverified50% confidenceOpinionExact time
When showcasing its OS World leaderboard results, Meta used test sets from different time periods to compare against older models, artificially inflating the magnitude of improvement.
1
Sources
50%
Confidence
Medium-term (~90 days)
Relevance
9/10/2026
First Seen
Valid until: 12/9/2026
Sources
Related Entities
Related Claims
UnverifiedMeta AI research shows that having a model iteratively optimize its own prompts can improve performance by 15-30% on certain standard benchmarks, but carries approximately a 20% probability of falling into a degenerative loop.65% similarUnverifiedIn the safety report released alongside Llama 2, Meta acknowledged that safety fine-tuning caused the model's scores to decline on certain benchmarks.61% similarUnverifiedTraditional benchmarks such as GLUE, SuperGLUE, and MMLU were scored above human average levels by models within just a few years.58% similarUnverifiedWhen multiple models all achieve accuracy rates above 90% on benchmarks such as MMLU and HumanEval, user attention shifts to practical metrics such as inference latency, service availability, and total cost of ownership.53% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/886011API
curl https://kongchang.com/api/v1/knowledge/claims/886011MCP
get_claim(id=886011)