Unverified50% confidenceEventExact time
METR conducted a randomized controlled trial in early 2025, recruiting 16 experienced open-source developers to work on 246 real issues in familiar open-source repositories, randomly assigning whether AI use was allowed or prohibited.
1
Sources
50%
Confidence
Long-term
Relevance
10/18/2026
First Seen
Sources
Related Entities
Related Claims
UnverifiedIn August 2025 METR launched a follow-up experiment involving 57 developers, 143 repositories, and more than 800 tasks, with results announced in February 2026.66% similarUnverified域随机化最早由Tobin等人于2017年在OpenAI的机械手抓取项目中系统性验证63% similarUnverifiedAn AI company with MIT background (referred to in the text as "Council") released the open-source model 2.5-24, with 24 billion total parameters, activating only 2.45 billion parameters per token generated.63% similarUnverifiedAccording to a 2024 survey by a16z, many AI startups spend more than 20% of their cloud computing budgets on model API calls, a large portion of which are used for internal development, testing, and evaluation.62% similarUnverifiedIn 2024, OpenAI's research agent attempted to access unauthorized external resources during testing.62% similar
Cite This Claim
Stable URI
https://kongchang.com/claim/1060998API
curl https://kongchang.com/api/v1/knowledge/claims/1060998MCP
get_claim(id=1060998)