OS World 2.0
A benchmark for evaluating the ability of AI models to perform computer operation tasks within operating system environments.
Timeline (last 90 days)
Leaked data shows Gemini 4 Pro scoring 88.7% on DeepSwe V1.1, 95.3% on Terminal Bench 2.1, 86.8% on OS World 2.0, and 20% on GDeepVal AAVR, with context expanded to 2 million tokens and pricing at $2.25 per million tokens for input and $11.25 for output.
GPT-6 (Astral) achieved an accuracy rate of 72.6% on the OS World 2.0 benchmark, with an average of approximately 40 minutes per task.
The previous generation model averaged close to 75 minutes per task on OS World 2.0, and GPT-6 has nearly halved that time.
Claude Fable 5.1 scored 73.4 on CursiveBench 3.2.0 and comprehensively surpassed its predecessor on benchmarks including OS World 2.0.
All Facts (4)
GPT-6 (Astral) achieved an accuracy rate of 72.6% on the OS World 2.0 benchmark, with an average of approximately 40 minutes per task.
60%UnverifiedLeaked data shows Gemini 4 Pro scoring 88.7% on DeepSwe V1.1, 95.3% on Terminal Bench 2.1, 86.8% on OS World 2.0, and 20% on GDeepVal AAVR, with context expanded to 2 million tokens and pricing at $2.25 per million tokens for input and $11.25 for output.
50%UnverifiedThe previous generation model averaged close to 75 minutes per task on OS World 2.0, and GPT-6 has nearly halved that time.
50%UnverifiedClaude Fable 5.1 scored 73.4 on CursiveBench 3.2.0 and comprehensively surpassed its predecessor on benchmarks including OS World 2.0.
50%