CAD World
The first computer-use benchmark targeting long-horizon mechanical CAD workflows, comprising 200 tasks across 11 workflow categories. It evaluates computer-use agents on automated operation tasks within professional CAD software.
Timeline (last 90 days)
CAD World is the first computer operation benchmark targeting long-horizon mechanical CAD workflows.
In CAD World's 200-task evaluation, the best-performing GPT-5.4 achieved a success rate of only 17.5%.
In the CAD World benchmark, human experts achieved a success rate of 87%, requiring an average of only 26.5 steps.
CAD World tested seven mainstream computer operation agents.
GPT-5.4 achieved a completion rate of 62.5% on CAD World, with an average of 104.6 action steps and an average of 46.9 steps for successfully completed tasks.
Opus-4.8 achieved a success rate of 16% on CAD World, with an average cost of $10 per task.
Kimi K2.6 achieved a success rate of 7.5% on CAD World, OpenCUA achieved 1.5%, and two other models achieved a success rate of 0.
CAD World contains 200 tasks, covering 11 workflow categories and 183 distinct knowledge points.
In CAD World, the sketch category contains 63 tasks and the part modeling category contains 77 tasks.
CAD World uses the open-source FreeCAD as its runtime environment, with FreeCAD running inside a pre-built Ubuntu virtual machine managed via Docker and QEMU.
3 more timeline events
All Facts (13)
CAD World is the first computer operation benchmark targeting long-horizon mechanical CAD workflows.
50%UnverifiedIn CAD World's 200-task evaluation, the best-performing GPT-5.4 achieved a success rate of only 17.5%.
50%UnverifiedIn the CAD World benchmark, human experts achieved a success rate of 87%, requiring an average of only 26.5 steps.
50%UnverifiedCAD World tested seven mainstream computer operation agents.
50%UnverifiedGPT-5.4 achieved a completion rate of 62.5% on CAD World, with an average of 104.6 action steps and an average of 46.9 steps for successfully completed tasks.
50%UnverifiedOpus-4.8 achieved a success rate of 16% on CAD World, with an average cost of $10 per task.
50%UnverifiedKimi K2.6 achieved a success rate of 7.5% on CAD World, OpenCUA achieved 1.5%, and two other models achieved a success rate of 0.
50%UnverifiedCAD World contains 200 tasks, covering 11 workflow categories and 183 distinct knowledge points.
50%UnverifiedIn CAD World, the sketch category contains 63 tasks and the part modeling category contains 77 tasks.
50%UnverifiedCAD World uses the open-source FreeCAD as its runtime environment, with FreeCAD running inside a pre-built Ubuntu virtual machine managed via Docker and QEMU.
50%UnverifiedThe annotation and verification process for CAD World took approximately three months and 1,300 person-hours, with a median instruction length of 63 words.
50%UnverifiedIn CAD World, each task is limited to a maximum of 100 steps, and agents cannot directly access files or call APIs.
50%UnverifiedAs model capabilities improve, the failure stage in CAD tasks shifts later rather than disappearing entirely, with stronger models' failures attributable to document structure errors, geometric errors, and modeling workflow errors.
50%