Benchmark
Deep SW1
A benchmark that evaluates a model's ability to understand and modify code in real-world codebases. Aligned with actual production environment requirements, it provides greater engineering reference value than purely question-and-answer style evaluations.
Timeline (last 90 days)
Sep 30
In the Deep SW1 codebase evaluation, Sol's highest tier achieved 68.8%, approaching the top model Cloud Fable 5's 69.9%, at approximately 80% lower cost.
Unverified50%