BenchmarkSWE-bench Lite

SWE-bench Lite

A streamlined version of SWE-bench containing 300 real GitHub issue tasks drawn from 12 Python repositories, used to evaluate the ability of language models to solve real software engineering problems.

Timeline (last 90 days)

Oct 21

UC Berkeley research tested 7 frontier models on a variety of harnesses and found that for newer, smarter models, the choice of harness made no significant difference in SWE-bench Lite scores.

Unverified50%

All Facts (1)

Unverified

UC Berkeley research tested 7 frontier models on a variety of harnesses and found that for newer, smarter models, the choice of harness made no significant difference in SWE-bench Lite scores.

50%

Source Articles

Is the Agent Harness Dead? A Cursor Engineer's Deep DiveOct 21 Is the Agent Framework Dead? A Cursor Engineer Counters with DataOct 21