New-generation large language models such as GPT-4, Claude 3.5 Sonnet, and Gemini 1.5 Pro have achieved significantly higher scores on code capability benchmarks such as HumanEval and SWE-bench compared to previous versions.