Which LLM Writes the Most Efficient C/Rust Code? We Tested 4 Models on Low-Level Optimization

Most public leaderboards evaluating LLM coding ability focus narrowly on whether code passes basic tests and compiles successfully. For low-level systems programming, that's nowhere near enough — what actually matters is execution efficiency, compilation overhead, and algorithmic complexity. A computer and data science student identified this blind spot and designed a targeted benchmark specifically testing how leading models perform on C and Rust, two languages where runtime efficiency is paramount.

Benchmark Design: Built for Low-Level Optimization

Rather than following generic functional testing approaches, this evaluation constructed 15 test cases targeting low-level tasks across three core scenarios: custom memory management, fundamental OS kernel loops, and heavily data-sorted array processing.

The specific dimensions examined include complex pointer arithmetic, memory allocation strategies, and optimization techniques like loop unrolling. The benchmark's core objective is clear — identify which model generates code that is "directly executable with the lowest CPU overhead," not just code that "runs."

This design addresses a critical gap in current LLM coding evaluations: the chasm between semantic correctness and performance. A piece of code compiling successfully does not mean it makes sensible performance trade-offs.

Loop unrolling is a classic compiler and manual optimization technique that boosts throughput by duplicating the loop body multiple times, reducing the number of loop control instructions executed (conditional branches, counter updates), and creating more opportunities for the processor's instruction-level parallelism (ILP) and SIMD vectorization. For example, a loop that executes 4 times can be unrolled into 4 sequential instructions, eliminating branch prediction overhead entirely. Modern compilers (such as GCC with -O2/-O3, or LLVM's auto-vectorization) perform partial loop unrolling automatically when optimization flags are enabled, but manual unrolling or using #pragma unroll hints can still yield additional gains in specific scenarios. For LLMs, the ability to proactively generate loop-unrolled code or insert relevant compiler hints at appropriate points is a direct indicator of low-level optimization awareness.

The Contenders: Four Distinct Technical Approaches

The evaluation selected four models representing different technical paths, spanning the spectrum from code-specialized to general-purpose reasoning:

• DeepSeek^Ad-Coder-V2-Instruct: A highly popular open-source code-specialized model, known for deep contextual understanding of code logic.
• Claude 3.5 Sonnet: Widely recognized as a benchmark for complex reasoning, with standout performance in multi-file software construction and code architecture.
• GPT-4o: Included as a general-purpose heavyweight baseline to observe how generalized logic handles aggressively structured compiler-level problems.
• Gemini 1.5 Pro: Selected for its enormous context window, which allows complex hardware architecture specifications to be fed directly into prompt constraints.

These four models represent four differentiated strategies — "open-source specialist," "reasoning architecture," "general baseline," and "ultra-long context" — making the comparative results more informative.

Key Findings: Compilation Success ≠ Execution Efficiency

The results challenged the author's prior assumptions about how these models should be applied in low-level systems programming.

The Gap Between Semantic Correctness and Performance

All four models achieved over 90% pass rates on syntax and compilation — appearing neck-and-neck on the surface. But once execution performance entered the picture, the gaps opened up immediately. On complex C pointer manipulation tasks, code generated by DeepSeek-Coder-V2 ran up to 18% faster than GPT-4o's output, primarily because it successfully avoided redundant stack-to-heap copies.

This data illustrates the problem clearly: pass rate alone is completely inadequate for distinguishing real engineering quality between models.

Stack-to-heap copying is a common topic in low-level performance optimization. Stack memory is managed automatically by the compiler with extremely low allocation and deallocation overhead, fast access, but limited capacity and scope-bound lifetimes. Heap memory is explicitly allocated via malloc/free (C) or Box (Rust), offering flexibility but introducing system call overhead, memory fragmentation, and cache locality degradation. When LLMs generate code, imprecise reasoning about data lifetimes can lead to incorrectly allocating temporary variables or small structs to the heap that could have stayed on the stack, with unnecessary data copies occurring between function calls. This kind of "defensive programming" habit has limited impact in interpreted languages, but in a language like C that maps directly to hardware, it amplifies into measurable performance losses. DeepSeek-Coder-V2's advantage in these tasks demonstrates more accurate modeling of C's memory semantics.

Rust Safety Overhead Differences

When generating Rust code, Claude 3.5 Sonnet delivered impressive results — it consistently found optimal zero-cost abstractions and passed the borrow checker on the first attempt.

Gemini 1.5 Pro, by contrast, had no structural problems but frequently fell back on unsafe blocks to force compilation when dealing with complex data ownership nodes. This means it was sacrificing Rust's hallmark memory safety to achieve compilability — a concerning signal for real-world projects.

Zero-cost abstraction is one of Rust's core design philosophies, extending principles articulated by language designer Bjarne Stroustrup: features you don't use cost nothing, and features you do use couldn't be implemented more efficiently by hand. In Rust, this means iterators, generics, trait objects, and other high-level syntactic sugar compile down to machine code equivalent to handwritten assembly, with zero runtime overhead. The borrow checker, by contrast, is the component of the Rust compiler responsible for enforcing ownership rules at compile time through static analysis, ensuring the absence of dangling pointers, data races, or double-free errors — at the cost of requiring developers to explicitly declare data lifetimes and ownership transfers. When a model generates unsafe blocks to bypass the borrow checker, it is fundamentally abandoning Rust's core advantage over C/C++ — compile-time memory safety guarantees — and pushing safety responsibility back onto the programmer. In production code audits, this is typically a significant risk marker.

The Biggest Surprise: Specialization Doesn't Mean All-Around Dominance

The most unexpected finding was that the general-purpose reasoning model Claude slightly outperformed the code-specialized model DeepSeek in code readability and comment quality. Yet Claude couldn't match DeepSeek in the use of micro-optimization flags.

This suggests that "specialization" and "generality" each have their strengths — DeepSeek excels at pushing performance to the limit, while Claude provides greater confidence for engineering maintainability. Which one to choose depends on whether your priority is running fast or being easy to modify.

Implications for Developers

This benchmark offers several practical takeaways for low-level systems developers. If you're chasing maximum runtime performance and micro-optimization, code-specialized models like DeepSeek-Coder-V2 deserve priority consideration. If you value code readability, complete documentation, and adherence to Rust safety conventions, general-purpose reasoning models like Claude 3.5 Sonnet may be the more reliable choice.

The author also mentioned planned extensions: measuring these models' ability to optimize code specifically for ARM vs. x86 assembly architectures. As edge AI and heterogeneous computing become more prevalent, this kind of architecture-aware optimization capability will grow increasingly important.

It's worth noting that this benchmark was completed by a single author, the test scale (15 tasks) is relatively limited, and performance figures like the 18% gap depend on specific task scenarios and shouldn't be extrapolated to all workloads. But the phenomenon it reveals — the disconnect between semantic correctness and performance — is genuinely worth attention across the entire LLM coding evaluation field.