llama2.c
A minimalist C-language LLM inference implementation written by Andrej Karpathy. It runs the Llama 2 architecture in a single run.c file, with core design goals of simplicity, readability, and ease of modification. It is suited for educational use and end-to-end CPU performance measurement.
Timeline (last 90 days)
Restoring the fflush(stdout) call after each token without modifying the optimized mathematical logic allows output to begin after 37 milliseconds and complete at 477 milliseconds, balancing computational optimization with a streaming experience.
The benchmark uses llama2.c locked to a specific commit, paired with TinyStories FP32 checkpoints (15M, 42M, 110M parameters); these are small models running on llama2.c rather than the full Llama 2.
llama2.c is a minimal C inference implementation written by Andrej Karpathy, with the core logic concentrated in a single run.c file.
Claude's original candidate removed per-token fflush(stdout) in favor of flushing only at the end; replay measured a completion speedup of 2.73× but a time-to-first-byte speedup of only 0.44× (i.e., streaming output became slower).
Andrej Karpathy implemented the llama2.c project for Llama 2 inference in approximately 700 lines of C code.
All Facts (5)
Restoring the fflush(stdout) call after each token without modifying the optimized mathematical logic allows output to begin after 37 milliseconds and complete at 477 milliseconds, balancing computational optimization with a streaming experience.
50%UnverifiedThe benchmark uses llama2.c locked to a specific commit, paired with TinyStories FP32 checkpoints (15M, 42M, 110M parameters); these are small models running on llama2.c rather than the full Llama 2.
50%Unverifiedllama2.c is a minimal C inference implementation written by Andrej Karpathy, with the core logic concentrated in a single run.c file.
50%UnverifiedClaude's original candidate removed per-token fflush(stdout) in favor of flushing only at the end; replay measured a completion speedup of 2.73× but a time-to-first-byte speedup of only 0.44× (i.e., streaming output became slower).
50%UnverifiedAndrej Karpathy implemented the llama2.c project for Llama 2 inference in approximately 700 lines of C code.
50%