Karpathy on Vibe Coding and Agentic Engineering: A Paradigm Shift for Programmers

Introduction: Never Feeling This Behind

Andrej Karpathy—OpenAI co-founder, creator of Tesla Autopilot, and the person who coined the term "Vibe Coding"—recently admitted in an interview that he has never felt this "behind" as a programmer. This AI pioneer, known for making complex technical concepts accessible, shared his profound insights on the fundamental shift in the software development paradigm.

Last December, while using the latest AI agent tools, Karpathy experienced an "aha moment": the code blocks generated by models started being consistently correct, and he couldn't even remember the last time he needed to make a manual fix. From that point on, he fully trusted the system and began truly "vibe coding." His side project folder rapidly expanded, filling up with all kinds of random projects.

Software 3.0: Prompting as Programming—A Whole New Computing Paradigm

Karpathy proposed a clear evolutionary framework:

• Software 1.0: Programmers write explicit rule-based code
• Software 2.0: "Programming" through datasets and training neural networks
• Software 3.0: Programming becomes prompting, and the context window becomes the lever you use to manipulate the LLM "interpreter"

This framework didn't come out of nowhere. As early as 2017, Karpathy systematically articulated the paradigm shift from Software 1.0 to 2.0 in his blog post Software 2.0: Software 1.0 is traditional imperative programming, where programmers explicitly write every logical rule in Python, C++, and other languages; Software 2.0 "writes" programs using neural network weights, where developers no longer write rules directly but instead "train" desired behaviors through constructing datasets, designing loss functions, and running backpropagation. The Software 3.0 he now proposes is a further extension following the emergence of Large Language Models (LLMs): natural language prompts themselves become the programming medium, LLMs serve as universal "interpreters," and the context window—the length of text a model can process in a single pass, which has expanded from an early 4K tokens to the million-token scale of Gemini 1.5 Pro—becomes the core lever for programmers to manipulate this interpreter. The three paradigms don't replace each other but coexist in layers, though the center of gravity is rapidly shifting toward 3.0.

The context window is essentially the LLM's "working memory"—the full range of information the model can "see" and reference when generating each token. This expansion has made AI Agent architectures possible: agents are no longer limited to single-turn Q&A but can maintain task state within a persistent context, call external tools (code executors, web search, file systems), observe execution results, and plan their next actions accordingly, forming a "perceive-plan-act" loop.

He gave a highly compelling example: installing OpenClaw is no longer a complex shell script but a piece of text you copy-paste to an AI agent. The agent, leveraging its own intelligence, can observe your environment, perform intelligent operations, and debug issues in a loop—the entire process unfolding dynamically within the context window, whose size directly determines the upper bound of task complexity the agent can handle. "What text you copy-paste to your agent—that's the programming paradigm now."

The MenuGen Revelation: A Lot of Code Shouldn't Exist at All

An even more extreme example is his MenuGen project. Originally, he wrote a complete application to photograph restaurant menus, OCR the dish names, and use image generators to produce pictures of the dishes. But the Software 3.0 version stunned him: just hand the photo to Gemini and let it use an image generation model to render dish images directly into the pixels of the menu photo.

"All of my MenuGen code was superfluous. It worked in the old paradigm. These apps shouldn't exist."

This isn't just about coding getting faster—it's entirely new information-processing capabilities. For instance, his LLM knowledge base project lets an LLM create a Wiki for an organization, something that previously couldn't have existed because no code could create a knowledge base from a pile of facts.

Verifiability: Understanding AI's Jagged Capability Frontier

Karpathy spent considerable time thinking about the concept of "verifiability." His core argument is: traditional computers can easily automate things you can specify in code, while LLMs can easily automate things you can verify.

Frontier labs train LLMs using massive reinforcement learning environments. Models excel in verifiable domains (like math and code) but exhibit a "jagged" capability distribution in other areas. This phenomenon is rooted in the training methods of contemporary frontier models—Reinforcement Learning from Human Feedback (RLHF) and its successors RLAIF and Process Reward Models (PRM). The core mechanism of reinforcement learning is the reward signal: the model generates output, the environment provides reward or punishment, and the model adjusts its strategy accordingly. Math problems have standard answers, code has compilers and test cases—reward signals in these domains are clear, objective, and automatable, allowing models to rapidly iterate and improve through massive self-play. OpenAI's o-series models, DeepSeek-R1, and other "reasoning models" have achieved breakthrough progress on such verifiable tasks through large-scale reinforcement learning. Conversely, commonsense judgments like "walk 50 meters to the car wash"—which have no definitive correct answer and for which it's difficult to construct automated reward functions—leave model performance highly dependent on statistical distributions in pretraining data, creating that bewildering "jagged frontier."