A Deep Dive into OpenAI Codex: How AI Engineers Are Reimagining Software Development with Agents

The Golden Age of AI Engineers

At a recent AI Engineer conference, Romain and Alexander from OpenAI painted a rapidly materializing vision of the future for an audience of over 7,000 AI engineers. They likened the event to a "World's Fair"—where ideas that once sounded impossible are now within reach, and people can not only see them but step inside and believe in them.

The AI Engineer conference (AI Engineer Summit/World's Fair) is an industry event that has emerged in recent years, focused on the engineering practice of the AI application layer. Unlike traditional ML research conferences such as NeurIPS and ICML, it targets the community of engineers integrating AI models into products and systems, reflecting the industry's shift from lab research to engineering deployment. NeurIPS (Conference on Neural Information Processing Systems) and ICML (International Conference on Machine Learning) were born in an academic research context, with their core audience being researchers publishing new algorithms and theoretical breakthroughs. The AI Engineer conference, by contrast, concerns itself with "how to reliably embed model capabilities into real products." This distinction itself is a marker of industry maturity—when a technology begins to spawn a dedicated community of engineering practice, it often signals that it has crossed the critical inflection point from lab to industry.

Facing the widespread claim that "engineers are about to disappear," the two speakers offered a starkly opposite judgment. Their core logic follows this evolutionary chain: "Software is eating the world, AI is eating software, and now, AI engineers are eating the world." "Software is eating the world" was first coined by venture capitalist Marc Andreessen in a 2011 Wall Street Journal piece, describing the trend of software disrupting traditional industries. "AI is eating software" is the natural extension of this evolutionary logic, referring to how large language models and AI systems are reconstructing the very way software is developed. In their view, engineering was never just about writing code—it's about solving problems, combining the cutting edge of science with design, taste, judgment, and imagination to build things people can actually use. In this sense, this is not the end of engineering, but a return to its roots.

What's worth noting is the dramatic change in the pace of model iteration: OpenAI used to release a new model roughly every 15 months, but that cycle has now shrunk to about once every 6 weeks. Behind this acceleration lies a systemic upgrade of the entire large-model industry's "train-evaluate-deploy" pipeline: the scaling of H100/H200 GPU clusters has lowered the cost of a single training run, the level of automation in RLHF (Reinforcement Learning from Human Feedback) and RLAIF (Reinforcement Learning from AI Feedback) workflows continues to rise, and a culture of rapid iteration driven by "capability evaluation benchmarks" has gradually matured. It's worth mentioning that H100/H200 are data-center-grade GPUs designed by NVIDIA specifically for large-model training and inference; their NVLink high-speed interconnect technology allows thousands of cards to work in concert, compressing the cost of a single pretraining run from hundreds of millions of dollars to a level that can be repeatedly executed. RLAIF, by having AI models automatically generate preference-annotation data, effectively breaks through the speed bottleneck of human annotation, allowing alignment iteration to advance in step with model capability improvements. A 6-week cycle means model capability improvements can directly respond to developers' real feedback, forming a tighter product feedback loop. The GPT 5.6 series preview is a direct embodiment of this rhythm.

From Code Completion to Autonomously Achieving Goals

The leap in AI programming assistance capabilities has gone through several clear stages, each of which brought an experience powerful enough to change the way we work:

• Completion: the earliest form of assistance
• Inline Prediction: smarter continuation
• Command K: lets the model make edits directly, but without self-verification capability
• Build & Test Loop: the model begins to verify its own output
• Long-term goal mode: the model can continuously handle complex, long-horizon goals through to completion

This evolutionary trajectory is closely tied to the expansion of the underlying model architecture and context window. Early code completion (such as when GitHub Copilot launched in 2021) was based on GPT-3-derived models, relying mainly on local context for token-level prediction—essentially an extension of statistical language patterns. The emergence of the "Build & Test Loop" marked the evolution of the Agent from a "generator" into an "execute-verify" closed-loop system: the model can not only generate code but also invoke a code interpreter, run unit tests, read error output, and iteratively correct itself. This capability depends on the maturity of tool-calling (Function Calling/Tool Use) mechanisms, as well as the expansion of the context window from the initial 4K tokens to 200K+ to accommodate full-codebase context. The significance of the tool-calling mechanism is that it transforms the language model from a closed text-generation system into an execution agent capable of interacting with the external environment—the model can dynamically invoke external tools such as a code interpreter, search engine, or database query during reasoning, and incorporate the tools' returned results into the next step of reasoning. This "perceive-act-perceive" feedback loop is precisely the core distinction between an Agent architecture and an ordinary LLM call.

Romain shared a telling comparison: at a certain Dev Day, he built a mini-drone demo from scratch using the then-available o1 preview, but because the model couldn't run code or self-verify, he could only "pray" the demo would succeed—o1 preview used a "Chain-of-Thought Reasoning" architecture with stronger reasoning capability, but at the time it did not yet support invoking a code execution environment, which is precisely the technical root of the "could only pray" predicament. Chain-of-Thought Reasoning (CoT) is a technique formally named by Google researchers in 2022 that, by having the model explicitly output intermediate reasoning steps before giving a final answer, significantly improves performance on tasks requiring multi-step reasoning such as math, logic, and programming. The breakthrough of the o1 series lies in internalizing this process into the model's implicit "thinking" phase, allowing reasoning depth to adaptively scale with problem complexity—but at that point its integration with external tools was still incomplete. And the "Build & Test Loop" wasn't even part of the model's capabilities not long ago. Today, the Codex Agent can autonomously test code and control an entire camera and lighting system to complete a real-time demo.

The more fundamental shift is this: Codex and Agents can now perform any task you could do on your own computer—not just help you write code, but also handle the work before and after coding. The speakers emphasized the concept of the "Loop": connect the Agent to the work itself and the reasons behind it, and it can proactively initiate more tasks; connect it to the downstream review and deployment stages, and it can genuinely drive results to completion.

Codex App: A Chat-First AI Programming Collaboration Interface

Faced with the reality that "model capabilities have surpassed humans," OpenAI made its product direction clear: the goal is absolutely not to replace engineers with automation, but to maximally empower engineers.

Alexander distilled a plain insight from sci-fi and superhero movies—the ideal product form contains two modalities: Chat and Hands-on. He specifically defended Chat: "Some think Chat is dead, but I think Chat is severely underrated." What engineers need is a unified entry point where they can ask any question anytime, anywhere, while also having a powerful collaborative UI for when they need to inspect, guide, or intervene personally.

He used "collaborating with a team" as an analogy: most of the time, you're just communicating with the team while the team autonomously advances the work—you don't need to watch every execution unit; only occasionally, when you need to dig into the details, do you dive into the problem together. Based on this philosophy, OpenAI built the Codex App—a clean chat interface suitable for both programming tasks and anything else, with support for drilling down into any level of detail at any time.

Alexander's judgment about the limitations of CLI and IDE touches on the deep logic of Human-Computer Interaction (HCI) design: the CLI was born in the time-sharing computing systems of the 1960s-70s, and its interaction paradigm is essentially a linear "command-response" dialogue, lacking state persistence and a visual collaboration layer. The design philosophy of Vim and Emacs was deeply influenced by the Unix philosophy, while IDEs organize information around "code files"—the user must first open a file before starting work. The "Chat-first" paradigm subverts this starting point—driving the workflow by Intent rather than file path, which aligns closely with the "Task-centered Design" concept in product design. From an HCI research perspective, this shift echoes the "user-mental-model-centered" principle Don Norman put forward in The Design of Everyday Things: what the user has in mind is "I want to implement some feature," not "I want to open the src/utils/parser.ts file." The Chat interface, for the first time, aligns the way tools are organized with the natural flow of human thought, rather than requiring users to adapt to the tool's file-system metaphor. This also explains why even heavy Vim/Emacs users who "never leave the terminal" were eventually convinced: the two paradigms are not mutually exclusive but hierarchical—"chat first, dive deeper when needed."

An Open, Layered Architecture: Building the AI Engineering Ecosystem with Developers

OpenAI repeatedly emphasized that Codex cannot be a closed product that only OpenAI can improve. To that end, they designed Codex as a layered, open stack that anyone can build upon, and insisted on the "same system" principle—what OpenAI uses internally is exactly what it provides to external developers.

This open architecture includes:

• Model layer: opened via the Responses API; the Codex App itself uses the same model and API. Compared to the earlier Completions API, the Responses API adds native support for tool calling, built-in file search, code interpreter, and other capabilities, so developers don't need to implement complex Agent-loop infrastructure themselves. The Completions API's design assumes a stateless "one request, one response" interaction, requiring developers to manually manage conversation history and the injection and parsing of tool-call results. The Responses API, by contrast, internalizes this complexity at the API layer, providing stateful multi-turn interaction primitives—effectively pushing the foundational capabilities of an Agent framework down from the application layer to the platform layer, greatly lowering the engineering barrier to building reliable Agent systems. When Codex needs a new capability (such as context compaction for long tasks), it's built into the API first for developers to share.
• Codex Harness: fully open source—inspectable, forkable, adaptable. The model is the default configuration but not hard-coded; developers can swap in open-source models while keeping the same Agent loop.
• AGENTS.md: rather than inventing a new instruction format specifically for Codex, they chose a generic naming convention to facilitate interoperability with other Agents. This naming strategy is similar to robots.txt or .github/CONTRIBUTING.md in the web ecosystem, reducing ecosystem lock-in through convention rather than proprietary specification—sharing the same open philosophy as Anthropic's Model Context Protocol (MCP). MCP is an open protocol proposed by Anthropic in 2024, aimed at standardizing the interface definitions between AI models and external data sources and tools. Its core idea: rather than every company inventing its own "tool-calling format," an open protocol allows the entire ecosystem's tools to be reused across models and platforms. AGENTS.md's naming philosophy is cut from the same cloth.
• AppServer: open source, used to uniformly control the harness. The developer Thomas (demelian) even built a native app with it before the official Codex App was released, and later joined the team to lead Codex for iOS.
• Plugin system: capabilities such as browser use and computer use are all built as plugins, along with dedicated plugins for roles like data science and design—all open source.

Currently, mainstream IDEs including OpenCode, PyDroid, OpenClaw, and even Xcode and JetBrains already support users connecting via their existing subscriptions. The "same system" principle (dogfooding) is also an important endorsement of product credibility, avoiding the trust erosion of "the internal version being stronger than the external one." The speakers' core judgment is: every time a developer forks the harness and explores the boundaries of model capabilities, they help OpenAI learn and improve.

Value Maxing: From Cost and Speed to Parallel Collaboration

Once Agents became "explosively useful," the real question became how to systematically extract value from them—OpenAI calls this approach Value Maxing, not Token Maxing. "Token Maxing" refers to developers' tendency to pile on context and choose the most expensive models regardless of cost in pursuit of output quality. "Value Maxing" is a corrective to this behavioral pattern, essentially introducing the economic concept of "marginal utility" into AI engineering decisions—performing "Intelligent Routing" based on task complexity. In practice, intelligent routing means engineers need to establish a task-classification system: simple tasks like code formatting and comment completion are routed to lightweight models; tasks requiring cross-file understanding, architectural design, or complex debugging invoke the flagship models. This is highly similar to the "compute instance tiering" logic in cloud computing (such as AWS's t-series vs. c-series vs. p-series)—use the right resource, not the most expensive one.

In terms of cost efficiency, GPT 5.6 Terra can provide near-GPT-5.5-level intelligence at roughly half the cost; Luna beat several well-known models in benchmarks, yet is priced as low as $1 per million input tokens and $6 per million output tokens. Names like Terra, Luna, and Sol correspond to OpenAI's internal model-tiering system, where models of different scales form a hierarchy balancing intelligence level against inference cost, and engineers need to choose as appropriate.

In terms of speed, GPT 5.6 Sol running on Cerebras can reach 750 tokens per second of frontier intelligence output—equivalent to completing a fairly sizable PR in about 10 seconds. Cerebras, as a chip company optimized specifically for AI inference, uses a Wafer Scale Engine (WSE) architecture that makes an entire silicon wafer into a single chip, with on-chip SRAM capacity reaching hundreds of GB, allowing model weights to be stored entirely in on-chip memory—fundamentally eliminating the latency bottleneck caused by data movement between High Bandwidth Memory (HBM) and compute units in GPU clusters. Traditional GPU-cluster inference speed is limited by the Memory Bandwidth Wall, while the WSE architecture raises this bottleneck by an order of magnitude through extremely high on-chip memory bandwidth (tens of TB/s)—this is precisely the physical root of its several-fold advantage in inference latency over standard GPU clusters. But the real value of speed is not just "getting the answer faster," but enabling the Agent to try five or six different approaches in parallel before selecting the best one—the traditional approach of trading time for quality is replaced by trading parallelism for quality, and the collaboration model shifts from "waiting for the AI to respond" to "a colleague presenting results as they work."

In terms of parallelism and runtime environment, many developers dare not turn off their computers just to keep the Agent working. OpenAI believes the future should not have a divide between "local tasks" and "cloud tasks"—the user simply talks to the Agent, and it decides which environment to execute in. Codex Cloud, as the first major release, is also about to receive an important upgrade.

Manager of Agents: From "Polling Scheduling" to Managing an AI Team

The headline special guest was Peter Steinberger, known as "the claw father," who shared a strikingly impactful real-world transformation.

He used to operate more than a dozen terminal windows simultaneously, constantly waiting for one Agent to finish so he could pick up the next task. He joked at his own expense: "I thought I was orchestrating, but I was really just polling—I became the scheduler, the router, and the memory." In computer science, "Polling" is a resource-intensive task-scheduling pattern that decides the next action by continuously checking task status; the counterpart is the "Interrupt-driven" pattern—responding only when an event occurs. The trade-offs between these two patterns have accumulated decades of research in operating system design: polling's advantage is low-latency response (no need to wait for an interrupt signal to be delivered), while its drawback is continuous consumption of CPU cycles (corresponding to Peter's continuous attention drain); the interrupt-driven pattern's advantage is efficient resource use (the processor can execute other tasks while waiting), while its drawback is the overhead of interrupt handling itself. Peter's old way of working turned the human himself into an inefficient polling scheduler, wasting the most precious cognitive resource on status-checking rather than decision-making.

Today, his working model has completely transformed: he mainly talks to a long-running Manager Agent, which delegates tasks to a team of Agents; only when a thorny problem arises does he personally step in to pair on it. Three key capabilities made this transformation possible:

1. Server-side compaction: makes long tasks reliable enough that there's no longer a need to optimize around "new sessions." As conversation history grows, early information is gradually forgotten (constrained by the context window); Compaction preserves the most critical state information within a limited window by summarizing and compressing historical messages, similar to an operating system's memory page-replacement algorithm. From an information-theory perspective, Compaction is essentially a form of lossy compression: it prioritizes retaining high-information-density content (key decisions, current task state, important constraints) and discards low-density content (redundant reasoning processes, repeated context explanations), enabling a limited context window to support long-horizon tasks spanning hours or even days. This is precisely the key infrastructure that upgrades an Agent from a "conversation tool" into a "persistent work partner."
2. Coordination: allows a single Manager thread to create and correctly guide multiple sub-projects
3. Triggers: the key technology that allows automation to wake the same Manager when an event occurs, upgrading the way humans work from polling to interrupt-driven. The essence of the trigger mechanism is to connect events in the external world (code commits, CI failures, user feedback, scheduled tasks) with the Agent's work context, enabling the Agent to perceive and respond to real-world changes rather than passively waiting for the human's next command. This aligns perfectly with the design philosophy of modern Event-Driven Architecture (EDA): each component in the system responds only to the events it cares about, decoupling producers and consumers through an event bus, ultimately achieving autonomous, coordinated operation of the entire system.

Persistent context, delegation, triggers—the combination of the three constitutes a complete "Agent work loop." He also pointed out the constant migration of the bottleneck: from the initial token limits, to compute resource limits, and finally converging on the scarcest resource that cannot be simply scaled—attention. This is an engineering articulation of the "limited attention resources" theory in cognitive psychology—when compute resources and token costs are all scalable, the human engineer's cognitive bandwidth becomes the truly incompressible constraint variable, which is precisely the theoretical basis for the "outer-loop / inner-loop" division-of-labor model. Cognitive psychologist Kahneman's dual-system theory divides human thinking into System 1—"fast, automatic, intuitive"—and System 2—"slow, deliberate, analytical." In the context of Agent division of labor, engineers should retain System 2-style high-level decisions (setting direction, judging quality, making value trade-offs) and delegate System 1-style execution operations (code generation, test running, documentation updates) to the Agent—this is both the optimal allocation of cognitive resources and the most practical human-machine collaboration model given the current boundaries of model capabilities.

Peter's core point is thought-provoking: the new generation of models understand intent precisely enough that watching an Agent generate code line by line is "a bit of a waste of time." The real value lies in setting direction, making decisions in the outer loop, and handing the inner loop over to the Agent to execute. The ultimate form he envisions: the Manager should not be trapped in a single session or on a single laptop, but should become an Agent that "you can text anytime, guide from Slack, and reach anywhere."

He also candidly pointed out the current most core contradiction: the pace of model evolution has far outstripped the iteration pace of the harnesses and organizational structures built around them. This may well be the true question left for every engineer to answer in this "golden age of AI engineering."

Key Takeaways

Key Takeaways