How Engineers Can Avoid Becoming Mere Code Reviewers in the AI Era: Addy Osmani's Answer

At the AI Engineer conference, Google engineering lead Addy Osmani posed a question that strikes at the heart of every developer: when AI agents can produce code far faster than we can review it, what value does an engineer still have? His answer isn't panic — it's a redefinition of the engineer's role, from "someone who writes code" to "someone who owns outcomes."

What is an AI Agent? An AI Agent is an AI system capable of perceiving its environment, making decisions, and autonomously executing tasks. Unlike traditional single-turn large language models, agents have Tool Use, Multi-step Planning, and Memory Management capabilities. Typical implementations include OpenAI's Function Calling, Anthropic's Claude tool use, and orchestration frameworks like LangChain and AutoGen. The core value of an agent lies in upgrading a "single prompt" into a "continuous loop" — the system can self-check outputs, call external APIs, read/write the file system, and decide the next action based on results. This is the technical foundation for the evolution from "model as everything" to "software factory."

The Core of Future Engineering: Owning the "Verdict"

Osmani argues that future engineers will be defined by one ability: choosing what's worth doing. They must master the evidence, deeply understand the business, and render final verdicts on increasingly automated work.

The "verdict" here doesn't mean playing judge — it means owning production decisions: Should a change ship? Should it be blocked? Redirected? Or should the risk be accepted? He draws a sharp distinction between three concepts: quality produces evidence, verdicts assign responsibility, and answerability is what gives us the confidence to stand behind our verdicts.

Answerability and AI Governance Answerability is a core concept in AI governance. The EU AI Act, IEEE's AI ethics standards, and major tech companies' AI principles all list accountability as a fundamental requirement of responsible AI. Its technical implementation relies on Explainable AI (XAI) and AI Audit Trail capabilities — systems must be able to record "who made what decision, based on what rationale, with what outcome." By bringing this governance-level concept into individual engineering practice, Osmani is essentially saying: in an era where AI systems are massively involved in code production, every engineer needs to become an AI governance executor within their own domain — not outsource that responsibility to an algorithm.

This resonates with what many in the industry have observed. As Boris Cherny has noted, old craft boundaries are blurring and roles are reorganizing around the work itself. The key question is no longer "what's your title" but "what part of the system do you own."

The Technical Evolution from Single Model to "Software Factory"

Osmani traces the technical arc behind this transformation. We've long moved past the "model as everything" stage. Engineering practice is undergoing three key transitions:

Three Key Transitions

• Harness Engineering: Coding agent = model + the harness around it, including context, tools, and software. The harness is the key to turning "intelligence" into "something you can delegate to."
• Loop Engineering: No longer a single prompt, but designing systems that continuously prompt, check, remember, and decide the next step. Agents begin to operate like infrastructure.
• Software Factory: When all the pieces come together, agents run in their own private loops while engineers stand at a higher level to coordinate the whole.

One notable quality insight: clean codebases and messy ones have similar pass rates, but clean code consumes fewer tokens and causes less rework.

Token Economics and Code Maintainability The computational cost of large language models is measured in tokens. A token is the basic unit a model uses to process text — roughly 3/4 of an English word or 1–2 Chinese characters. Models like GPT-4 charge per input and output token count, and have a Context Window limit — the maximum number of tokens a model can process at once. The insight that clean code consumes fewer tokens has profound economic implications: in AI-assisted development workflows, code readability and modularity directly affect the context length an agent needs to understand a codebase, which in turn affects API call costs and task success rates. This transforms "clean code" from a software craftsmanship virtue into a quantifiable factory operational efficiency metric, providing a new economic incentive for long-neglected code quality practices.

In other words, maintainability now directly translates into "factory" operational efficiency.

The Dangerous State of "Distrust Without Bandwidth"

Osmani identifies a sharp contradiction: making "generation" cheap doesn't automatically make "review" cheap.

According to Sonar research, 96% of people don't fully trust AI-generated code, yet only about half always verify it before committing. This creates a dangerous state — "distrust without bandwidth." Safety must come from making verification cheaper, clearer, and harder to skip.

When the lens widens from individual reviewers to entire organizations, the problem is even more acute: governance can't keep up, with adoption speeds far outpacing companies' ability to form policies. Review and validation become the new bottleneck. We must answer a series of hard questions: Did the model actually touch this file? What constraints guided its work? What evidence was produced? Who accepted the risk? Who owns the final result?

Alpha and Decay: A Career Game About Capability Half-Lives

To explain career strategy, Osmani introduces two key terms:

• Alpha: The gap between what you can do today and what current models can do.
• Decay: The countdown on that gap. If your uniqueness is built on a particular capability, frontier models will eventually catch up.

Alpha's Financial Origins The "Alpha" concept Osmani borrows comes from the world of quantitative finance. In quantitative investing, Alpha refers to a portfolio's excess return above a benchmark — representing the value created by an investor's unique judgment and information advantage. Beta represents returns that simply track the broader market, the "follower's" gain. Translated to career development, an engineer's "Alpha" is the portion of their capabilities that exceeds the current AI baseline — derived from experience, intuition, domain knowledge, and judgment, rather than operational skills that can be replicated by models. "Decay" corresponds to time decay in finance, reminding practitioners that any static capability advantage has an expiration date and they must continually seek new sources of excess advantage.

He uses this lens to examine the popular concept of "taste." He agrees with Paul Graham — when anyone can create anything, "choosing what to make" becomes critically important. But he also warns: taste can become a "magic word" we hide behind when we don't want to explain our work.

Mitchell Hashimoto offers a more useful definition: taste is the ability to make high-quality qualitative judgments in the absence of objective metrics. This means taste runs ahead of benchmarks and market votes. But Osmani emphasizes that taste is itself an Alpha — it decays more slowly, yet it still gets reset as models learn from examples and preferences.

The real strategy, therefore, isn't to cling to any single capability, but to continually migrate your own capability boundary up one level.

Three Traps Engineers Must Avoid

Osmani outlines three risks to avoid — the most practically actionable part of the talk.

1. Cognitive Debt

This is the ongoing erosion of your understanding and memory of how problems have been solved. For code, it's the gap between the amount of code in the repository and the amount the team truly understands. You might have builds that pass tests and PRs that merge, yet the team gradually loses the ability to explain the systems they're shipping. "Delegation depth" becomes critical here.

Cognitive Debt and Technical Debt The concept of Technical Debt was introduced by Ward Cunningham in 1992, referring to the compromises made on code quality in pursuit of short-term delivery speed — compromises that must eventually be repaid at higher cost. "Cognitive Debt" is a deeper extension of this concept: it's not just a code-level quality problem, but a systemic erosion of a team's understanding. When AI generates code at scale, a team may maintain surface-level compliance in code review processes while steadily accumulating a deficit in genuine system comprehension. This is closely related to the software engineering concept of the "Bus Factor" — how many team members can be hit by a bus before the project is in crisis. What makes cognitive debt so insidious is that it can quietly drive the entire team's bus factor toward zero.

2. Cognitive Surrender

This is the state of blindly accepting AI answers. Delegation is "go do the work, then give me enough evidence to judge it"; surrender is "your answer is my answer before I've formed any view of my own." Wharton research is a sobering reminder: when AI gets it wrong, 73% of people still think they chose the wrong answer — and feel more confident about it. The failure mode isn't using AI — it's "borrowed confidence."

3. Orchestration Tax

Running more AI agents doesn't mean there's more of "you" available. Your cognitive bandwidth cannot be parallelized. Every loop you open creates more decisions that need to be routed, merged, validated, and integrated. The solution isn't fewer agents — it's designing your attention the way you'd design a system.

Orchestration Tax and the Economics of Attention The claim that cognitive bandwidth cannot be parallelized is rooted in the attention resource theory of cognitive psychology. Psychologist Daniel Kahneman divided human thinking into the fast, intuitive "System 1" and the slow, rational "System 2" — the latter being a finite and indivisible resource. In software engineering, this maps to "Context Switching Cost" — research shows engineers need about 23 minutes to fully recover deep focus after each interruption. Osmani's "Orchestration Tax" is the new form of this cost in the AI era: each parallel Agent loop generates decision points that require human routing and validation, and the total cognitive load of those decision points may exceed the cost of doing the task alone. The solution is analogous to microservices architecture design — separating concerns, managing concurrency through asynchronous notification rather than synchronous monitoring, and treating attention as a scarce resource to be allocated and scheduled.

High Agency and the "Agency Ladder"

Osmani reframes the much-discussed concept of "High Agency" in hiring. It's not "I do everything myself" (that doesn't scale), nor performative busyness — it's ownership with judgment: knowing when to delegate, when to check in, when to stop, and when to put your name on the outcome.

He proposes an "Agency Ladder": the bottom rung is noticing a problem and throwing it back at the system; above that comes execution, diagnosis, proposal, recommendation, and resolution; but the rarest level at the top is discernment — noticing a problem and then judging whether it's worth pursuing, with the answer sometimes being "drop it and move on."

The Inner Loop Is Capability; The Outer Loop Is Agency

This is the most central operational model of the entire talk:

• Inner Loop = Capability: The agent can investigate, implement, test, and report — handling the bulk of execution.
• Outer Loop = Agency: Deciding, verifying, approving, owning — this remains the essence of engineering.

The agent returns evidence — diffs, tests, logs, rationale, traces, screenshots — and then real engineering begins: judging whether the work is worth doing, verifying whether the evidence is sufficient, deciding to approve, redirect, or take responsibility for what ships. So the boundary isn't "humans looking at AI output" — it's evidence and accountability.

This leads to an operational rule of thumb: "Explain it or don't ship it." This doesn't require humans to type or read every line of code — but someone must understand the work well enough to defend it.

The Bottleneck Shifts from "Can We Build It" to "Should It Exist"

Osmani closes with some career math: a competitive advantage may have a half-life of just one model version — speed, recall, verification, even taste, all shift as the frontier moves. But the half-life of a "signature" is much longer — your reputation, your expertise, and the name standing behind what gets shipped.

He remains optimistic about the future. History has repeatedly shown: every time we've made software easier to write — high-level languages, frameworks, the cloud, low-code — the world doesn't need less software, it needs more. Suppressed demand gets released. AI agents will follow the same pattern. They won't eliminate engineering work; they'll move the bottleneck from "can we build this" to "should this thing exist."

"Build the factory, keep the lights on, own your verdict." That's the maxim Osmani leaves with every engineer.

Key Takeaways