AI Can Write Code but Can't Run Systems? A Founder Breaks Down the Real Bottlenecks of Ops Automation

AI coding tools have completely changed the pace of software development over the past two years. From Copilot-style autocomplete to multi-agent collaboration driven by Claude Code, the marginal cost of writing code is visibly plummeting. But when the topic shifts from "generating code" to "keeping systems running reliably," the narrative is entirely different. In this in-depth conversation between Grepr CEO Jed and investor Martin, the pair unusually clarifies two key distinctions: "syntax" versus "semantics," and "code distribution" versus "real-world distribution." AI's success in the code world does not necessarily translate to the ops world.

From Autocomplete to 20 PRs: The Real Productivity of AI Coding

Jed's team has only 10 people, yet they're building a complete observability data pipeline. He describes his recent workflow: using Windows + WSL + Claude Code, with three to four session windows open simultaneously, letting a main implementation agent and a "tech lead reviewer" agent review each other's PRs via the command line. They first produce a PRD and design doc, a human confirms the algorithm and division of labor item by item, and then subtasks are dispatched in parallel. He mentions a specific project — auto promotion (automatic promotion/columnar shredding) in a data lake, turning each field of semi-structured logs into an independently filterable column, involving complex mechanisms like metadata systems, commit coordination, and multi-writer consistency. Previously this would take a single person about six months; he completed it in roughly ten days, all production code rather than a demo.

This productivity gain is not evenly distributed. Jed's observation is straightforward: good engineers get better with AI, bad engineers do not. The reason is that the model cannot bear the complexity of distributed systems for you — task decomposition, context management, and subtask orchestration still depend on how much complexity your brain can hold. He specifically mentions a practical detail: use command-line sub-agents rather than built-in sub-agents, because the former maintain persistent sessions and can be reused across sessions, yielding higher context efficiency. At the same time, you must explicitly tell the model "don't spin up 20 agents and burn all the tokens for a simple investigation."

Claude Code is a command-line AI coding agent released by Anthropic. Unlike in-IDE autocomplete, it runs as a standalone process and can read and write files, execute commands, run tests, and autonomously plan multi-step tasks. Its key design is persistent sessions: each session has an independent context window and retains memory across multiple interactions, making it suitable for long-running subtasks. Developers can open multiple sessions simultaneously, letting different agents handle implementation, review, or research, and pass PR links and feedback to each other via the command line. This "multi-session + sub-agent" model can significantly improve context utilization efficiency, but it also requires users to actively manage token consumption and avoid launching too many agents for simple tasks.

Syntax Is Easy to Learn, Semantics Is Hard to Imitate: Why Ops Is Not a Simple Extension of the Code Problem

The most valuable theoretical framework in the conversation is the guest's distinction between "syntax" and "semantics." Syntax is the characters you type; semantics is what actually happens when the machine executes. The model's success at the syntax level is essentially because the training distribution of code and the solution space of code highly overlap — the code humans have written constitutes a manageable manifold. But the state space of a running system grows exponentially with each bit, quickly exceeding the number of atoms in the universe. This is precisely the fundamental reason chaos engineering has failed to land for years: the state space explodes too quickly.

More critically, much of the semantics cannot be derived from syntax and must be measured empirically. Jed gives the example of query optimization: the SQL generated by the model corresponds to code that "looks fine," but initially it did no optimization at all, repeatedly computing duplicate conjuncts. Only when performance benchmarks and real query datasets were placed in front of it did it propose hypotheses, verify them one by one, and fix them. And when asked "how much memory should be allocated to the system," the model has no clue — such questions can only be answered through experiments in the production environment.

Chaos Engineering aims to verify how a system behaves under abnormal conditions by proactively injecting faults (such as randomly killing processes or simulating network latency), thereby uncovering hidden risks in advance. Its theoretical foundation is precisely "state space explosion": the possible states of a distributed system grow exponentially with the number of nodes, message ordering, timing, and other factors, far beyond what can be exhaustively tested. In practice, chaos engineering is usually confined to controlled experiments, designing hypotheses for known failure modes rather than covering all states. This also explains why it is difficult for it to become a default day-to-day ops practice — experiment design itself depends on a deep understanding of system semantics, and semantics often cannot be derived from code syntax alone; it must be combined with real observations in the production environment.

Reverse Positioning: Why Most AI SREs Start from the Wrong End

Most AI SRE products on the market take the "rearview mirror" approach: after metrics spike or an alert fires, they trace back through the complex system to find the root cause. Jed believes this path doesn't work, because reverse root-cause tracing itself is extremely difficult in complex systems. Grepr's approach is the opposite: first assume the system has been running normally, then use large-scale clustering to find "never-before-seen patterns" in petabytes of logs per day — a completely new log template, a precursor to resource exhaustion — and then have a predictive model throw the anomaly signal to the AI, letting the AI read code, check history, and verify causal chain hypotheses.

He reveals several real cases: in internal systems and two customer environments, AI early warnings were ignored by humans several times, and each time they subsequently escalated into production incidents. This constitutes an interesting form of cross-validation — the effectiveness of warnings is proven in reverse by "things did go wrong after being ignored." Both parties reach an important consensus here: AI doesn't need to predict everything; it only needs to take over the debugging workflow of human ops engineers — reading error messages, following the trail, proposing hypotheses, and verifying hypotheses. This set of steps is finite and learnable.

AI SRE refers to using artificial intelligence to assist or replace part of the work of site reliability engineers (SREs), with typical scenarios including anomaly detection, root-cause analysis, and automated remediation. Most current products adopt an "after-alert retrospective" model: when monitoring metrics cross a threshold, AI searches historical logs, call chains, and change records for possible causes. The limitation of this approach is that root causes in complex systems often involve multiple layers of interaction, and the retrospective search space is huge and prone to false correlations. By contrast, starting from a "normal baseline" to detect never-before-seen patterns and then letting AI verify causal hypotheses is more in line with the debugging instincts of human engineers — first notice "something is off," then follow the trail, rather than guessing blindly in an alert storm.

Hand a Distributed Cluster to AI? The Answer Is More Conservative Than You Think

When asked "can AI run a distributed cluster," Jed's answer is layered and cautious: for tasks with clear boundaries and quantifiable benchmarks, such as query optimization, it is "very close" to deliverable; but for fully taking over distributed cluster operations, "not yet." The disagreement lies in the judgment of state space. Investor Martin cites lessons from the history of systems: from virtual memory to routing, humans have repeatedly abandoned clever algorithms in favor of over-provisioning resources, because real workloads are unpredictable; ops faces a world of heavy tails, anomaly-driven behavior, and power-law distributions, entirely different from the mild distribution of code.

Jed's optimism rests on another premise: if ops is viewed as "completing a causal chain search within a finite number of steps" rather than "predicting infinite states," the problem becomes manageable. He acknowledges this requires the model to autonomously plan experiments and actively acquire information, not merely reason passively. The two ultimately converge on a judgment: code problems are close to solved, ops problems will come much later, and security ops — because there are adversaries actively fighting back — will come even later.

Real-World Distribution Is AI's Next Wall

The conversation ends by returning to a broader framework: AI will continue to expand like wildfire in domains where the solution space distribution matches the training distribution — code, images, and text all fall into this category; but domains where the solution space is determined by the physical world will be much slower. Martin uses autonomous driving as an example: after the DARPA Grand Challenge, the outside world thought an inflection point was imminent, but it took about 20 years and $170 billion in investment, and unit economics are still worse than Uber's. This is not because the technology is bad, but because the economic equilibrium point is determined by the real-world distribution.

A thought-provoking business detail confirms this: Grepr pitches customers both "AI-driven reliability" and "save 90% on observability bills," and customers almost invariably buy the latter first. The value of code tools can be directly converted into engineering person-days, while the value of ops AI must be attached to an account like "avoiding one incident," which is hard to price — this may explain, more than the technology itself, why the commercialization pace of AI ops is far slower than that of AI coding.

Heavy-tailed and power-law distributions are common statistical features that describe real system workloads: most events are small, but the probability of extreme events is much higher than a normal distribution would predict. For example, request latency, failure intervals, and traffic peaks often exhibit heavy tails, meaning that "rare" extremely large values appear far more frequently than expected. System design under such distributions cannot rely on the average case; it must reserve redundancy for extreme cases. Martin cites the history of systems to point out that from virtual memory to routing, humans have repeatedly abandoned clever algorithms in favor of over-provisioning resources, precisely because real workloads are unpredictable and heavy-tailed. If AI is to gain a foothold in ops, it must understand and adapt to these distribution characteristics rather than assume a mild, modelable world.

One-line Summary

AI coding productivity gains are significant but hard to replicate in ops, because of fundamental differences between code distribution and real-world distribution.

Paragraph Summary

Based on a conversation between Grepr CEO Jed and investor Martin, this article explores the differentiated performance of AI in coding and ops. AI coding tools can deliver 10-20x productivity gains through multi-agent collaboration, but the gains are concentrated among excellent engineers, and the complexity of distributed systems still must be borne by the human brain. The core theoretical framework distinguishes "syntax" from "semantics": the overlap between code training distribution and solution space enables AI success, while the state space of a running system explodes exponentially with each bit, and semantics cannot be derived from syntax alone — they must be measured empirically. Most AI SREs start from post-alert root-cause tracing, which is the wrong direction; they should first detect "never-before-seen patterns" and then let AI verify causal chains. AI can take over debugging workflows with finite steps, but fully running a distributed cluster is still unrealistic; security ops is even harder because of active adversaries. Real-world distribution is constrained by economic equilibrium, and the long investment in autonomous driving is a cautionary tale.